A Pixel/Fragment Shader Example in Flash

This is super cool. Now that Flash gives you access to the graphics card you can write pixel shaders!

“uh ok… what’s a pixel shader? why would I want to write one?”

Because shaders can make stuff move! And in games moving stuff is way better than not moving stuff. Like this grass:

That is a kind of shitty animated gif of my pretty amazing moving grass. Like normal grass but with 100% more pixel shader magic. I’ve hacked up a quick shader sandbox here and with a little work you can see a live example of the grass there.

Pixel Shaders are programs you write in assembly and upload to the graphics card. Then whenever you UV map a texture to a triangle the program decides how the mapping is done. If you want to know more about UV mapping you can check out my UV mapping tutorial.

This post is an example of what you can do with pixel shaders in Flash’s Stage3d. If you want to replicate this then you need to first have something running in Stage3d. I recommend using Starling, a great 2d Stage3d platform. It takes a little work to get Starling working with custom meshes and pixel shaders but it makes everything else much easier. Also flash calls pixel shaders fragment shaders for some reason but that’s dumb so I’m going to call them pixel shaders.

Ok, I’m going to pick up right where the UV mapping tutorial left off. We have this mesh:

With this texture UV mapped onto it:

Now presumably, if you have something similar being displayed on your screen, you are already using a pixel and vertex shader. Without one you would see nothing. You probably have a really simple one though.

Here are Starling’s simple shaders slightly modified for this example:

//vertex shader
“m44 op, va0, vc0 // 4×4 matrix transform to output clipspace
“mov v0, va1 // pass color to fragment program
“mov v1, va2 // pass texture coordinates to fragment program

//pixel/fragment shader
“tex ft1, v1, fs0 <2d,linear,repeat,mipnearest> // sample texture 0
“mul ft2, ft1, v0 // multiply color with texel color
“mul oc, ft2, fc0 // multiply color with alpha

They’re simple but they’re written in AGAL. Which is Flash’s shader language. It’s basically assembly language you upload to the graphics card. But since flash just invented AGAL you’ve probably never seen it before.  I’ll explain how some of AGAL works as I go but if you’re really interestied in writing shaders Marco Scabia has already written a great primer on AGAL right here.

We can almost ignore the vertex shader above. All we care about is that it tells our pixel shader what pixel of the texture to grab. It does that by passing the coordinate in the variable v1. Which is a register shared by the two programs.

The pixel shader then copies the pixel at texture coordinate v1  into temporary register ft1 (tex ft1, v1, fs0).

Then it does some bullshit we don’t care about but ends up copying the pixel into the output register oc. If you wanted to you could write the worlds simplest pixel shader. It would looks like this:

tex oc, v1, fs0 <2d,linear,repeat,mipnearest> // sample texture 0

Which just copies the colour of the pixel at position v1 in texture fs0 to the output register 0c.

This gets called for every single pixel that gets displayed so we can do per-pixel manipulation and per-pixel animation. And if we can then we should.

To get our grass animatin’ we have to do some work both in Flash and in the pixel shader.

Lets start with the flash code since we need it to pass some constants into the pixel shader. Here is a pared-down example of rendering some triangles with a bunch of important stuff cut out.

AGAL is a little crazy about constants. Instead of defining them in your assembly you have to pass them to the graphics card when you render out the triangles. So if you want to do something like i = i + 1 you have to pass in the value 1 as a constant. That’s what most of the following lines are. They’re just constants we use in the shader program.

It’s also worth noting that in AGAL every register (variable) is a 4-dimensional vector. Or a four length array or however you want to think about it. These four components tend to represent x, y, z, w when you’re dealing with space and r, g, b, a when you’re dealing with colour.

So, lets pass in those constants.

context.setProgram(grassShader);

context.setProgramConstantsFromVector(
  Context3DProgramType.FRAGMENT, 1, Vector.<Number>([1, 1, 1, 1])); //fc1
context.setProgramConstantsFromVector(
  Context3DProgramType.FRAGMENT, 2, Vector.<Number>([3, 3, 3, 3])); //fc2
context.setProgramConstantsFromVector(
  Context3DProgramType.FRAGMENT, 3, Vector.<Number>([Math.sin(_grassCount), 0, 0, 0])); //fc3
context.setProgramConstantsFromVector(
  Context3DProgramType.FRAGMENT, 4, Vector.<Number>([.3, 0, 0, 0])); //fc4

context.drawTriangles(mIndexBuffer, 0, mTriangles.length/3);

_grassCount = (_grassCount + .015) % (Math.PI * 2);

Really, the only interesting thing here is _grassCount. We want the grass to wave back and forth so we need something to oscilate over several frames. Hey sin oscilates! So we keep a class variable that moves between 0 and 2*PI and pass the sin of that into the shader program. That variable will move back and forth between -1 and 1 so we can offset the grass pixels by that amount and it will wave back and forth.

Now it’s time to write some assembly! Woo! Writing assembly is just like writing normal code except it takes forever and in the end not even you can read your code. Comment the shit out of it.

Most of the grass code is preoccupied with finding the offset between what pixel we would normally display and the pixel we want to display in its stead. Think of the temporary register ft3 as a variable named offset that we build up over four lines. Here it is, 7 lines of glory. I’ll got over each line below:

sub ft3, v1.y, fc1.y //offset = 1-y
pow ft3, ft3, fc2 //offset = offset^3
mul ft3, ft3.y, fc3 //offset = sin(count)*offset
mul ft3, ft3, fc4 //offset *= .3
add ft2, v1, ft3 //texturePos.x += offset
tex ft1, ft2, fs0 <2d,linear,repeat,mipnearest> //pixel = texture(texturePos)
mov oc, ft1 //return(pixel)

You can see this shader in action over in my shader sandbox.

Remember, this is being called on every pixel but we want the pixels near the roots to stand still and the pixels at the very top to wave back and forth the most. The easiest way to do that is to multiply stuff by the height but in our UV mapping the top is at position 0 and the bottom is at 1 so first we reverse that:

sub ft3, v1.y, fc1.y //offset = 1-y

We don’t want a linear difference between the top and the bottom or the grass wouldn’t “bend”. So we pow that shit:

pow ft3, ft3, fc2 //offset = offset^3

Now we scale the offset by the number that is oscillating between -1 and 1:

mul ft3, ft3.y, fc3 //offset = sin(_grassCount)*offset

But that results in swaying that is over exaggerated so we tone it down:

mul ft3, ft3, fc4 //offset *= .3

It’s worth noting that fc4 is [.3, 0, 0, 0] which zeroes out any extraneous y values that might be hanging around. We only want to change the x.

Done! Well actually this doesn’t reproduce the results above. With this shader the grass all sways back and forth in time instead of following an invisible “wave”. But I’ll leave the rest as an exercise for the reader.

Wow, good thing it worked the first time eh? Debugging this stuff must be a mess since you can’t use a debugger on the graphics card or even print out variables.

Well next time I’ll give you a hand with that debugging thing!