I have demoed support for PixelBender shaders in JITB at the FOTB 2010 conference. It turned out that JITB can run PixelBender code really fast and well. I have demoed a Julia and Terrain raytracer which were both kindly contributed by David Lenaerts. I only showed the demos since I would have been running out of time. I still owe you an explanation of how it all works.
Continue reading ‘JITB’s PixelBender support’
Tag Archive for 'pixelbender'
Since PixelBender is becomming a more popular technology in the Flash ecosystem I decided to add first-class Apparat support for it.
As you might know I have done a couple of different tools for PixelBender already.
When I switched to Linux a while ago I had to discover that there is no compiler for PixelBender available and also no PixelBender toolkit. Even PBDT does not work on Linux since it requires a compiler.
In the current state Apparat can read and write PBJ files. I already implemented the format two years ago for the outline view plug-in. But this time Nicolas Cannasse’s great work on the hxformat library was my reference.
I will add two optimizations specific for PBJ files. TAAS will not be used here since it is only an unnecessary overhead. You do not have to crack a nut with a sledgehammer. The PBJ format is already register based and has certain useful invariants.
To cleanup some mistakes of the PixelBender compiler I will add:
- Copy propagation
- Dead Code Elimination
The compiler is not bad. Compared to the ASC it is a beauty of technology that makes use of the LLVM. However I think there must be some mistake in the code since unnecessary registers are used (Note: I doubt this is LLVMs fault but more the way how methods like sampleNearest are bound by Adobe). This can be solved with the simple optimizations I will add.
I will add some more things as well like a GLSL conversion or a DSL to write PixelBender kernels in Scala. If we get lucky this will also lead to a true cross platform opensource compiler for the PixelBender language. I think I will name that compiler Hydra :P
PBDT has now support for semantic highlightning. Thanks to the FDT guys again for answering a lot of Eclipse releated questions. Semantic highlightning works currently with parameters, input and output declarations and for defined preprocessor directives. Just have a look at the screenshot. I have also integrated complete color preferences support. If you go to Preferences > PBDT > Colors you can change everything the way you like.
I just released my PixelBender plug-in for Eclipse. It integrates for us very charming into the workflow and is a good help when developing PixelBender kernels. I have put up a tutorial and instructions at this page.
Features of the PBDT plug-in include code completion, syntax highlightning and some other nifty features. It can be integrated into existing projects in FDT or FlexBuilder. Let me know what you think and please report any bugs or issues back to me.
A special thanks goes to Arne Deutsch from the FDT team for helping me a lot and answering all my annoying questions.
As a response to Ralph Hauwert’s article I created a little example of what can be achieved using plain ActionScript 3 syntax. Ralph has put up a great example of how you can wire things like Alchemy, ActionScript and PixelBender together to achieve an astonishing result.
However I asked myself if it is possible to achieve the same result without making use of Alchemy, PixelBender or bytecode manipulation. I asked the guys sitting with me in the office to compare the results and they were unfortunately very different on various machines. Sometimes my version is faster, sometimes the version from Ralph is faster and sometimes they are about the same.
Now there are some very important things to note here and I am surprised that I got so close. Since Ralph is making use of PixelBender the number crunching is done on multiple cores. Something that is not possible with the ActionScript version which is the real bottleneck. And there is another big difference. Ralph’s calculations are done in 32bit while I am using 64bit precision. Therefore I am happy with the result and it shows that using pure ActionScript is still a good choice.
In order to optimize the code I used a linked list for the particles and minimized the comparisons between different data types. Here is the result.
Sources:
Because the PixelBender Toolkit editor is very annoying I started writing my own some weeks ago. All I want to have is a more comfortable way to write shaders in PixelBender and I think I have achieved that already but once you start doing something you want to implement even more features until you are really satisfied.
I am really thankful that Arne from FDT was answering a lot of my questions regarding the Eclipse framework. That is also why I could implement so many features already in a short amount of time.
The current features include:
- Syntax highlightning
- Matching brackets highlightning which handles also < and > for metadata correct
- Smart auto-indent strategy
- Context sensitive auto-completion
- Auto-insert of closing brace etc.
- Documentation for built-in methods on hover
There are two really important issues that I still would like to address. One is the ability to highlight stuff a little bit semantic. Defined constants for instance should look different in code. And I could also add very basic live error highlightning. An outline would be nice and some content-assist features like “Create parameter …” etc.
The only downside is currently that you can not have a live-preview of your shader in Eclipse. Maybe there will be a way of doing this. I am currently not sure. Please tell me what you think and leave feature requests in the comments. I can not garuantee for anything but I will try implementing as much as possible in a reasonable timeframe.
During the last weeks I have thought about a completly new version of the image processing library. I have done lots of scribbling about what kind of features I want to have and which would be really cool.
But there is still a big problem when it comes to dealing with image data in Flash. I have currently different strategies with their pro’s and con’s but I can not decide for myself which one makes the most sense.
-
BitmapData
When you are using only one BitmapData to represent an image you have of course access to all the native methods. Life will be a lot easier. But there are certain key issues with the BitmapData. The precision for each channel is limited to 8bit and the main problem is that you will always have to deal with pre-multiplied alpha. If I want to have a high-quality image processing library this is unacceptable.
-
Vector.<uint>
A vector filled with unsigned integers makes sense because this is what you get from a BitmapData and what you can use to set the contents of a BitmapData but working all the times with one integer value and having to do the bit-hassle is really annoying.
The main problem with a Vector of unsigned integers is simply that PixelBender does only accept a Vector of Number values with the length ofwidth * height * 4. So everytime you want to use PixelBender you would have to convert your vector. And also remember that you have only a precision of 8bit per channel. Instead of using a Vector.<uint> I could also use a BitmapData — the only thing I gain from the vector is that I do not have to deal with pre-multiplied alpha. -
Vector.<Number>
A vector of normalized Number values makes a lot of sense. You are able to use PixelBender with this kind of Vector without having to convert anything. The only problem occurrs when you want to convert your vector into a BitmapData — but this problem can be solved pretty easy with a PixelBender identity shader. This is also what I used to convert a BitmapData into a vector of Number values since
BitmapData.getVectorreturns only a vector of unsigned integers.Now all of this does not sound that bad. But there are a lot of problems actually. First of all every time you want to get/set a pixel you have to access four fields of a vector which is way slower than accessing only one field. And the main problem I have is the poor PixelBender support. First of all every time I start a ShaderJob with a vector of fixed length as the target I get an exception that the
ShaderJob.start()wants to change the length of my vector. What the fuck? If my Vector has a fixed length ofwidth * height * 4I do not see why the ShaderJob should change this at all.Another problem is PixelBender itself. In order to rebuild a method like
BitmapData.fillRect()you have to visit every channel for every pixel if the rectangle is the full image size. Using ActionScript for this task is out of the question because it will be way to slow. So I thought I write a fillRect-Shader. No problem. The shader works very well in the PixelBender Toolkit but I get only garbage in the Flash Player. Now I am really uncertain if I should continue with this approach since even simple tasks fail. If my target is a BitmapData it works very well by the way. So if I want to fill a rectangle with PixelBender and I have only a Vector.<Number> I have to create a BitmapData with equal size to use it as the target with the ShaderJob and then use an identity shader to convert the BitmapData back into a Vector.<Number>. And once we are using a BitmapData we have again lost the precision and we have pre-multiplied alpha. This would not be a problem if PixelBender would work with Vector.<Number> as good as with a BitmapData. -
Vector.<RGBA> and single-linked list
Having a Vector.<RGBA> in combination with a single linked list is by far the best what you can get in terms of speed. We use this in our audio engine as well. You need to access every field only once and you can iterate over the single-linked list very fast. There are only two major issues. In order to use PixelBender you have to convert this structure which has the length
width * heightinto a Vector.<Number> of the lengthwidth * height * 4and you will have the same problems as described above. If you want to have a BitmapData representation you will have to convert the Vector manually to a BitmapData which will also cost you a lot.
My main problem is that I do not like any of those possible approaches. In the old ImageProcessing library I used a BitmapData for each channel but I would not do that again. The main problem is that Adobe has released a very buggy version of the PixelBender run-time in Flash. I would criticize also that we have so many different formats for pixel data which do not fit together. Sometimes you need a Vector.<uint>, sometimes you need a Vector.<Number> — sometimes the endian is flipped (Adobe Alchemy) and working with the BitmapData only — which sounds reasonable — is a pain because you will never get around pre-multiplied alpha. I would love to have a primitive RGBA data type for image processing which I could feed into PixelBender (color4) and which I could get and set to a BitmapData as a Vector.<RGBA>. If those RGBA elements would be also a single-linked list — even better.
Ralph Hauwert put a very interesting blog post up no his blog about Adobe Alchemy. Basically he was reading my mind with this post.
I want to a step further now. But please keep in mind that I am definitly not a compiler expert or have deep knowledge of the Flash Player. Everything I write about is in my opinion common sense and comes from a long time of reverse engineering and autodidactic learning.
Continue reading ‘Alchemy, ActionScript and the ASC’
PixelBender Outline is a simple view for Eclipse. It allows you to browse through compiled PixelBender kernel files (pbj). Usually it is quite annoying when working with PixelBender. You always have to debug the shader if you did not write it yourself to know about parameters and stuff. The PixelBender Outline view allows you to navigate through that information in a comfortable way.
PixelBender Outline is working with FDT only. Just grab the JAR file and place it into your Eclipse plugins folder (…/Eclipse/plugins/) to install it. After restarting Eclipse you can open the view using Window->Show View->Other->PixelBender->PixelBender Outline.
The outline is updated everytime you select a *.pbj file in the Flash Explorer.
Tinic posted today his PixelBender assembler and disassembler. This makes me happy, because now I can post an experiment I could not show for a while.
If you know PixelBender, than you know that you can not create loops. What you could do is unroll all constant loops with a fixed length. If you know simple convolution filters like a blur, you know that you need an xy-loop and you know it should not be possible with PixelBender at all. Let me prove you wrong and have a look at dynamic loop unrolling with PixelBender (be careful with high values!).
I built a library to assemble and disassemble PixelBender kernels at runtime. I wrapped it also in a high level API so basically you can create a new Kernel by doing var kernel: Kernel = new Kernel();. Then when you need your shader as a ByteArray you simple call kernel.compile(). There are still some glitches here and there but I hope that I can release the source code pretty soon.