Because the PixelBender Toolkit editor is very annoying I started writing my own some weeks ago. All I want to have is a more comfortable way to write shaders in PixelBender and I think I have achieved that already but once you start doing something you want to implement even more features until you are really satisfied.
I am really thankful that Arne from FDT was answering a lot of my questions regarding the Eclipse framework. That is also why I could implement so many features already in a short amount of time.
The current features include:
- Syntax highlightning
- Matching brackets highlightning which handles also < and > for metadata correct
- Smart auto-indent strategy
- Context sensitive auto-completion
- Auto-insert of closing brace etc.
- Documentation for built-in methods on hover
There are two really important issues that I still would like to address. One is the ability to highlight stuff a little bit semantic. Defined constants for instance should look different in code. And I could also add very basic live error highlightning. An outline would be nice and some content-assist features like “Create parameter …” etc.
The only downside is currently that you can not have a live-preview of your shader in Eclipse. Maybe there will be a way of doing this. I am currently not sure. Please tell me what you think and leave feature requests in the comments. I can not garuantee for anything but I will try implementing as much as possible in a reasonable timeframe.
Because the PixelBender Toolkit editor is very annoying I started writing my own some weeks ago. All I want to have is a more comfortable way to write shaders in PixelBender and I think I have achieved that already but once you start doing something you want to implement even more features until you are really satisfied.
I am really thankful that Arne from FDT was answering a lot of my questions regarding the Eclipse framework. That is also why I could implement so many features already in a short amount of time.
The current features include:
Syntax highlightning
Matching brackets highlightning which handles also < and > for metadata correct
Smart auto-indent strategy
Context sensitive auto-completion
Auto-insert of closing brace etc.
Documentation for built-in methods on hover
There are two really important issues that I still would like to address. One is the ability to highlight stuff a little bit semantic. Defined constants for instance should look different in code.
During the last weeks I have thought about a completly new version of the image processing library. I have done lots of scribbling about what kind of features I want to have and which would be really cool.
But there is still a big problem when it comes to dealing with image data in Flash. I have currently different strategies with their pro’s and con’s but I can not decide for myself which one makes the most sense.
-
BitmapData
When you are using only one BitmapData to represent an image you have of course access to all the native methods. Life will be a lot easier. But there are certain key issues with the BitmapData. The precision for each channel is limited to 8bit and the main problem is that you will always have to deal with pre-multiplied alpha. If I want to have a high-quality image processing library this is unacceptable.
-
Vector.<uint>
A vector filled with unsigned integers makes sense because this is what you get from a BitmapData and what you can use to set the contents of a BitmapData but working all the times with one integer value and having to do the bit-hassle is really annoying.
The main problem with a Vector of unsigned integers is simply that PixelBender does only accept a Vector of Number values with the length of width * height * 4. So everytime you want to use PixelBender you would have to convert your vector. And also remember that you have only a precision of 8bit per channel. Instead of using a Vector.<uint> I could also use a BitmapData — the only thing I gain from the vector is that I do not have to deal with pre-multiplied alpha.
-
Vector.<Number>
A vector of normalized Number values makes a lot of sense. You are able to use PixelBender with this kind of Vector without having to convert anything. The only problem occurrs when you want to convert your vector into a BitmapData — but this problem can be solved pretty easy with a PixelBender identity shader. This is also what I used to convert a BitmapData into a vector of Number values since BitmapData.getVector returns only a vector of unsigned integers.
Now all of this does not sound that bad. But there are a lot of problems actually. First of all every time you want to get/set a pixel you have to access four fields of a vector which is way slower than accessing only one field. And the main problem I have is the poor PixelBender support. First of all every time I start a ShaderJob with a vector of fixed length as the target I get an exception that the ShaderJob.start() wants to change the length of my vector. What the fuck? If my Vector has a fixed length of width * height * 4 I do not see why the ShaderJob should change this at all.
Another problem is PixelBender itself. In order to rebuild a method like BitmapData.fillRect() you have to visit every channel for every pixel if the rectangle is the full image size. Using ActionScript for this task is out of the question because it will be way to slow. So I thought I write a fillRect-Shader. No problem. The shader works very well in the PixelBender Toolkit but I get only garbage in the Flash Player. Now I am really uncertain if I should continue with this approach since even simple tasks fail. If my target is a BitmapData it works very well by the way. So if I want to fill a rectangle with PixelBender and I have only a Vector.<Number> I have to create a BitmapData with equal size to use it as the target with the ShaderJob and then use an identity shader to convert the BitmapData back into a Vector.<Number>. And once we are using a BitmapData we have again lost the precision and we have pre-multiplied alpha. This would not be a problem if PixelBender would work with Vector.<Number> as good as with a BitmapData.
-
Vector.<RGBA> and single-linked list
Having a Vector.<RGBA> in combination with a single linked list is by far the best what you can get in terms of speed. We use this in our audio engine as well. You need to access every field only once and you can iterate over the single-linked list very fast. There are only two major issues. In order to use PixelBender you have to convert this structure which has the length width * height into a Vector.<Number> of the length width * height * 4 and you will have the same problems as described above. If you want to have a BitmapData representation you will have to convert the Vector manually to a BitmapData which will also cost you a lot.
My main problem is that I do not like any of those possible approaches. In the old ImageProcessing library I used a BitmapData for each channel but I would not do that again. The main problem is that Adobe has released a very buggy version of the PixelBender run-time in Flash. I would criticize also that we have so many different formats for pixel data which do not fit together. Sometimes you need a Vector.<uint>, sometimes you need a Vector.<Number> — sometimes the endian is flipped (Adobe Alchemy) and working with the BitmapData only — which sounds reasonable — is a pain because you will never get around pre-multiplied alpha. I would love to have a primitive RGBA data type for image processing which I could feed into PixelBender (color4) and which I could get and set to a BitmapData as a Vector.<RGBA>. If those RGBA elements would be also a single-linked list — even better.
During the last weeks I have thought about a completly new version of the image processing library. I have done lots of scribbling about what kind of features I want to have and which would be really cool.
But there is still a big problem when it comes to dealing with image data in Flash. I have currently different strategies with their pro's and con's but I can not decide for myself which one makes the most sense.
BitmapData
When you are using only one BitmapData to represent an image you have of course access to all the native methods. Life will be a lot easier. But there are certain key issues with the BitmapData. The precision for each channel is limited to 8bit and the main problem is that you will always have to deal with pre-multiplied alpha. If I want to have a high-quality image processing library this is unacceptable.
Vector.<uint>
A vector filled with unsigned integers makes sense because this is what you get from a BitmapData and what you can use to set the contents of a Bitma
Today I submitted my first patch to the FlexSDK after becomming a contributor. Lets see how this all will work in the future. It feels like I am working for free and fixing bugs that Adobe should work on but in several cases it is so much easier to just do the stuff and fix it yourself in 5min instead of writing 2h emails. Although I am not so happy about it.
Getting the Flex SDK to work is pretty easy if you manage several hurdles. It is quite cool to debug the ActionScript compiler while compiling your project.
The next thing I want to fix is the Bindable meta tag. [Bindable] in ActionScript projects is a pain if you do not compile with the flex.swc library and there is no reason for the compiler to embed the overhead of creating event dispatchers all over the place.
We are here already using “our” new compiler. Again it will be quite interesting to see how all this works out with Adobe and how they will judge about feature submissions.
Today I submitted my first patch to the FlexSDK after becomming a contributor. Lets see how this all will work in the future. It feels like I am working for free and fixing bugs that Adobe should work on but in several cases it is so much easier to just do the stuff and fix it yourself in 5min instead of writing 2h emails. Although I am not so happy about it.
Getting the Flex SDK to work is pretty easy if you manage several hurdles. It is quite cool to debug the ActionScript compiler while compiling your project.
The next thing I want to fix is the Bindable meta tag. [Bindable] in ActionScript projects is a pain if you do not compile with the flex.swc library and there is no reason for the compiler to embed the overhead of creating event dispatchers all over the place.
We are here already using "our" new compiler. Again it will be quite interesting to see how all this works out with Adobe and how they will judge about feature submissions.