Published on
Aug 19, 2010 in
Flash, as3, experiments and personal.
Tags: actionscript, apparat, fitc, flashplayer, java, jitb, opengl, san francisco.
My talk at FITC San Francisco is over and I want to share some of the anouncements from today with you. At the end of my talk I was showing JITB.
What you see in the YouTube video I posted a while ago is a Java program executing a SWF. For FITC I added some more code and an OpenGL based Display List renderer. In other words: I wrote a Flash Player.
However I should rephrase that statement and say I am attempting to build a Flash Player. The current state is available in the sf2010-sprint clone of Apparat. I will merge the changes into the main Apparat branch when I am back home in Germany.
JITB is currently able to translate a subset of ActionScript code at runtime into Java bytecode and runs nearly at the same speed as native Java. This is a really huge improvement compared to standard ActionScript performance. A lot of smart people worked on the JVM and made it really fast. Apparat will allow you to leverage all this hard work in the future. I am also shooting for Java interoperability at some level so that you can call Java classes from within ActionScript. Hopefully you will be able to use JITB on your desktop machine, on a server or on an Android phone. Basically everywhere Java runs.
There are still a lot of things missing. The whole Flash API needs to be implemented. And the Display List rendering needs a proper OpenGL implementation. However I thought this might be some cool stuff to share with you in its early stages.
My hope is that more people start contributing to the project. Maybe some OpenGL guru wants to take care of the Display List rendering or someone else likes to help implement the Flash API in Java.
I also showed a Raytracer by Nico Zimmermann during my presentation and promised to put the URL on my blog so here it is. His company is called Britzpetermann and the address is http://www.britzpetermann.com/.
Update: Please do not think that this implementation is 30x faster than the Flash Player developed by Adobe. One(!) microbenchmark is never a number you should count on. I would like to make clear that I never said this.
My talk at FITC San Francisco is over and I want to share some of the anouncements from today with you. At the end of my talk I was showing JITB.
What you see in the YouTube video I posted a while ago is a Java program executing a SWF. For FITC I added some more code and an OpenGL based Display List renderer. In other words: I wrote a Flash Player.
However I should rephrase that statement and say I am attempting to build a Flash Player. The current state is available in the sf2010-sprint clone of Apparat. I will merge the changes into the main Apparat branch when I am back home in Germany.
JITB is currently able to translate a subset of ActionScript code at runtime into Java bytecode and runs nearly at the same speed as native Java. This is a really huge improvement compared to standard ActionScript performance. A lot of smart people worked on the JVM and made it really fast. Apparat will allow you to leverage all this hard work in the future. I am also shooting for Java interoperability at some le
Since I started the development of some other library I completly put AS3V aside because the Eclipse integration with FDT is currently not possible. But since AS3V is working already I decided to release a version really quick that allows you to use AS3V as an Ant Task or simply from command line.
I know it would be better to have it as an Eclipse Plug-in with nice little markers etc. but for that I need some more extension points in FDT. I do not know about FlexBuilder but it would be probably the same.
The file includes an example build and some test cases that show you how AS3V can detect common errors and mistakes. There is currently not that much documentation available for the rules and parameters you can specify. Putting a list online with all rules and nice descriptions is of course something I would like to do but currently the time for that is missing. I hope everything is self-explaining if you have a look at the readme and example files.
Update: The AS3V package has been updated with one bug fix and some modifications so that AS3V will produce a more interesting log output if it fails for some reason. Using AS3V with Eclipse works only if you specify fork="true" like in the Flex SDK example from the build file that comes with the zip.
Since I started the development of some other library I completly put AS3V aside because the Eclipse integration with FDT is currently not possible. But since AS3V is working already I decided to release a version really quick that allows you to use AS3V as an Ant Task or simply from command line.
I know it would be better to have it as an Eclipse Plug-in with nice little markers etc. but for that I need some more extension points in FDT. I do not know about FlexBuilder but it would be probably the same.
The file includes an example build and some test cases that show you how AS3V can detect common errors and mistakes. There is currently not that much documentation available for the rules and parameters you can specify. Putting a list online with all rules and nice descriptions is of course something I would like to do but currently the time for that is missing. I hope everything is self-explaining if you have a look at the readme and example files.
Download AS3V
Update: The AS3V package has been
When you are implementing a pathfinding algorithm you have usually some sort of defined map available. This is in most cases a tile map or a graph of nodes. If you have no nodes and no tiles you have two options: build a graph of nodes or put your smooth environment into a raster with a defined detail. Option two is usually faster but we have choosen number one for asthetic pleasure in the AudioTool. And we showed that building such a graph at runtime in Flash is something you can do if you implement clever optimizations.
You will always have problems without some kind of raster. Sometimes more, sometimes less. Usually you have to build your own context of how things are connected and how they behave in your environment. We were discussing a lot about the audio engine and the core structure behind. The most important part of the audio engine is probably the ability to build a linear chain of plugins which process signals that arrive in your speakers at the end of that chain. When we started this part we had a lot of optimizations in mind. Plugins that do not generate any audible output should not go into that chain etc. The code became quite complex and results in a lot of special case scenarios. Imagine a scope which is a pure visual element that shows the incomming waveform. This is a plugin which should not be kicked out of any chain but generates no audible output.
You can imagine that we used a graph for that kind of task. To be more specific: a directed cyclic graph since there is a direction of the flow and we are allowing feedbacks for audio signals. After some time of discussion I started researching for a better way of solving our signal chain. Because I beleive that a problem can be solved the easiest way on a pure mathematical basis I started with a plain graph — but this time a real graph. After some time I could produce the same output as our old solver but with less lines of code and we got rid of all the special cases.
Now having a real graph is fun and I was not surprised that lots of people invested their brainpower in graph algorithms. For the desktop in the AudioTool I always wanted to implement some kind of automatic layout algorithm. My first idea was to build a spring between two editors connected by a cable. I thought that this could be too complex but after spending the last days with graphs this became a possible solution and there are already algorithms which do the same thing: build springs for connections and let the system solve itself. I did some tests in a sandbox already but hopefully I can implement this in the AudioTool before FITC Amsterdam. The graph you see in the picture is a result of the graph layout engine.
But working with graphs opened my eyes even more. I am currently the poor bugger that has to implement a layout engine. Implementing a layout engine is by the way the most ungrateful task I could think of. 
Anyways, what if you think about the dependencies in your layout engine as a graph structure? You have usually three cases when calculating a size: relative known size to “some kind of available size”, absolute known size, absolute unknown size. Those cases can be reduced to their most simple form on a piece of paper and a graph structure evolves. You can get rid of observers and listeners in such a system if a connection in the graph has the meaning of invoking the layout engine again on a subset of nodes. Those subsets could be optimized even more if you figure out the strongly connected components in the graph. I have to admit that I did not implement this system yet but it makes a lot of sense in theory currently… something which makes working on a layout engine interesting again ;)
When you are implementing a pathfinding algorithm you have usually some sort of defined map available. This is in most cases a tile map or a graph of nodes. If you have no nodes and no tiles you have two options: build a graph of nodes or put your smooth environment into a raster with a defined detail. Option two is usually faster but we have choosen number one for asthetic pleasure in the AudioTool. And we showed that building such a graph at runtime in Flash is something you can do if you implement clever optimizations.
You will always have problems without some kind of raster. Sometimes more, sometimes less. Usually you have to build your own context of how things are connected and how they behave in your environment. We were discussing a lot about the audio engine and the core structure behind. The most important part of the audio engine is probably the ability to build a linear chain of plugins which process signals that arrive in your speakers at the end of that chain. When we started this part we h
During the last weeks I have thought about a completly new version of the image processing library. I have done lots of scribbling about what kind of features I want to have and which would be really cool.
But there is still a big problem when it comes to dealing with image data in Flash. I have currently different strategies with their pro’s and con’s but I can not decide for myself which one makes the most sense.
-
BitmapData
When you are using only one BitmapData to represent an image you have of course access to all the native methods. Life will be a lot easier. But there are certain key issues with the BitmapData. The precision for each channel is limited to 8bit and the main problem is that you will always have to deal with pre-multiplied alpha. If I want to have a high-quality image processing library this is unacceptable.
-
Vector.<uint>
A vector filled with unsigned integers makes sense because this is what you get from a BitmapData and what you can use to set the contents of a BitmapData but working all the times with one integer value and having to do the bit-hassle is really annoying.
The main problem with a Vector of unsigned integers is simply that PixelBender does only accept a Vector of Number values with the length of width * height * 4. So everytime you want to use PixelBender you would have to convert your vector. And also remember that you have only a precision of 8bit per channel. Instead of using a Vector.<uint> I could also use a BitmapData — the only thing I gain from the vector is that I do not have to deal with pre-multiplied alpha.
-
Vector.<Number>
A vector of normalized Number values makes a lot of sense. You are able to use PixelBender with this kind of Vector without having to convert anything. The only problem occurrs when you want to convert your vector into a BitmapData — but this problem can be solved pretty easy with a PixelBender identity shader. This is also what I used to convert a BitmapData into a vector of Number values since BitmapData.getVector returns only a vector of unsigned integers.
Now all of this does not sound that bad. But there are a lot of problems actually. First of all every time you want to get/set a pixel you have to access four fields of a vector which is way slower than accessing only one field. And the main problem I have is the poor PixelBender support. First of all every time I start a ShaderJob with a vector of fixed length as the target I get an exception that the ShaderJob.start() wants to change the length of my vector. What the fuck? If my Vector has a fixed length of width * height * 4 I do not see why the ShaderJob should change this at all.
Another problem is PixelBender itself. In order to rebuild a method like BitmapData.fillRect() you have to visit every channel for every pixel if the rectangle is the full image size. Using ActionScript for this task is out of the question because it will be way to slow. So I thought I write a fillRect-Shader. No problem. The shader works very well in the PixelBender Toolkit but I get only garbage in the Flash Player. Now I am really uncertain if I should continue with this approach since even simple tasks fail. If my target is a BitmapData it works very well by the way. So if I want to fill a rectangle with PixelBender and I have only a Vector.<Number> I have to create a BitmapData with equal size to use it as the target with the ShaderJob and then use an identity shader to convert the BitmapData back into a Vector.<Number>. And once we are using a BitmapData we have again lost the precision and we have pre-multiplied alpha. This would not be a problem if PixelBender would work with Vector.<Number> as good as with a BitmapData.
-
Vector.<RGBA> and single-linked list
Having a Vector.<RGBA> in combination with a single linked list is by far the best what you can get in terms of speed. We use this in our audio engine as well. You need to access every field only once and you can iterate over the single-linked list very fast. There are only two major issues. In order to use PixelBender you have to convert this structure which has the length width * height into a Vector.<Number> of the length width * height * 4 and you will have the same problems as described above. If you want to have a BitmapData representation you will have to convert the Vector manually to a BitmapData which will also cost you a lot.
My main problem is that I do not like any of those possible approaches. In the old ImageProcessing library I used a BitmapData for each channel but I would not do that again. The main problem is that Adobe has released a very buggy version of the PixelBender run-time in Flash. I would criticize also that we have so many different formats for pixel data which do not fit together. Sometimes you need a Vector.<uint>, sometimes you need a Vector.<Number> — sometimes the endian is flipped (Adobe Alchemy) and working with the BitmapData only — which sounds reasonable — is a pain because you will never get around pre-multiplied alpha. I would love to have a primitive RGBA data type for image processing which I could feed into PixelBender (color4) and which I could get and set to a BitmapData as a Vector.<RGBA>. If those RGBA elements would be also a single-linked list — even better.
During the last weeks I have thought about a completly new version of the image processing library. I have done lots of scribbling about what kind of features I want to have and which would be really cool.
But there is still a big problem when it comes to dealing with image data in Flash. I have currently different strategies with their pro's and con's but I can not decide for myself which one makes the most sense.
BitmapData
When you are using only one BitmapData to represent an image you have of course access to all the native methods. Life will be a lot easier. But there are certain key issues with the BitmapData. The precision for each channel is limited to 8bit and the main problem is that you will always have to deal with pre-multiplied alpha. If I want to have a high-quality image processing library this is unacceptable.
Vector.<uint>
A vector filled with unsigned integers makes sense because this is what you get from a BitmapData and what you can use to set the contents of a Bitma
I wrote a while ago about our tween engine at Hobnox but I did not post any performance demos.
Here are the results for 1000 DisplayObjects with manipulation on their scaleX, scaleY, alpha, x, y and rotation properties. It is also very important that you take a look at the memory behaviour. A click starts the tweening.
As you can see it makes a lot of sense to stay type-safe and to manage the memory you are using on your own.
I will talk about those concepts in my next session AudioTool’s Private Parts in Brighton and Boston. Since optimization on a code level is trivial the main performance boost is achieved by re-thinking algorithms, structures and concepts. I guess one of the most interesting topics will be the optimization of our cable solver which was running O(n^2) (really!), then O(n(n+1)/2) and could be minimized to a rare O(n(n-4)/2) worst-case.
I wrote a while ago about our tween engine at Hobnox but I did not post any performance demos.
Here are the results for 1000 DisplayObjects with manipulation on their scaleX, scaleY, alpha, x, y and rotation properties. It is also very important that you take a look at the memory behaviour. A click starts the tweening.
Hobnox
GTween
TweenLite
As you can see it makes a lot of sense to stay type-safe and to manage the memory you are using on your own.
I will talk about those concepts in my next session AudioTool's Private Parts in Brighton and Boston. Since optimization on a code level is trivial the main performance boost is achieved by re-thinking algorithms, structures and concepts. I guess one of the most interesting topics will be the optimization of our cable solver which was running O(n^2) (really!), then O(n(n+1)/2) and could be minimized to a rare O(n(n-4)/2) worst-case.
