This time recorded on Ubuntu. Did I mention 64bit already?
Do not miss my session at FOTB for a live demo.
Archive for the 'projects' Category
I am back home in Germany from my trip to San Francisco where I initially announced JITB at FITC. I did not expect it to get around that quickly and that a quick-and-dirty video filmed with my phone would get so much coverage.
I want to put a lot of things regarding JITB in perspective here as a followup to the various comments that I received via mail, reddit, etc.
Flash Replacement
JITB is not a Flash replacement. Imagine a F1 race car and a family van. The family van comes with air conditioning, a radio and a lot of comfort. For the F1 car you will probably have to wear a helmet and need an engineering team to get it running.
The important factor for me is that JITB will run very fast under certain conditions. A race car is also not a jeep. JITB will not support all features of the Flash Player. If you search for a Flash replacement you might want to take a look at lightspark.
Performance
Java is slow and the world is flat. That being said I will not talk about Java performance. JITB compiles to native Java code with a certain overhead. An ActionScript 3 closure is converted into an anonymous class for instance.
JITB converts ActionScript 3 bytecode to Java bytecode at runtime and performs various optimizations. This way we can leverage the speed of the JVM for ActionScript and get great results.
That being said there is also a problem of course. Startup time is not very good. JITB and its parent project Apparat are implemented in the Scala programming language and written for multi core architectures. The startup costs are high and JITB is only useful for long living applications. However compiling ActionScript to Java can be done AOT. This means you need to deal only with the normal JVM startup time. The Flash Player API is implemented in pure Java.
The compiler used for JITB is doing relatively naive optimizations at the moment. Namely constant folding, copy propagation, dead code elimination and strength reduction. An older proof of concept version I implemented a year ago featured also loop invariant code motion, inline expansion and tail-recursive optimizations. The new compiler framework is very powerful and I will add even more algorithms and a graph coloring register allocator for example. So there is some more NP-complete fun ahead making startup time even worse.
Purpose
We need a Flash Player that performs very fast for special purposes. We could use JITB at Audiotool to render mixdowns of tracks or other companies could use it to run their ActionScript code on the server side. You can use JITB to create an offline application that runs on a client machine. You can hack JITB to do what you want. Someone could potentially write a web framework for ActionScript and you could execute that code on Google App Engine if compiled AOT. And it is definitly possible to create Android applications using ActionScript. JITB’s terrain is everything but the browser in my opinion. However I can imagine that someone would be interested in creating a plugin version. With some clever simple caching algorithms it could make sense to get around startup costs but I am not interested in doing this.
Legal
There are no legal obligations to face since both the AVM2 and SWF specifications are published by Adobe. It is also not illegal to compile code for the JVM.
ActionScript Subset
By “a subset of the language” I really mean the language and not the API. JITB supports currently only statically typed ActionScript code. However the internal language model can be adjusted to support dynamic typing as well and with Java 7 things got even easier. In fact JITB does support dynamic code as long as it is able to infer types correct. Does this mean that I am willing to implement the full Flash Player API all by myself? Hell no.
In the next weeks I will probably create a better example that is easier to understand and get some other stuff ready so that you can try everything for yourself.
Good news everyone. The Apparat inline expansion works now to full extent after fixing some minor bugs. A complete example is also available. Just change the paths in the build.properties file and compile everything using Ant.
Use the inline feature with care. Apparat does not try to optimize your code and performs nothing but dead simple inlining. This can lead to slower code due to the creation of lots of local registers. Your code gets also much bigger and will require more space in memory. I am actually not a fan of manual inlining at all. I think it makes only sense to inline code if you have a powerful optimizer available that will cleanup the whole mess.
The fun story about this example is that the inlined version is slower using the lastes Flash Player release candidate if you have only 40.000 particles. That is why I increased the number of particles to 80.000 ;). I developed the example using an old standalone player and the inlined version was nearly twice as fast. However when I watched the example in the browser with the latest release candidate the game was completely different. Kudos to Adobe for significantly improving the Flash Player performance!
In addition to macro expansion Apparat has now inline expansion as well. It works nearly the same way as macro expansion but without most of its limitations. To define a class for inline usage it must extend apparat.inline.Inlined and all its methods must be static. However the cool thing is that you can also return values in contrast to macro expansion. You can also pass normal parameters. For instance FastMath.sin(FastMath.sqrt(2.0)) is valid code using inline expansion. It is enabled by default in TDSI.
Apparat is now available using the Scala 2.8-RC2 build. You can find appropriate downloads now on Google Code by searching for the Scala version of your choice. 7z compression on OS X should work now as well.
Apparat has another new feature called Macro Expansion. I talked about this with Nico Zimmermann at FFK in Cologne. Nico was using TDSI for a project but he was not very satisfied with it because you have to inline all inverse-square root tricks manually.
This is why Apparat has now macro expansion. I am actually not a big fan of it. I think a good compiler would do this for you without you having to go through all the steps. Unfortunately writing this compiler will take longer than the couple of hours I have spent on the macro expansion today.
So if you want to have quick and dirty inlining capabilities: this is for you. It is an easy fix for a feature a lot of people have asked for. I will continue working on TAAS to implement this much better in the future.
Apparat has a great new feature that allows you to create even smaller files. Basic SWF files are compressed using Java’s standard java.util.zip.Deflater class. This is perfectly fine. And I would even go with no compression at all during development.
However when deploying you can spend some extra time compressing your SWF files with the best tools available. 7-Zip is for instance such a tool and achieves a great compression ratio.
If you have the 7z executable on your PATH Apparat will make use of it to achieve the best compression for your SWF files. More configuration options are available here.
This feature is currently only implemented for SWF files and not SWCs. However every tool will make use of it by default. So if you are running Reducer and have no graphics in your SWF files you can still get a better compression.
An example is straight from the sources:
Note please that this SWF file does not contain any graphical assets and we got still a ratio of 18%. Our main audiotool.swf file is compressed by about 10% — about 200kb — and contains no graphical assets either.
You can download the latest Apparat on Google Code. Please note that you will need Scala 2.8 RC1 as well.
We have finally released the first version of the Audiotool. I am very thankful for all the kind tweets and happy that our work is now available to the public.
However the current version is named Firestarter for a reason. What you see online is just the tip of the iceberg. I can not wait to start implementing all the cool features that will separate us from traditional software ;)
I am a big fan of Scala and used it for a couple of internal tools here at Hobnox. What I really like about Scala is the fun factor and the programming speed. That is the reason why I decided to port the whole Apparat framework from Java to Scala.
This was not an easy decision since Apparat is a big project. However I have decided to rewrite Apparat because of the following reasons.
- Some parts of the framework are not well thought.
- TAAS would have been hard to optimize.
- This is my (n+1)-th iteration of implementing the ABC format. I think a lot of things can be simplified again.
- Scala is an ideal language for Apparat’s tasks.
The Scala source code is already available on Google Code.
Commits will happen early and often. The current revision is able to parse Swf and Swc files. I have also implemented most of the SWF tags which have been part of the Java framework.
I could compare the Scala and Java source every day. Everything is so concise now.
Take some time and think about this tweet for a moment. It took me a while to realize that Joel Hooks is right. I was embaressed of myself. How could I forget about that? But I had also a big smile on my face at the same same time. Let me explain why.
The Java to SWF compiler does not compile Java sourcecode but JVM bytecode to ActionScript bytecode. This means I do not have to teach my program the Java language. It only understands JVM bytecode. This seems like an akward decision on the one hand since working on the bytecode level implies lots of problems. But it turns out that this was a really cool decision on the other hand. “Java to SWF compiler” is maybe the wrong description. “any language that compiles to JVM bytecode to SWF compiler” is maybe better.
So what does any language mean? Here is a list of JVM languages. Now you feel maybe like I did after reading that tweet. And I am really looking forward to get Scala up and running.
Some problems still exist. Threading is one issue and I will basically have to do what Scott Peterson did for Alchemy. But reflections, annotations and method overloading have to be solved as well. Some glitches may exist even after figuring everything out. Stacktraces will look pretty weird. However I think this is a really cool project.
Update: I forgot to mention something important. Java supports native code. This means you can build a library that works with OpenGL for instance. Those native methods can not be converted. There are also some other things that do not work. File access is just one of them.
