Archive for the 'Flash' Category

Page 2 of 22

Apparat And Maven

Two days ago I made a major change to the Apparat repository. I completely restructured the layout and fully integrated Apparat with Maven. We have now a plugin and archetypes for easy tooling. I also synchronized Apparat with the Maven central repository two weeks ago.

You might ask what the hell this is about. So in the current state you can download Apparat from Google Code. Then you have to have a working Scala installation which has been used to build Apparat. Whenever Scala is updated, I have to updated Apparat, and you will have to download Apparat again and install a new Scala version. This is absolutely cumbersome. This will change of course when Scala 2.8 is officially released.

I know that some people will always require command line access for Apparat. I am sorry for you at the moment that we have to play this game until 2.8 is out.

For the rest this is good news. If you have a working Maven 3.x version you can use the Apparat plugin and get automatic updates. This means you have to configure your project only once and do not bother with Scala and Apparat anymore. In fact if you want to you could also use the Apparat snapshot repository and get live updates.

Project configuration and setup is something one should not have to deal with. Therefore you can use the archetypes. I have built one for TurboDieselSportInjection.

mvn archetype:generate \
  -DarchetypeRepository=http://oss.sonatype.org/content/repositories/snapshots \
  -DarchetypeGroupId=com.googlecode.apparat \
  -DarchetypeArtifactId=apparat-archetype-tdsi \
  -DarchetypeVersion=1.0-SNAPSHOT

This terminal command is all you need to do to create a new project that is ready to compile — with TurboDieselSportInjection enabled. And it will automatically use the latest Apparat version.

New Apparat Example

Good news everyone. The Apparat inline expansion works now to full extent after fixing some minor bugs. A complete example is also available. Just change the paths in the build.properties file and compile everything using Ant.

Apparat Example

Use the inline feature with care. Apparat does not try to optimize your code and performs nothing but dead simple inlining. This can lead to slower code due to the creation of lots of local registers. Your code gets also much bigger and will require more space in memory. I am actually not a fan of manual inlining at all. I think it makes only sense to inline code if you have a powerful optimizer available that will cleanup the whole mess.

The fun story about this example is that the inlined version is slower using the lastes Flash Player release candidate if you have only 40.000 particles. That is why I increased the number of particles to 80.000 ;). I developed the example using an old standalone player and the inlined version was nearly twice as fast. However when I watched the example in the browser with the latest release candidate the game was completely different. Kudos to Adobe for significantly improving the Flash Player performance!

Inline Expansion

In addition to macro expansion Apparat has now inline expansion as well. It works nearly the same way as macro expansion but without most of its limitations. To define a class for inline usage it must extend apparat.inline.Inlined and all its methods must be static. However the cool thing is that you can also return values in contrast to macro expansion. You can also pass normal parameters. For instance FastMath.sin(FastMath.sqrt(2.0)) is valid code using inline expansion. It is enabled by default in TDSI.

Apparat For Scala 2.8 RC2

Apparat is now available using the Scala 2.8-RC2 build. You can find appropriate downloads now on Google Code by searching for the Scala version of your choice. 7z compression on OS X should work now as well.

Macro Expansion

Apparat has another new feature called Macro Expansion. I talked about this with Nico Zimmermann at FFK in Cologne. Nico was using TDSI for a project but he was not very satisfied with it because you have to inline all inverse-square root tricks manually.
This is why Apparat has now macro expansion. I am actually not a big fan of it. I think a good compiler would do this for you without you having to go through all the steps. Unfortunately writing this compiler will take longer than the couple of hours I have spent on the macro expansion today.

So if you want to have quick and dirty inlining capabilities: this is for you. It is an easy fix for a feature a lot of people have asked for. I will continue working on TAAS to implement this much better in the future.

Apparat: Crunching SWF Files Since 2009

Apparat has a great new feature that allows you to create even smaller files. Basic SWF files are compressed using Java’s standard java.util.zip.Deflater class. This is perfectly fine. And I would even go with no compression at all during development.

However when deploying you can spend some extra time compressing your SWF files with the best tools available. 7-Zip is for instance such a tool and achieves a great compression ratio.

If you have the 7z executable on your PATH Apparat will make use of it to achieve the best compression for your SWF files. More configuration options are available here.

This feature is currently only implemented for SWF files and not SWCs. However every tool will make use of it by default. So if you are running Reducer and have no graphics in your SWF files you can still get a better compression.

An example is straight from the sources:

reducer -i as3\Apparat.Tests.AS3\bin\Test07.swf
[i] Apparat -- http://apparat.googlecode.com/
[i] Launching tool: Reducer
[i] Waiting for 7z ...
[i] Compression ratio: 18.224573%
[i] Total bytes: 310
[i] Completed in 547ms.

Note please that this SWF file does not contain any graphical assets and we got still a ratio of 18%. Our main audiotool.swf file is compressed by about 10% — about 200kb — and contains no graphical assets either.

You can download the latest Apparat on Google Code. Please note that you will need Scala 2.8 RC1 as well.

Audiotool 1.0

We have finally released the first version of the Audiotool. I am very thankful for all the kind tweets and happy that our work is now available to the public.

However the current version is named Firestarter for a reason. What you see online is just the tip of the iceberg. I can not wait to start implementing all the cool features that will separate us from traditional software ;)

Compiling ActionScript In The Enterprise

If you ask me if ActionScript is ready for the enterprise then this question is hard to answer. Every fairly huge codebase requires some maintenance. You will have to figure out a strategy to identify modules and separate them. You will also have to handle the dependencies between modules. Last but not least you have to compile those.

An important aspect is that when I am talking about modules I am not talking about “Flex Modules” but the more generic term for a part of a project.

First of all you have to make sure to fulfill some requirements. You can not assume that every developer is working on the same platform using the same IDE with the same settings. But you must assume that your project will not be built from an IDE. You must also assure that the dependencies between projects are up to date across different platforms and IDEs. Once you have figured out how to do that you might want to think about a strategy to compile your modules.

Today I want to explore the idea of parallel compilation across a cluster of machines to yield optimal compile speed for ActionScript projects. First of all, you can not assume that a developer can always keep all modules up to date. Depending on the dependency graph of your projects compilation migh take very long. For instance we have 103 modules at Hobnox. This means we have 103 different compile targets with different dependencies, each yielding its own SWF/SWC file. We also have a “standard library” that is included in nearly every project. Changing this library will require the developer to recompile all 103 modules to update them.

To illustrate this example I want to explain what we went through.

  1. We used FDT to develop our project. Since FDT is Eclipse based every module was represented as a separate project with dependencies to other projects. FDT however compiles your project only once you launch it. And it does not do this automatically for your dependency graph. As a result you have to manually compile 103 modules in the correct order. This would not be the problem if FDT’s live error highlightning and code analysis would not work on a SWC basis. But since the actual source-code between projects is not analyzed we had to launch all 103 modules to propagate a possible error or change. This was for us the point when we had to make a switch.
  2. Ant was our second idea to build our FDT projects automatically. However we had to keep module dependencies up to date in all the different Ant files because the FCSH requires that you include external libraries of a library also in a successor of another library. This means if A requires B and C requires A. Then C requires also B. So we had to keep 103 Ant targets up to date for each change.
  3. A custom build. We started working with a custom build tool I have developed in Scala. This build tool was smart enough to figure out which modules have to be recompiled without launching the ActionScript compiler. It also figured out which projects could be compiled in parallel since the ActionScript compiler is written in a way that IO is not the only bottleneck. We had an enormous improvement in compile time and usability for the developer since module dependencies where handled automatically.
  4. We switched from FDT to IntelliJ IDEA since it offered us more features that we needed. IntelliJ came also with a much better strategy to compile modules. Basically it had the feature we always missed for FDT to automatically include libraries and to compile modules in the correct order automatically. However the guys at JetBrains made a mistake by counting on FCSH. The short story is that FCSH is absolutely not capable of handling a large codebase. What happened to us was that when we started a build FCSH would take up as much memory as it could, getting much slower until it finally crashed. Then FCSH started from scratch, consumed the maximum amount of memory again and crashed. This happend until a project was finally compiled but took also more than 15 minutes for a single change we made. This was of course unacceptable. Fortunately JetBrains implemented a feature request from us into IntelliJ IDEA which comes close to our custom build. From now on projects are compiled in parallel without using FCSH and without asking launching the compiler if no change has been made. This way we are able to work again. However only if we compile just a short amount of all modules since the main bottleneck is still the compiler.
  5. A continuous integration server was our last option. This is a custom build server and we are using it until today. After each SVN commit we are building the required modules and their dependencies now automatically on a dedicated server. A change in the standard library takes about four minutes to complete now on a quad core. But the good thing is that we have set our main project to exclude a lot of libraries. The custom build takes care of that and we are loading now modules from the server. That way we can still compile local and include libraries we frequently change. Other libraries are loaded from the server and included at runtime.

So the question is if this is now the ultimate solution. A custom continuous integration server that takes more than 4 minutes to propagate a change after a SVN commit? I do not think so. And here comes Scala into the game again. There is a project available called Swarm. It makes use of Scala’s support for serializable delimited continuations. This means you can pause the state of your current program, send it to a different node in your cluster and continue with the program on that machine. And Swarm makes this task extremely simple. So for now a new research project is to take advantage of this feature. We can basically have one master server like Hudson or TeamCity and write a plugin for it that makes use of Scala Swarm. Swarm could be based on nodes in the office, normal computers and maybe a couple of dedicated servers to compile modules in parallel on multiple nodes. Each node would have to keep the VCS up to date. That way we should be able to compile projects much quicker and have a much better workflow in a large codebase. Besides that if you work with a CI server than you could also have the task of code analysis using FlexPMD for instance done on another machine as well.

I am really looking forward to this project when I have some time to work on it. My goal is to develop a plug-in for TeamCity that we will use internally first. I expect to reduce the time for a full rebuild from about 4min to something like 30sec. This would be a huge benefit. I also hope that we will be able to make that plug-in available to the public.

Polyglott Programming On The AVM2

Take some time and think about this tweet for a moment. It took me a while to realize that Joel Hooks is right. I was embaressed of myself. How could I forget about that? But I had also a big smile on my face at the same same time. Let me explain why.

The Java to SWF compiler does not compile Java sourcecode but JVM bytecode to ActionScript bytecode. This means I do not have to teach my program the Java language. It only understands JVM bytecode. This seems like an akward decision on the one hand since working on the bytecode level implies lots of problems. But it turns out that this was a really cool decision on the other hand. “Java to SWF compiler” is maybe the wrong description. “any language that compiles to JVM bytecode to SWF compiler” is maybe better.

So what does any language mean? Here is a list of JVM languages. Now you feel maybe like I did after reading that tweet. And I am really looking forward to get Scala up and running.

Some problems still exist. Threading is one issue and I will basically have to do what Scott Peterson did for Alchemy. But reflections, annotations and method overloading have to be solved as well. Some glitches may exist even after figuring everything out. Stacktraces will look pretty weird. However I think this is a really cool project.

Update: I forgot to mention something important. Java supports native code. This means you can build a library that works with OpenGL for instance. Those native methods can not be converted. There are also some other things that do not work. File access is just one of them.

Getting Rid Of null

A couple of weeks ago I started learning Scala. I can highly recommend it. The language has a lot of great approaches to multithreading and scalability. The reason why I like Scala is because it is so simple yet powerful.

Of course doing something at home influences work. I decided to write a build tool for the Hobnox AudioTool which is about 200 lines of Scala code. Cool thing is that this tool replaces a manually maintained Ant build and all project dependencies are always correct. Plus analyzing the dependencies in a more powerful way allows me to spawn the compiler in parallel. Building the AudioTool is now twice as fast and much more comfortable.

When learning a new programming language you also learn about new concepts. Functional languages in general have a different approach to nullable types. I know Scala is not the only one but let me introduce the concept in terms of ActionScript.

When you have a method that returns either a result or nothing: what do you do? Imagine you have some kind of service and a Dictionary of users. Requesting a user works by his unique id. The Dictionary is private to the class since you want to keep it read only.

function getUser(id: String): User {
  return hasUser(id) ? users[id] : null;
}

If I would now simply ask the service for an unknown user and do something like if(getUser('xyz').isLoggedIn) { trace('Hooray'); } I could probably and up with a null-reference error. No one checks for me if the user exists. So what else could we do? Write a lot of boilerplate code and prepend a check if the user is null or not each time we request one from the service. A much better approach in my opinion is to throw an error as early as possible. In this case we would rewrite the method to something like this:

function getUser(id: String): User {
  if(hasUser(id)) return users[id];
  throw new NoSuchUserError(id);
}

In this scenario we get informed about the error as early as possible. But we are stuck again. First of all ActionScript does not enforce you to catch possible exceptions. This means if you do not read the documentation of a method carefully or look into the source code of a method before calling it somewhere you will never know that it throws an exception. And what if we are actually in a scenario where we do not expect errors for non-existing objects? Think of the Dictionary object throwing each time an error when you access it and the result is null. How could I even check if an object exists in a Dictionary?

try {
  dictionary[key];
  return true;
}
catch(noSuchElementError: NoSuchElementError) {
  return false;
}

I guess you see that this can not be the solution to our problem. In a real world example you may deal with your own collection of objects instead of a Dictionary of course. So we have to get rid of exceptions and null for the optimal solution. Scala’s approach to this problem is the Option type. We always abuse null as a placeholder when we want to express that an element does not exist. The Option means that either Some or None result exists. Rewriting our getUser function using this approach would yield the following ActionScript code.

function getUser(id: String): Option {
  return hasUser(id) ? new Some(user[id]) : new None();
}

Why is this much better than the old approach? When calling the method you will always know that the method has only an optional result value. We get rid of the exception and null values. Our only problem at the moment is ActionScript. The result is now untyped. In an ideal world this method would be written as:

function getUser(id: String): Option.<User> {
  return hasUser(id) ? new Some.<User>(user[id]) : new None.<User>();
}

However we can still tackle this issue by implementing null-representations of our objects. Imagine the User class. You could rewrite the code to something like this.

function getUser(id: String): IUser {
  return hasUser(id) ? user[id] : new NullUser();
}

final class NullUser implements IUser {
  public function get isLoggedIn(): Boolean { return false; }
  public function get name(): String { return 'null'; }
}

And even if you are interested in null-reference errors you could rewrite your code to something like this:

final class NullUser implements IUser {
  public function get isLoggedIn(): Boolean {
    CONFIG::ThrowNullReferenceErrors { throw new NullReferenceError(); }
    return false;
  }
  public function get name(): String {
    CONFIG:: ThrowNullReferenceErrors { throw new NullReferenceError(); }
    return 'null';
  }
}

It is definitely a very different approach. A functional language like Scala allows you to deal much better with Options. But it makes sense to diferentiate between an uninitialized variable which is null and an optional result of a function. Unfortunately this is at the moment very painful with the lack of generics in ActionScript.