Tag Archive for 'apparat'

Page 2 of 3

Apparat RC6: Say Hello To An Old Friend

Patrick Le Clec’h is an active committer to the Apparat project and recently we just merged his work into the main branch. With a couple of other changes this is now a good time for another release candidate. Patrick added a good old friend to Apparat: The __asm function. You might remember __asm from my work on the now deprecated AS3C project.

However the Apparat version is much better. First of all we have put a lot of work into Apparat to make such transformations rock-solid. AS3C had its issues and was never a reliable tool. But there are a lot of new great features Patrick implemented. You can mix AS3 in your bytecode as well. __as3 is the best friend of __asm. Because sometimes writing pure bytecode is very verbose and not necessary.

A simple trace('Hello World!') with pure bytecode would look like this. Please note the FindPropStrict and CallPropVoid operations which reference trace.

__asm(
  FindPropStrict(AbcQName('trace', AbcNamespace(NamespaceKind.PACKAGE, ''))),
  PushString('Hello World!'),
  CallPropVoid(AbcQName('trace', AbcNamespace(NamespaceKind.PACKAGE, '')), 1)
);

Finding the object in the correct namespace is often a very cumbersome task. Thanks to __as3 we can also write this in a much more conciese way.

__asm(
  FindPropStrict(__as3(trace)),
  PushString('Hello World!'),
  CallPropVoid(__as3(trace), 1)
);

Note that the ASM compiler will try to guess the required name once it is requested by an operation. You can use __as3 also for other tasks.

var x: int = 1;
__asm(
  FindPropStrict(__as3(trace)),
  __as3(x < 10),
  CallPropVoid(__as3(trace), 1)
);

This would trace "true" for instance. If you are curious about the ASM syntax I can recommend you using the dump tool. It produces code which is nearly __asm-ready. We will probably write another output so you can directly transform existing code to __asm calls.

If you are interested in some more examples the Apparat Math replacements make use of __asm now as well. IntMath is a good example for an inlined class where you are using maybe a simple method like IntMath.abs and the heavy lifting is done behind the scenes using inline assembler. To use the ASM expansion you have to process your SWF file with TDSI. It is by default turned on.

LZMA compression in Apparat RC5

Matryoshka avec moustache

I have released Apparat RC5 at GoogleCode. It contains a really cool feature which is called LZMA compression.

Reducer has advanced a lot during the last couple of weeks. It is now also a strong SWF compression tool even if you do not have any PNG files it can compress. You may ask: “What is that Matryoshka doing there? And why the hell the top hat?” The top hat: I do not know. The Matryoshka: I can explain.

The Flash Player does not understand LZMA. SWF files are compressed using good old DEFLATE. So what happens? Apparat extracts your original SWF. It compresses it again using the LZMA algorithm. The compressed SWF is injected into another SWF that contains an LZMA decoder. Size, background color, frame rate, etc. get adjusted. Finally you get a new SWF that contains your old SWF and a decoder to extract it at runtime. The overhead of the decoder is currently at around 5kb and I hope I can get it even smaller.

When you open that SWF with the Flash Player it will extract your original file and load it. Another nice feature is that I created different versions of the runtime decoder. One is using a classic preloader which is great if your SWF is a little bit bigger. And hey: it is a preloader for free so you do not have to deal with the [Frame] hassle. But here is the catch. We at audiotool.com always write our SWF files in the same style and I can just hope you do the same or use the great InitInjector by Valentin Simonov.

Your main SWF class or the so called DocumentClass must make sure that a stage is available before accessing it. This is really easy:

public function Main() {
  addEventListener(Event.ADDED_TO_STAGE, init)
  if(null != stage) init()
}

private function init(event: Event = null): void {
  removeEventListener(Event.ADDED_TO_STAGE, init)
  // your original constructor code goes here ...
}

Stick to that rule or InitInjector can automate it for you. Otherwise you will get a runtime exception that the stage is null. So this is one new feature. The other one is actually pretty standard. If you compile and link against a SWC file all classes will come in their own ABC file. Each ABC file has a constant pool. So if you link against 1000 classes you get 1000 constant pools. Reducer can merge all those files into a single one with some minor exceptions. It can also sort the constant pool so that constants which are used frequently consume less bytes when accessed. The funk-as3 test runner which links against FlexUnit and the Flex framework is only loosing 3.01% of its weight with the old Reducer. The new version reduces it by 17.81% already thanks to the ABC merging. Combine that with some LZMA love and we get 35.34%.

Besides you can call the LZMA compression as often as you want for some basic obfuscation. The LZMA compression alone is already a (weak) obfuscation of your bytecode.

Last but not least: Good news everyone! Scala 2.8 arrived so this is the last time you will have to update it for a while.

You can download the latest Apparat version at GoogleCode. Scala 2.8.0 is required.

Apparat And Maven

Two days ago I made a major change to the Apparat repository. I completely restructured the layout and fully integrated Apparat with Maven. We have now a plugin and archetypes for easy tooling. I also synchronized Apparat with the Maven central repository two weeks ago.

You might ask what the hell this is about. So in the current state you can download Apparat from Google Code. Then you have to have a working Scala installation which has been used to build Apparat. Whenever Scala is updated, I have to updated Apparat, and you will have to download Apparat again and install a new Scala version. This is absolutely cumbersome. This will change of course when Scala 2.8 is officially released.

I know that some people will always require command line access for Apparat. I am sorry for you at the moment that we have to play this game until 2.8 is out.

For the rest this is good news. If you have a working Maven 3.x version you can use the Apparat plugin and get automatic updates. This means you have to configure your project only once and do not bother with Scala and Apparat anymore. In fact if you want to you could also use the Apparat snapshot repository and get live updates.

Project configuration and setup is something one should not have to deal with. Therefore you can use the archetypes. I have built one for TurboDieselSportInjection.

mvn archetype:generate \
  -DarchetypeRepository=http://oss.sonatype.org/content/repositories/snapshots \
  -DarchetypeGroupId=com.googlecode.apparat \
  -DarchetypeArtifactId=apparat-archetype-tdsi \
  -DarchetypeVersion=1.0-SNAPSHOT

This terminal command is all you need to do to create a new project that is ready to compile — with TurboDieselSportInjection enabled. And it will automatically use the latest Apparat version.

FITC San Francisco 2010

When you have people like Scott Dadich, Ben Fry, Kevin Lynch and Yugo Nakamura gathered at one conference you can be sure to expect nothing less than a stunning event.

Shawn Pucknell is obviously raising the bar. This will be a tough time for an ordinary speaker like me. However I have no other intentions than giving a lecture that proves itself worthy of this event.

New Apparat Example

Good news everyone. The Apparat inline expansion works now to full extent after fixing some minor bugs. A complete example is also available. Just change the paths in the build.properties file and compile everything using Ant.

Apparat Example

Use the inline feature with care. Apparat does not try to optimize your code and performs nothing but dead simple inlining. This can lead to slower code due to the creation of lots of local registers. Your code gets also much bigger and will require more space in memory. I am actually not a fan of manual inlining at all. I think it makes only sense to inline code if you have a powerful optimizer available that will cleanup the whole mess.

The fun story about this example is that the inlined version is slower using the lastes Flash Player release candidate if you have only 40.000 particles. That is why I increased the number of particles to 80.000 ;). I developed the example using an old standalone player and the inlined version was nearly twice as fast. However when I watched the example in the browser with the latest release candidate the game was completely different. Kudos to Adobe for significantly improving the Flash Player performance!

Inline Expansion

In addition to macro expansion Apparat has now inline expansion as well. It works nearly the same way as macro expansion but without most of its limitations. To define a class for inline usage it must extend apparat.inline.Inlined and all its methods must be static. However the cool thing is that you can also return values in contrast to macro expansion. You can also pass normal parameters. For instance FastMath.sin(FastMath.sqrt(2.0)) is valid code using inline expansion. It is enabled by default in TDSI.

Apparat For Scala 2.8 RC2

Apparat is now available using the Scala 2.8-RC2 build. You can find appropriate downloads now on Google Code by searching for the Scala version of your choice. 7z compression on OS X should work now as well.

Macro Expansion

Apparat has another new feature called Macro Expansion. I talked about this with Nico Zimmermann at FFK in Cologne. Nico was using TDSI for a project but he was not very satisfied with it because you have to inline all inverse-square root tricks manually.
This is why Apparat has now macro expansion. I am actually not a big fan of it. I think a good compiler would do this for you without you having to go through all the steps. Unfortunately writing this compiler will take longer than the couple of hours I have spent on the macro expansion today.

So if you want to have quick and dirty inlining capabilities: this is for you. It is an easy fix for a feature a lot of people have asked for. I will continue working on TAAS to implement this much better in the future.

Polyglott Programming On The AVM2

Take some time and think about this tweet for a moment. It took me a while to realize that Joel Hooks is right. I was embaressed of myself. How could I forget about that? But I had also a big smile on my face at the same same time. Let me explain why.

The Java to SWF compiler does not compile Java sourcecode but JVM bytecode to ActionScript bytecode. This means I do not have to teach my program the Java language. It only understands JVM bytecode. This seems like an akward decision on the one hand since working on the bytecode level implies lots of problems. But it turns out that this was a really cool decision on the other hand. “Java to SWF compiler” is maybe the wrong description. “any language that compiles to JVM bytecode to SWF compiler” is maybe better.

So what does any language mean? Here is a list of JVM languages. Now you feel maybe like I did after reading that tweet. And I am really looking forward to get Scala up and running.

Some problems still exist. Threading is one issue and I will basically have to do what Scott Peterson did for Alchemy. But reflections, annotations and method overloading have to be solved as well. Some glitches may exist even after figuring everything out. Stacktraces will look pretty weird. However I think this is a really cool project.

Update: I forgot to mention something important. Java supports native code. This means you can build a library that works with OpenGL for instance. Those native methods can not be converted. There are also some other things that do not work. File access is just one of them.

TAAS As A Decompiler

The TAAS compiler is different from the ActionScript compiler since its input is not ActionScript source code but already compiled SWF or SWC files. Just like the haXe compiler can output AS3 instead of a SWF the TAAS compiler can do the same.

Now if you add one and one together you see that the TAAS compiler can be used as a very strong decompiler. My own tests have shown that it will work flawlessly where other commercial decompilers output rubbish. Since the compiler behaves like the Flash Player it will “execute” the bytecode in order to parse it which means it has a very highlevel understanding of the structure inside the SWF.

The only question is now what to do with the source code. I wrote the decompiler for my session at FOTB to show much easier how the optimizations behave. It is also a great tool to debug errors. But should it be opened or not?

To take it one step further one might also be able to write an obfuscator using the TAAS compiler. In my opinion it would be cool to have a strong decompiler and obfuscator, both being open source. We might also add an option to protect SWFs from the decompiler by adding something to the SWF metadata for instance. Of course this is just a simple rule which could be removed by someone once the code is open. What do you think?