blog.joa-ebert.com - Blog of Joa Ebert » de.je2050.*

Some updates

joa — Thu, 08 Feb 2007 14:52:33 +0000

I added two the Imageprocessing Library some new blurs. RadialBlur, ZoomBlur and GeometricDisplacedBlur. The GeometricDisplacedBlur is very nice, because you can use any GeometricMap with this filter.

I also added a RedEyeRemoval and made some updates to the Color class. Some other filters have been updated as well. I will focus now on some examples and more GeometricMap objects. I am also thinking about filters for HSV color correction and this stuff. But that is not very hard since it is only the ColorMatrix that has to be extended. I am also trying to find good ways for texture synthesis (that will not freeze Flash for 10 seconds).

If someone has any idea for filters that are missing just post it here please.

Render, noise and morphological filters

joa — Thu, 01 Feb 2007 23:52:50 +0000

I had a lot of stuff on my mind the last days (besides the Wiimote). Check the documentation for nice pre-views. Here are the resuts:

de.popforge.imageprocessing.filters.*

Closing
Dilation
Erosion
Opening
Outline
TopHat
Curves
Blobs
CellGenerator
Corona
FractalPlasma
MakeTileable
Mandelbrot
OldschoolPlasma
Tile
AverageCut
MedianCut

de.popforge.imageprocessing.geom.*

Curve
StructuringFunction

Curves

joa — Sun, 28 Jan 2007 16:38:29 +0000

I think that I would sell this if I were an artist. Unfortunately I am not. What you see here is a fractal plasma with applied Curves filter. The UI in this example is of course not tweaked. But you can add and remove points from the curve. Also there are some other options like changing the interpolation etc.

Let’s do some serious art!

It is also nice that I have no lags in the stand-alone player even if I use live mode.

Library updates

joa — Thu, 25 Jan 2007 13:51:22 +0000

Just after the last release I have a couple of updates ready to use. Here is a list of new stuff:

QuickNormalMap (thanks to Ralph Hauwert)
DisplacementMap
GeometricDisplacementMap
QuickBumpMap (thanks to Ralph Hauwert)
Ripple
Spherize
Twirl
InterpolatedPlasma (thanks to toxie^ainc. and Hopper)
SinePlasma (thaks to tocie^ainc. and Hopper)
Interpolation (bicubic, bilinear, nearest neighbour)
Set of mathematical described displacement maps (RippleMap, SphereMap, TwirlMap)

As you can see it is growing. I am also happy about the nice help I got. There is a lot more to implement since I got the ok to use some rendering filters that theese demo-scene people use to create textures at runtime.

New ImageProcessing Library Version

joa — Sun, 21 Jan 2007 19:44:58 +0000

There is a new release of the ImageProcessing Library out for you people. I do not have that much time to spend but I hope that this is a helpful library and will even grow a lot.

Here is a small changelog:

Publish class util to send images in an easy way to sockets or websites
Format exporter (PNG, JPEG)
New core class for multiple layers
ToneMapping filter
Noise filters (GaussianNoise, UniformNoise, PerlinNoise)
Drawing util

I guess that there are a lot of small changes here and there but nothing special. I also want to implement some more filters later maybe. Right now the next thing I really would like to have is an online playground where you can load pictures from flickr or take snapshots from your webcam, mix them together and save them. The only problem is that this needs some sort of GUI and this is where the problems begin and my timescale shrinks. So you will have to wait for that. I am also looking for contributors. If somebody wants to share his filters or effects just send me an e-mail (address is on je2050.de).

Here is also a code sample. I will come up with more stuff and update the zip-archive aswell. This little piece of code generates the image you see on top of the post.
[as]var backGround: Image = new Image( 700, 200, ImageFormat.RGB );
var text: Image = new Image( 700, 200, ImageFormat.RGB );
var textGlow: Image;

Drawing.drawText( text, 0xffee3399, 120, 20, ‘ImageProcessing Library’, ‘verdana’, 32, true );

textGlow = text.clone();

new PerlinNoise( 700, 200, 8, 0xABC, false, false, PerlinNoise.CHANNEL_RED | PerlinNoise.CHANNEL_GREEN | PerlinNoise.CHANNEL_BLUE ).apply( backGround );
new Blur( 8, 8, 3 ).apply( textGlow );

var layerManager: LayerManager = LayerManager.fromImage( backGround );

for ( var x: Number = -240; x < layerManager.width; x += layerManager.width / 100 )
Drawing.drawLine( layerManager.backgroundImage, 0×80008888, x, 0, x + 240, 200, true );

for ( var y: Number = -240; y < layerManager.height; y += layerManager.height / 50 )
Drawing.drawLine( layerManager.backgroundImage, 0×80888800, 0, y, 700, y + 240, true );

new GaussianNoise( .16, true ).apply( backGround );

layerManager.addLayer( new Layer( text, 1, BlendMode.ADD ) );
layerManager.addLayer( new Layer( textGlow, 1, BlendMode.ADD, -5, -5 ) );
layerManager.addLayer( new Layer( textGlow, 1, BlendMode.ADD, 5, 5 ) );

var pixelate: Pixelate = new Pixelate( 3 );
pixelate.apply( text );
pixelate.apply( textGlow );

layerManager.merge();
textGlow.dispose();
text.dispose();

layerManager.backgroundImage.updateHistogram();

var histoGram: Image = Image.fromBitmapData( layerManager.backgroundImage.histogram.getBitmapData( 700, 400 ), ImageFormat.RGB );

layerManager.addLayer( new Layer( histoGram, 1, BlendMode.ADD, 0, -200 ) );

layerManager.merge();

histoGram.dispose();

new ContrastCorrection( 1.4 ).apply( layerManager.backgroundImage );
new LevelsCorrection().apply( layerManager.backgroundImage );

addChild( new Bitmap( layerManager.bitmapData ) );[/as]

Upcomming Imageprocessing Library Release

joa — Sat, 06 Jan 2007 14:35:03 +0000

I am sorry that I do not have much time theese days but I am planning to release a brandnew version of the imageprocessing library. This time I hope some of you already used it and have some ideas about improvements.

Here is the plan for the new features:

JPEG encoding (complete, using modified version of Tinic Uro’s encoder)
PNG encoding (complete, using modified version of Tinic Uro’s encoder)
Experiments on very fast mean values (using a Matrix approach orginally comming from Mario Klingemann)
Implementation of a fast interpolated/non-interpolated warp filter (using tesselation technique by André Michelle)
Interpolated levels correction
Simple drawing API (anti-aliased and normal lines complete, text and other primitives missing)
New core class for a layered image (supporting also blend-modes on layers)
Full alpha-channel support

As you can see some of the stuff is already complete. Other stuff is missing. The warp filter for instance has never been tested by me before but I have a lot of code here to implement that beast on a fast way. I could also think of two resolutions like I have done it with the sepia filter (QuickSepia and Sepia classes). Beacuse creating a simple warp filter using bilinear interpolation is not that hard. The only problem is, that it will be slow. The other technique uses the Flash drawing API to warp the image (and also for interpolation) but it is not 100% correct.

Another problem is that I lost the batch file for the ASDoc generation. But this is of course a minor issue. Stay tuned!

ImageProcessing Library (beta)

joa — Sun, 08 Oct 2006 19:40:18 +0000

A first beta release of the imageprocessing library can be downloaded right now. I like to experiment with my webcam and images. This is a fast library with a couple of implemented algorithms. Have a try and tell me what you think about it.

ImageProcessing Library
ImageProcessing Library Documentation

Updates:

Added QuickSepia filter which is using a ColroMatrixFilter instead of YIQ transformations (de.popforge.imageprocessing.filters.color.QuickSepia)
Added Color.add() and Color.substract() for easier blend-mode calculations.
The zip-file contains now the laboratory sources and a batch to compile them.

ImageProcessing library

joa — Thu, 05 Oct 2006 08:03:36 +0000

Got an idea what imageprocessing means? You always wanted to create som eyeToy like game, but it is a struggle to fight against bad lighting? You know that AS3 is fast. So what about some real-time imageprocessing?

This is a personal point of interest and I had a lot of fun creating this library. It is currently in an early state but it is very difficult to create acceptable results. This library is not very scientific but the output is nearly the same. For some reasons I had to speed up some processes of real imageprocessing. I will continue the fun and add some filters. I do not know from which categorie, maybe morphologic or some more basic binary stuff like Erosion or Dilation (and therefore Closing).

Have a small preview here. I will post either the sources (after having another look) or a compiled SWC. But Flex Builder 2 is some sort of annoying. If I compile the SWC with the compc it works fine until I want to include the SWC into a project. The project also compiles but there is no auto-completion in Flex Builder.

ImageProcessing Library Documentation

AS3 WebCam LevelsCorrection (updated)

joa — Sun, 07 May 2006 09:23:04 +0000

The imageprocessing-lab yields fruits. What you can see here is automatic levelscorrection applied on a 160×120 image. The fps decrease a little bit because I have to calculate the histogram three times. Two times for the histogram images and one time for the levels filter. So there would be another speed boost if the histograms are not shown.

My webcam is doing the auto-levels correction already. This is shit, because it is hard to correct corrected levels. And you will not see a big difference. Maybe your webcam is not doing this and you will achieve the grateness of this filter.

After a lot of tests I have got my favorite image format now. It is fast and easy to create new filters. Basically the image consists of an array which holds a BitmapData for each channel. The BrightnessCorrection filter works for example like this:
- Create a look-up table for new brightness values. This is for every channel the same.
- Apply paletteMap to every channel of the image

Easy, huh? And fast. paletteMap is one of the fastest native functions. It is even faster to use BitmapData.paletteMap instead of BitmapData.threshold. The basic function to binarize a picture is done by applying paletteMap with a palette of 0 and 1. This is faster then using threshold!

One last thing. The histogram is way to slow in my eyes. I am still searching for a solution to speed this up. Currently I read the value of each pixel using getPixel. This makes width * height iterations. Then I tested BitmapData.threshold with the == operator. Of course only 256 (non native) iterations but totally 256 * width * height. It was very slow. Another test was using a combination of BitmapData.fillRect and BitmapData.compare. Also slower than reading every pixel.

Maybe another good solution? The comments function is working. I am waiting for your suggestions :-)

Webcam with levels correction

If you can not see any difference in the two pictures. Try holding your hand in front of the webcam. This will result in very dark input and you see the levels-correction working.

Update:
Compiled using Flex Builder 2.0 Beta 3 which Adobe released a few hours ago. Now there is a big performance boost in the whole ImageProcessing. I do not really know where it comes from. setPixel seems to be slower than in Beta 2 but since I do not use setPixel at all in the whole package everything is faster. I think that some native functions (also filters) work faster.

The example is doing the following:
- Webcam whith (some sort of) double framebuffer
- Convert BitmapData to Image class with grayscale conversion (using BT709 algorithm)
- Render original Image
- Calculate and draw(!) original histogram
- Apply automatic levels correction
- Apply sharpening
- Render filtered image
- Calculate and draw histogram of new image

This runs at ~20fps for me in the browser. Disabling both unneeded histograms increases fps in standalone player to ~50.

AS3 Imageprocessing library

joa — Sat, 29 Apr 2006 13:17:27 +0000

Continuing with my work on some “research” releated to webcam and object recognitionI got a little bit addicted to imageprocessing. You can see some of a lot of working filters here. The latest beauty is the automatic levels correction by stretching the histogram. The auto-levels takes 250ms for a 640×480 px image which is not bad. Most filters are look-up based.

Thr problem is that I can never decide what seems to be the best and fastest method. An image is of course built from a BitmapData input. First I started with a BitmapData for every color-channel (4 bitmaps for a RGBA image). I am currently using an Array for every channel, because looping through the whole image is faster and I get rid of the get-/setPixel method.

But some new ideas using native methods could speed up everything again. I am not sure, but palletteMap has the potential to make all look-up-table based filters incredible fast. I will give it a try…tomorrow maybe.

flIRC and Flex 2 Beta2

joa — Fri, 28 Apr 2006 10:41:32 +0000

Updated flIRC to work with Flex 2 Beta 2 again.
Enjoy.

flIRC

AS3 Retinex implementation

joa — Mon, 24 Apr 2006 09:12:15 +0000

Here an implementation of the holy Retinex-filter. The idea behind it is splitting up the digital image into two parts. The luminance and the reflection image. With both parts you can for example increase the brightness without loosing color information. The retinex filter creates images that look more like the reality which is very interesting if you use a camera.

Imagine this in real-time on a webcam. All your problems with the scene appereance would be kicked away since you could level all images to the same brightness. EyeToy-like games could be played at night without even turning the light on.

This is a conversion from the MSCR (multiscale retinex with color restoration) implementation in gimp-2.3.7.
Have a look at the example picture and /as3/ip/filter/rgb/Retinex.as.

Simple Auto-Levels

joa — Mon, 24 Apr 2006 06:37:11 +0000

Just a quick note on auto-levels. You may know about that function in various graphics software like Photoshop. There was a simple question about how to implement auto-levels in Flash (especially AS3).

Therefore I googled around searching for an algorithm that fits my needs but I had no success. There was also no article in my beloved image-processign bible. What I found out is that the levels is about brightness, contrast and midtones. I am just using brightness and contrast here. You can find the whole source in /as3/ip/img/Image.as in my file-browser.

Now about the auto-levels with are not as good as the Photoshop results but it is a fast way to do it in Flash.

[as]
//– create an image
var img: Image = new Image( width, height );
img.setType( Image.IMAGE_GRAY );
img.loadBitmapData( bmp );

//– grayscale auto-levels
var a: int = img.getAvgValue();
img.adjustBrightness( Math.exp( (0×80 – a) / 0×80 ) * 0×10 );
img.adjustContrast( Math.exp( (0×80 – a) / 0×80 ) );
[/as]

Extensible Particle System

joa — Sat, 14 Jan 2006 12:50:54 +0000

I am sorry for that period of absence. I hope that I can manage to write more frequently in the comming next months but I still have to pass my A-levels. There are also some other projects I still have to finish and my new car wants to get ready for the comming summer ;-) I worked the last weeks some time on this piece of Actionscript3. This is not only just another particle-example like all the others. What I wanted to create was an extensible particle system which could create nearly every effect like ParticleIllusion does it.
My source of inspiration was “Designing an Extensible Particle System using C++ and Templates” which is an article about extensible particle systems in C++. Since Flash is still not able to handle templates I had to think about other possibilities. An easy way to simulate templates is using interfaces and dataholder classes. But this is also a loss of performance. Not the best solution. A new language that handles templates is haXe. All what I can say is that haXe got a lot of very nice things like the templates, enums, etc. but I wanted to create this particle system for AS3 projects. This is one of the reasons why I used AS3 instead of haXe. Another reason is that I really like the Flex2 environment with all the nice code hints and so on.
The particle system is build up on the Policy Pattern. You can read about that in the article I talked about. The policy pattern allows you to combine effects in lots of ways. Most attributes of a particle have their own policy. One for the color, one for the position, for the velocity etc. Theese policies tell the particle how to behave. An easy example is the LifeAction which tells the particle to decrease the life value with a given speed.
The basis of my particle system is of course the smallest unit: a particle. Particles stay inside a ParticleGroup which handles memory management and emits particles at a given position. ParticleGroups can be attached to a ParticleTimeline or a ParticleSystem. A timeline is something that allows you to create a special effect. You are able to create a simple explosion on frame one for example and then show the blast wave after some frames. The nice thing is that you only have to emit this effect once and do not care about any memory management or objects.
Then there are the ParticleSystems which render all particles. A ParticleSystem can use textures or simple pixels. It can be additive (BlendMode.ADD) or whatever you like. Some systems I already created are additive and simple pixel systems. The same for textured systems.
The last thing is about policies. You can create combined policies. For example the ForceFieldAction uses a forcefield to set the velocity. You could combine this policy with the GravityAction which adds gravity to your particle. There are a lof of possibilities. The next thing which has to be done are more policies to allow a huge set of effects and also more ParticleSystems. The timelines also needs the ability to emit particles for a given interval. Physics would also be nice. But the goal is an editor that allows you to create nice effects for easy use.

Some shots:

Two examples:

Fireworks (Timeline, Textured System)
“PerlinForce” (Additive Pixel System, ForceField based on PerlinNoise)

Zelda – Two (tile)worlds in AS3

joa — Wed, 09 Nov 2005 22:01:15 +0000

This time a pixelscroller in Actionscript3.
I hope you know Zelda from the good old SNES. Maybe you do not, but there are two worlds, the light- and the darkworld. As Link you are able to switch between these two worlds using a teleporter.
Now what about Flash and two worlds?
This is just a quick result of a bored evening but you can see what is possible. The world is 2048×2048 px large and has 128×128 tiles with a size of 16×16 px. This makes it easy to imagine what huge chunks of data Flash has to manage and I think the speed is ok. Of course the 640×480 px version could be faster.
I also made a test while working on the tile engine with a Mario-Level. The speed at 640×480 px was ok (!) so with Flash 8.5 you are able to build nice tile engines that run with a high framerate at 640×480 px.
Press your left mouse button to switch between both worlds.

By the way I found an old bug. I am not sure if it really is a bug but this is annoying since Flash5. You have two listeners. One for onKeyDown and one for onKeyUp. Now you have an array of boolean values which shows you if a key is pressed or not. Something like function onKeyDown( event: KeyboardEvent ): Void { isKeyDown[ event.keyCode ] = true; } and the same with false for onKeyUp. Now if you press two keys at the same time and you release one and then the other there won’t be any event for the second onKeyUp.

Particles based on Motion

joa — Mon, 31 Oct 2005 12:54:00 +0000

Here an experiment which shows 256 particles on screen. They get actived by the motion of the user (Webcam needed). This experiment does not show the full power of the routine because there are only the particles flying around. You can (and that is the nice thing) calculate and interpolate motion vectors. I did this in Flash8 with a 160×120 px webcam image which has been transformed to 320×240 px and the user was able to punch a ball around.
The routine is not very hard to understand. You have an image of the current webcam and greyscale it. Then you substract the last image, which you stored in a BitmapData from the current. The BlendMode.DIFFERENCE uses the formula newl = abs(old – new) and is exactly what we need. When I worked with Aibo’s camera I used some screen division method. You can also use scanlines which do the job for you. I use another method to get more inaccuracy (yes). I divide the screen in for example 16×16 parts and then I look onyl vor every 2nd pixel for example. This gives good results and keeps the number of total iterations low.
Anyway you know how much motion has been recognized in a given field.
If you want to punch a ball you would look for the best motion you can find around the ball and then look for the last motion. If you use pythagoras you get the distance between theese two points which is an indicator for the strength.
I had to interpolate the points a little bit to get this working with a ball in Flash8. Now have a look at theese nice particles. The performance depends on the number of particles and their size. There are currently 256 particles on screen and they have a size of 8×8 px. All particles get a different color at runtime and they fade using a ColorTransform. This experiment is very simple if you don’t think about the particles. It is only some sort this code:

[as] search.analyze( screen );

var m: Array = search.motion;
var n: int = 0×100;
var x: int;
var y: int;

while ( –n > -1 )
{
x = n & 0xf;
y = (n & 0xff) >> 4;

if ( m[ x ][ y ] > 0 )
addParticle( x * 20 , y * 15 );
}[/as]

I have to optimize this more and more. The array where the motion gets stored could have only one dimension. But the full 320×240 px webcam gets analyzed. There are 256 particles using blendmodes and colortransforms. If you do this with one ball only you could use a higher detail level for the analyzation.

Particles based on Motion