Some days ago I whined about my problems with C++ and my webcam. After I figured out how to get the raw pixel-data with high speed I started with the interesting things.
I started with a datatype for the color. Since I am able to copy all bytes I recieve from the webcam into a struct of B,G,R (yes, reverse order) I can now use my 3Byte datatype for more tricky operations. I overloaded most of the operators. This is not possible in Flash (AS1,AS2,AS3) which is a shame. Maybe you want to compare RGB(255,0,255) with 0xff00ff. You have to write your own functions etc. in Flash and it is so boring and of course not a very sexy solution. Another example is the data conversion.
Imagine you have a RGB class with 3 components that are from the type uint (AS3) or Number (AS1,AS2). In functions like lineStyle() you need the integer representation 0xRRGGBB. Therefore you have to write a function that puts your r, g and b bytes into this format. The code is simply (r << 16) | (g << 8) | b BUT you have to write lineStyle( color.toInt(), … ); everytime and it slows the code down.
I am so excited about all the features that C++ offers that I overloaded most of all operators and type casts. ;-) Something like BlendMode.ADD is done by rgb0 + rgb1 etc. This makes coding much more easier.
After I played around with the datatypes and operators I started with filters. Filters are very important for my solution to solve the problem of the object recognition. First of all I do not want to track moving objects. So I need the outline of a not moving object. The hard thing is to remove all the noise (e.g. the background). It is hard to decide wich parts of the picture belong to the object and an outline in the background.
But I started only four days ago with working on this project. There are lots of things that have to be done. What I want to do is to create a polygon of the outline and then use the polygon to check it with a neuronal network. I will try to build the polygon from the outline that I get using different filters.
What you see on the left in the 2nd picture is a combination of blendmodes, convolution filter and binary filter. The third picture is my evil tricky part ;-) It is the outline of objects in motion with different weights. This will be used to get the extract of the picture where the object has been placed. It could also be used for motion vectors (maybe).
So what will be next? I have to buy something that allows me to use USB2.0. My camera is doing 1 FPS. This was also a problem when I did some experiments in Actionscript3. The next steps in developement are creating an extract from the whole picture, calculating a polygon of the outline and then finally using the neuronal network.
Share This
This week was a very successful one for me. I found a professor at the computing faculty of the University Bielefeld who is able to verify me in my fith a-levels exam. This is a special topic in informatics. Normally you have an exam in four topics. My fifth is an optional topic because I wanted to do this. It is about object recognition. I wanted to start with simple motion detection and maybe some object deteciton. Something like detecting a red ball.
Now everything gets more complicated. My new task is to create an application that is doing the following:
- Some objects are placed on a table. There is a camera directed in front of theese objects with a known distance
- User grabs an object and holds it in front of the camera
- The user puts the object back in any rotation. The object can also be flipped horizontally or vertically.
- The program has to identify that object
This will be a lot of work. I will use artificial neural networks for that process. Also some motion detection and maybe a lot of filters. I am just starting right now with an application written in C++. But there is the first problem. I need the rgb values in something like an Array (Flash) or int[WIDTH][HEIGHT][3] if you know about C. The current problem is that I do not have any clue about the WinAPI and all that stuff. There are a lot of tricks I do not know about. Maybe I will try using DirectShow for the video capture because I am currently using the Video For Windows (VFM) API and everything becomes very slow (9fps ;-)).
If there won’t be any solution (for me, as an unskilled person in C/C++) to capture the webcam nice and fast and put all the rgb values in an Array I have to use proce55ing or Actionscript3 for the whole project. But I need the speed.
Anyway. Thanks to David Rowald from extrajetzt for the nice stickers. They will look nice on my new ride - a Fiat126.
Update:
Phew. It was not that difficult to get more speed. Instead of using slow GetPixel functions of the WinAPI I am using now the direct information from the camera. One memcpy() call is doing me the work to divide all pixels into a rgb array, where I do not even have to split the values from 0xRRGGBB into RR,GG,BB. This is now done in one call. FPS still greater than 2*10^5.
Share This
I am sorry for that period of absence. I hope that I can manage to write more frequently in the comming next months but I still have to pass my A-levels. There are also some other projects I still have to finish and my new car wants to get ready for the comming summer ;-) I worked the last weeks some time on this piece of Actionscript3. This is not only just another particle-example like all the others. What I wanted to create was an extensible particle system which could create nearly every effect like ParticleIllusion does it.
My source of inspiration was “Designing an Extensible Particle System using C++ and Templates” which is an article about extensible particle systems in C++. Since Flash is still not able to handle templates I had to think about other possibilities. An easy way to simulate templates is using interfaces and dataholder classes. But this is also a loss of performance. Not the best solution. A new language that handles templates is haXe. All what I can say is that haXe got a lot of very nice things like the templates, enums, etc. but I wanted to create this particle system for AS3 projects. This is one of the reasons why I used AS3 instead of haXe. Another reason is that I really like the Flex2 environment with all the nice code hints and so on.
The particle system is build up on the Policy Pattern. You can read about that in the article I talked about. The policy pattern allows you to combine effects in lots of ways. Most attributes of a particle have their own policy. One for the color, one for the position, for the velocity etc. Theese policies tell the particle how to behave. An easy example is the LifeAction which tells the particle to decrease the life value with a given speed.
The basis of my particle system is of course the smallest unit: a particle. Particles stay inside a ParticleGroup which handles memory management and emits particles at a given position. ParticleGroups can be attached to a ParticleTimeline or a ParticleSystem. A timeline is something that allows you to create a special effect. You are able to create a simple explosion on frame one for example and then show the blast wave after some frames. The nice thing is that you only have to emit this effect once and do not care about any memory management or objects.
Then there are the ParticleSystems which render all particles. A ParticleSystem can use textures or simple pixels. It can be additive (BlendMode.ADD) or whatever you like. Some systems I already created are additive and simple pixel systems. The same for textured systems.
The last thing is about policies. You can create combined policies. For example the ForceFieldAction uses a forcefield to set the velocity. You could combine this policy with the GravityAction which adds gravity to your particle. There are a lof of possibilities. The next thing which has to be done are more policies to allow a huge set of effects and also more ParticleSystems. The timelines also needs the ability to emit particles for a given interval. Physics would also be nice. But the goal is an editor that allows you to create nice effects for easy use.
Some shots:
Two examples:
Share This
Some days ago I whined about my problems with C++ and my webcam. After I figured out how to get the raw pixel-data with high speed I started with the interesting things. 
