Tonight I finally tried to put in practice some thoughts about the dreaded CMOS cameras static noise.The need came from the poor quality of Aibo’s camera. However, for the time being, I’m using a relatively cheap USB camera. With this camera, much like in Aibo’s case, the temporal noise reduction doesn’t maximize the quality of a single image. So, I finally tried to write some code to try locate the static noise.To make things easier, I decided to sample the noise in complete darkness (..with a finger covering the objective 8). This does not give complete darkness, however. There is of course the noise that I want to detect, plus the camera driver fiddling with white balance.. adding artificial brightness to the image.
At the current stage, my algorithm is very simple: I take the average color of all pixels in the image and subtract it from the reference “dark” frame. The one captured with my thumb covering the camera 8P. This goes for red, green and blue components. The result is an RGB image comprised of values that need to be subtracted from the subsequent frames.. those that I want to correct by removing the static noise.
Actually, I added slightly more complexity by subdividing the image in a grid of 32×32 pixel squares and calculating color averages by those squares rather than globally for the whole image. But I’m not sure that that’s helping much.This is a very simple approach. In fact, a simple subtraction isn’t the best solution. To suppress the noise, it’s very likely that I’ll need to scale values (multiplication/division), rather than offset them (addition/subtraction). Or possibly scale and offset at the same time..
In any case, I think that the early experiments are very promising ! I’m actually pretty excited about the results.
Here are a couple of examples, please keep in mind that the image quality is not meant to be good. I kept the environment relatively dark on purpose, to strengthen the noise effect and the countering effect of the algorithm.
| A mousepad… |
As it came from the driver. |
With temporal NR and dark-frame subtraction.
|
| A sleepy programmer ! |
|
As it came from the driver.
|
With temporal NR and dark-frame subtraction.
(Gone is the rGbRgBRGbeard 8)
|
..hopefully, my Aibo’s sight will soon improve too !zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
Posted by Davide Pasca as Image-processing at 5:59 AM EDT
15 Comments »
4:30 AM.. and not feeling sleepy at all !
Saturday, I did the mistake of being out till the morning. It wasn’t planned, but it ended up like that. So, both Sunday night and tonight (Monday), I’m wide awake. I just got out of the shower. I had nothing better to do. After spending two hours in bed not being able to even get close to sleep.. and that’s under the effect of the sleeping pills (expensive, useless bullshit !).
Part of the problem is feeling guilty for not really being productive at home. I watched too much useless TV and browsed around the web. I wrote little code, and practiced little Japanese.. at the last minute of course. Then it’s 2AM and I want to feel relaxed to go to sleep. It didn’t help that I rested when I came back home this evening. But even then, I had to do it for the body, inside I really didn’t want to rest.
I’ll have to try discipline myself and cut on time wasting activities. In the end, if I don’t feel like I’ve achieved something for the day, I just can’t sleep ! The sleeping pills only help me sleep better, but don’t do anything if I don’t want to sleep in the first place. Going running at the gym also, is probably worse. It puts more stress on the body, while the mind keeps it awake.
I truly think that I was built to stay awake at night. It’s always been natural for me, having to work during the day is a big problem.
Posted by Davide Pasca as Uncategorized at 4:43 AM EDT
4 Comments »
After much work after installing Octave, I realized that it wasn’t working for me. Installing the for-Windows package is lame if you already have Cygwin, because it comes with another Cygwin environment, and apparently that can conflict with the first Cygwin environment. Making Octave coexist with my Cygwin environment instead, is not so easy. Specifically, some weird inconsistencies arose between my system at home and at work. Bha !
So I decided to go for Scilab. Scilab is pretty much like Octave and Matlab. The Windows version is actually a whole Windows application.. and that’s about the main feature I was after 8). That done, I proceeded to do some tests to get used to the language and the plotting. That took quite a bit of time, but I’m pretty happy now.
Playing so much time with this stuff has had it’s inevitable effect. I came back late from clubbing and this morning I somewhat dreamt that I was using Scilab to simulate some of my life events.. ouuuu !!
As for the night out… I went out with a couple of friends. I’ve become more and more influenced by a friend that is trying to push me the idea of collecting girls’ phone numbers. So, I socialized, and when the time came.. when the friend of the girl came to take her away, I hesitated and didn’t ask for her number ! That, apparently, was a big mistake. I wasn’t too interested in getting her number, but my friend tells me that I should have done it automatically. The basic idea is that certain reactions have to become instinctive !
The other friend, later on, showed how it’s done: he danced for a while with a girl that came with some less enthusiast friends. At one point they all gathered as to move away. My friend went, and pulled her back to the dancing floor. And that’s basically how it works. One has to act with instinct and determination in order to achieve success 8) This seems to be especially true with Japanese girls, because of their softer characters.
The cheap psychology behind is:
- The girl may or may not like you, it mostly depends on what you do. She doesn’t know you and you have to pursue her to some length. Basically one has to be somewhat pushy.
- When her female friends are around, they’ll try to take her away at some point. The girl will comply because she’s loyal to her friends (unless she really hates you 8). At that point one also has to show some assertiveness and take the burden to pull the girl away from the situation.. and basically excuse her from her friends.
…would be nice to put down a mathematical formula for all this !
Posted by Davide Pasca as Uncategorized at 9:02 PM EDT
2 Comments »
Yesterday (Tuesday), I read a good portion of the Octave manual and an interesting paper about PCA (Principal Component Analysis) (PDF of the paper Here). It’s interesting because it has a simple introduction to covariance matrices, eigenvectors and eigenvalues. Stuff that it’s used in video-games for collision detection. Something that someone at work asked me to help with, recently.. but I couldn’t really do much 8(
In this case PCA is more related to image compression and artificial vision (face recognition).
Then, at home, I fixed the system I made for building the photo-album on the home server. Mostly, I replaced some sad DOS batch file with a Bash script.. which I make run under Cygwin. It also uses ImageMagick to resize images and make thumbnails and Perl to build the final HTML pages.
Today, at work, I did quite a bit of research on face recognition using PCA, DCT and more esoteric stuff like Hidden Markov Fields, of which I know nothing ! Then I had to go to the gym. Then, at home, I installed and setup Octave (which so far I did install only in the office), and got started doing some programming with Octave, following the PCA Tutorial paper.
..so far, no graphic output 8) ..but I’m happy enough to have the opportunity to grasp some Octave/Matlab programming.
Posted by Davide Pasca as Uncategorized at 1:20 AM EDT
4 Comments »
The day at work was pretty slow. So, I took the opportunity to properly install Octave under cygwin (it wasn’t as easy !). Cygwin is basically the standard GNU (Unix-like) environment running under MS-Windows. It allows to re-compile and run most programs that will run on Linux/Unix/BSD/etc. Octave is one of such programs. It’s a Matlab clone. A program that allows to play around with applied math. It also has a programming language that is useful to work on algorithms.. like for image processing. That’s actually my goal. To use Octave to play around with images, rather than write directly ‘C’ code. This would allow to make quick prototypes of algorithms, and also verify data in a more practical way. Basically a tool for research.
Unfortunately Octave doesn’t have the same support for image processing that Matlab has. For the time being, I’ll just try to deal with it.
We’ll see !
Posted by Davide Pasca as Uncategorized at 1:03 AM EDT
3 Comments »
Past 6AM, Sunday morning already. Back from a club. Had my food, going to have water every time I wake up for it.
On Friday I sort of organized a meeting with a girl I met a couple of weeks before. I wanted to meet her again. As the ritual suggests. She mentioned about bringing her friends, I agreed by saying I’d bring my friends. So It became a gokon: a meeting of and equal number of men and women.. which came out right, although any of the sides didn’t quite know which number of people the other side would bring.
At the gokon I eventually got the famous question: why don’t you have a girlfriend.. then: what’s your type. And, after I showed a photo of Aibo: you should really get a girlfriend !!! Thanks !!!
Saturday, I went out with a friend, again clubbing. One place, then next place, eventually some connections were made. There, something funny happened: a girl honestly told her friend that I looked too much like a good guy. She kept saying that. I tried to explain to her that I might have some dark sides, but she insisted that I’m a good boy, that “I look like a student”. She even laughed at it.. as to say “get out of here, you good boy !!”.
Looks does matter, but one’s static look doesn’t really make much difference. Looks means also behavior. That is hard to fake.. and it’s funny how one girl can spot you so quickly.
Right.. I may be a good guy, but no one is really 100% good. Watch out !
Posted by Davide Pasca as Uncategorized at 6:05 AM EDT
5 Comments »
Lack of sleep is taking a toll on me. I can’t focus on things, I tend to be less positive than I could.. and my under eye bags are getting worse. Of the recent pictures that have been taken of me, I can only keep those where I wear sunglasses. I’ve been doctoring pictures before, to fix the tired look on my face … 8) … but that’s not a solution. The truth is that I need to sleep !!
Right now I’m using Crimson Editor to edit this post. I found out that it has a spell checker. Great ! But the real reason why I’m not editing within GMail is that I need a custom black background to lessen the pain on my eyes.
A few months ago I tried some popular sleeping pills that can be bought here in Japan. Unfortunately, those don’t seem to help in getting me convinced to fall asleep. They did give me some solid dreams, but as far as getting me quick to sleep.. not a chance.
On a different note. I’ve been looking around image processing, a field that interests me very much. This relates to improving image quality of Aibo, but also to video-compression. One very fascinating thing is how a thumbnail can look decent at its own resolution, but look terrible when stretched. Obviously all the “information” introduced by (usually) bilinear filtering is largely spurious.
I’ve been working on a better re-sampling algorithm for my next video-chat software. It’s nothing too new. Basically, I try to individuate the logical direction formed by adjacent pixels rather than blidly interpolate rows and columns.
Here is an example, on the left is a portion of an image down-sampled to 50% and then up-sampled back to the original size, using The Gimp (which probably uses bilinear filtering), on the right my up-sampling:
| 
|
| Bilinear up-sampling
| Cheap 2×2 matrix up-sampling
|
A lot more can be done. Specifically, working on sub-matrices of 2×2 pixels, like I’m doing, is still not nearly good enough. A better re-sampling should try to individuate edges and shapes in an image and preserve those features as much as possible. Ultimately, the best result would come out of an algorithm that has somewhat experience and can at some level understand what the contents of the image are, and fill the missing pixels at re-sampling time, with those that fit the algorithm’s experience. This is what our brain does, and that’s why we can “zoom” image thumb-nails in our mind, much better than bilinear or trilinear filtering do 8)
Posted by Davide Pasca as Uncategorized at 10:48 PM EDT
3 Comments »
Recently blogger.com seems to be rater unstable. I’ve been writing posts with GMail instead. GMail’s spelling works really nice and, most of all, has the Save Draft button. I use it rather frequently. As a programmer I’ve learned long time ago to be paranoid on saving what I write frequently.. and that’s on normal text editors, which, unlike web-based applications, have the full control of the window (I could close this browser right now and lose everything I wrote so far… better save now 8).
On the actual update: I temporarily removed the Aibo view form its own camera and focused on the outside camera which is now refreshing every 4 seconds. Also, I pointed a LED light on Aibo’s charging station, so that there is never a completely dark image.
I’ve been toying with the idea of merging the Aibo-capture program with the new videochat program, or at least to find a way to share more code between them. They both have the feature of saving JPEG frames at a constant rate. Now, what’s really missing is Aibo video input for videochat 8)
Posted by Davide Pasca as Uncategorized at 11:15 PM EDT
No Comments »
Monday I completed the automatic dialog creation thing. It also handles loading and saving of the status of a class. Basically I decide which fields in a class need to be edited, I give the types and tag names, then, a routine will create a dialog, while another will save or load the fields data in a given file. This is a very basic need to quickly tweak things around.
After that I reviewed my DirectShow capture code. DShow is really messy, possibly the worst part of DirectX. Complexity aside, I had some serious flaws in my handling of the capture. DShow captures in a separate thread (one that takes a lot of CPU time unfortunately 8( ). Testing with my Sony MiniDV camera attached to the Firewire port, it seems that DShow will always try give me 29.97 frames per second. That’s the NTSC standard, but really too many for a videochat/webcamera application. I changed my code to reject the extra frames, but I hope in future to be able to tell to DirectShow ahead what’s the desired framerate.. as it would save a lot of CPU time. Where the flaw really came was in the multi-threaded code, which wasn’t being nice about sharing the captured buffer data. Now it’s all much smoother, and the videochat/webcam application seems to be doing its job well.
Next, I may work a bit more on the videochat portion of it. It’s all still in black&white and the compression sends whole frames every time. The reason why I’m speeding the videochat thing is that recently I introduced my parents to Skype, and as a response, they have been asking for video too. We used to videochat before with my software, but that was without voice.
Posted by Davide Pasca as Uncategorized at 11:12 PM EDT
No Comments »
Saturday I went out to a club with a friend. Wasn’t too bad. There, we met some people we already knew.. that’s the easiest way to meet new people. I mean through friends, instead of going around bugging random girls 8)
Sunday I was rather tired, but was a nice day. So, I went to the Yoyogi Park with Marco. After that, I went to buy some colored plastic/foam sheets. One red, two green and one blue. To be used for checking Aibo’s camera fixed noise and possibly as chroma key background for an eventual update of JavaKazRace. In fact, I also bought 3 cheap car mini-models. I’m not sure if and when I will work on that though. Taking pictures of car models from all sides is rather time consuming !
On the coding side, I must admit that with the new console’s SDKs I get at work and with the Aibo SDK, I ended up having to deal with C++ more than I planned to. So, I started to write some of my stuff in form of classes. If anything to refresh my memory about C++. Still a very limited approach. With already some bumps.. for example using callbacks in C++ turns out to be a delicate issue. Declare lots of stuff public or start using templates, etc. Geez !!
I’ve been rather sleepy for the past days, and not really being able to sleep much (the small heart quake that I felt yesterday night didn’t help much !). On the coding side I’m focusing on organizing a system that allows me to quickly build dialogs where I can fiddle with data. Instead of having to do everything through the command shell like I’m doing right now.
Hopefully for tonight I’ll be done with the GUI side, and I can start working on implementing features in the Aibo camera capture software thing.
Posted by Davide Pasca as Uncategorized at 8:59 PM EDT
6 Comments »
