Skip to main content.
April 29th, 2005

Trying to get the best off cheap CMOS web-cams and expensive robo-dogs

Tonight I finally tried to put in practice some thoughts about the dreaded CMOS cameras static noise.The need came from the poor quality of Aibo’s camera. However, for the time being, I’m using a relatively cheap USB camera. With this camera, much like in Aibo’s case, the temporal noise reduction doesn’t maximize the quality of a single image. So, I finally tried to write some code to try locate the static noise.To make things easier, I decided to sample the noise in complete darkness (..with a finger covering the objective 8). This does not give complete darkness, however. There is of course the noise that I want to detect, plus the camera driver fiddling with white balance.. adding artificial brightness to the image.
At the current stage, my algorithm is very simple: I take the average color of all pixels in the image and subtract it from the reference “dark” frame. The one captured with my thumb covering the camera 8P. This goes for red, green and blue components. The result is an RGB image comprised of values that need to be subtracted from the subsequent frames.. those that I want to correct by removing the static noise.
Actually, I added slightly more complexity by subdividing the image in a grid of 32×32 pixel squares and calculating color averages by those squares rather than globally for the whole image. But I’m not sure that that’s helping much.This is a very simple approach. In fact, a simple subtraction isn’t the best solution. To suppress the noise, it’s very likely that I’ll need to scale values (multiplication/division), rather than offset them (addition/subtraction). Or possibly scale and offset at the same time..
In any case, I think that the early experiments are very promising ! I’m actually pretty excited about the results.

Here are a couple of examples, please keep in mind that the image quality is not meant to be good. I kept the environment relatively dark on purpose, to strengthen the noise effect and the countering effect of the algorithm.


A mousepad…

As it came from the driver.

With temporal NR and dark-frame subtraction.



A sleepy programmer !

As it came from the driver.

With temporal NR and dark-frame subtraction.

(Gone is the rGbRgBRGbeard 8)

..hopefully, my Aibo’s sight will soon improve too !zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz

Posted by Davide Pasca in Image-processing

This entry was posted on Friday, April 29th, 2005 at 5:59 am and is filed under Image-processing. You can follow any responses to this entry through the comments RSS 2.0 feed. You can leave a response, or trackback from your own site.

15 Responses to “Trying to get the best off cheap CMOS web-cams and expensive robo-dogs”

  1. BettyBoop says:

    eehhhhh……who is that man????
    Uhhhh……I’m having some….
    …I’m getting….mmhmm…uhh
    getting…some…..body..reaction??
    See you’re getting some response
    from my sexy-body!!!
    But why are you covering your
    face?? I want to see those soft-lips of you!!! Show me those lips, please…..

  2. BettyBoop says:

    Wait….I forgot about the hands.
    uhmmm….what’s up with those big
    hands of yours???? What is that supposed to mean?? Maybe we
    should meet!!…I’m mean…maybe those big hands of yours can teach me how to code….Does it sound exciting to you. Well, it does to
    mee….eehhh

  3. Davide Pasca says:

    That was weirddddd !!!!!

    ..but, at least now I know what to do to seduce women: cover my face 8)

  4. ragin' lion says:

    What color space are you working in? RGB or HSV or other? Performance issues aside, it might be better to work in a different color space to give you more room to play around in.

    I haven’t done much image processing myself, but the little I’ve done was very fun/interesting. Of course I started thinking, “I wonder if I could write a pixel shader to do this?” 8P

  5. Rince says:

    bettyboop is a gay man.

  6. Davide Pasca says:

    I thintk hat the CMOS sensor works in RGB space. However, I’m not sure if there are exactly 3 receptors per pixel. Some configuration have more receptors for green for example.
    The driver of this webcam can return YUV (YCrCb) 4:1:1 (four pixels of luminance and one each of chrominance) or RGB. I’m currently getting RGB images, then tranformrming them in YUV411 before performing the necessary temporal noise reduction.. then back to RGB for the dark-frame thing.
    Usually it’s better not to mess with RGB images because slight errors can bring very visible artifacts. In this specific case however, it would probably be best to work at the lowest level, with what excatly the camera samples, without any drivers in between… I think !

  7. Rasty says:

    On the Dilbert mouse mat picture the image processing is definitely an improvement, but on the second image I’d say we’re getting mixed results..
    Yes, the “beard” is gone, but I generally see more artifacts in the processed image than in the raw one, mostly on the arm and the wall.
    The wall also shows some horizontal lines which I believe are due to the 32×32 squares averaging..!

    But it’s still definitely cheaper than having to shave! :)

  8. Davide Pasca says:

    Ahh you spotted me 8)
    Yes indeed, the 32×32 average thing is probably doing more harm than good. But, it’s actually hinting me on the fact that the formula is wrong. It highlights the fact that the compensation that I’m applying is affecting the average brightness.

    As for the increased artifacts on the wall… you are right there too, that’s why I said that subtraction isn’t good enough. Doing subtraction works on dark areas, but is not working on brighter areas.

    In any case, it has to be added that much of this “static noise” thing started when I saw images from that old webcamera of yours. So, at some point you’ll have to do some tests for me.. because that’s when it all started.. when I realized of the sadness of the static noise 8)

  9. Davide Pasca says:

    Rince said…
    bettyboop is a gay man.


    This definitely comforts me !

  10. Anonymous says:

    Davide, I’m telling you now that
    I’m not “gay” (for sure!!). That’s
    so funny!! Why did he say something like that? I’m
    naturally born female and grew-up with a nice package…..and beautiful too. So, your friend Rince is absolutely wrong. Just
    ignore that bad comment( and mostly a joke). He’s just probably envy of you because someone like me, finally start noticing you.
    But no matter what, I still have a feeling that you’re a brilliant and beautiful man inside and out.
    And I believe I’m right!!

  11. bettyboop says:

    Sorry, Davide I forgot to put
    my name bettyboop so it shows anonymous instead!

  12. ragin' lion says:

    This is probably a stupid idea/suggestion … but have you considered finding what the “global average” of the image is then using that in conjunction with the 32×32 squares averaging?

    The analogy is similar to “local illumination” vs. “global illumination”. I’m no expert on image processing, but (it seems) that having some global information (histogram or something of the nature) might help clear up some of the artifacts you are seeing with your current method.

  13. Davide Pasca says:

    It’s not fair to throw intuitions like that 8P
    I’m actually not even sure I remember why I wanted to take the average on sub-squares.

  14. Rasty says:

    The problem with my crappy webcam (which I should really replace :)) is that the noise is not static but changes randomly with every frame. This requires a dynamic algorithm different to the one you could use with AIBO’s webcam I believe..

    Surely it would be easier for you to just send me a better cam! :)

  15. Davide Pasca says:

    Ehh.. wait.. I remember clearly the static noise !
    The dynamic one is not a big problem, I already fix most of it (although I only tested it with my cameras).
    I’ll make a version of the program where you can disable the dynamic noise reduction and we’ll see 8)

Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>