Sunday, January 04, 2009

Noise "Reduction"?

I have not purchased newer Pentax DSLR body for quite a while for many reasons, but one of the biggest reasons is: current Pentax DSLRs are all not night scene friendly. As I do like night photography quite much, I find it is totally illogical to "upgrade" to get worse results.

Actually, the K20D and K200D (and previously the K10D) are just some of the hottest and most noisy DSLRs ever made on this planet. Whilst the "hot" things about the K20D have been well known to the public already (look at this, this and this if not yet learned, just in case), the K200D is yet not a "cold" camera, neither. Just first look at the following K200D user report with an example "hot" and noisy night scene photo posted:-
(Text in Traditional Chinese, use Babelfish to translate if needed)

So, at this point, some people may query that if the user had turned on the "Noise Reduction" for long time exposure, or not? Well, I can answer that he hadn't, just because I can see so many hot pixels, even in the small downsized picture he posted! But do you think if he did, what would happen? Well, look at this:-

As seen in the crop, now black dots appear instead of white dots. So, hot pixels are all now becoming cold (or totally dark) ones after the "noise reduction". But wait, what does that long exposure noise reduction" actually do? In fact, it is just a dark frame subtraction - the camera takes another frame with shutter closed with the same amount of exposure time and then the dark frame is subtracted from the original one.

The problem? Subtracting two hot and bright pixels at the same position will just give totally dark pixels, as shown in the above lightning picture. These will become very visible once the background of the picture is not dark enough. In fact, a noisy camera (which has a noisy sensor) with such a NR choice will only give the user one of the following two outcomes: hot pixels that appear bright OR dark pixels in picture, for those longer time exposure shots. Of course, depending on the scene compositions, both can be visible or not that noticeable, just by chance. Worse thng can even happen if (the positions of) the hot pixels in the dark frame are not exactly the same as the original shot, just imagine! (But that's possible!)

Yet, for K200D users, they may be just a bit more "fortunate" than those K20D users as they *still* have the option to turn off the long time NR - at least not to waste an equal amount of lengthy time exposure for just taking the second "dark frame".

All in all, the real solution to this noise problem which appear during long time exposure could by no means be resolved just by the "dark frame subtraction" "technique".

But still, there is one *trick* that those camera makers are doing, it is the pixel (re-)mapping.

Pixel (re-)mapping is just a little trick to hide those dead or hot pixels. Primitively, this is a "technique" (for the manufacturers, but actually just *trick*) to hide all those dead pixels in the sensor which are existent because of manufacturing (process) imperfection and production defects (yield rate far from 100%). Those "unwanted" pixels which the manufacturer does not want the user to see are hidden. And they are disabled such that they will have NO function. Their pixel values are then inserted by the interpolated value derived from the neighbourhood pixel values, for each dead/hot pixel as marked.

The K20D has such a really "special" built-in custom function for letting the user to do the pixel-mapping by themselves, marking more dead and hot pixels as they (continue to) appear and update and store the dead pixel table. But as pointed out by the technical reviewers who had an early review on different production K20D units, the most terrible thing is that there is no resetting function made available for the user. So, when more and more hot pixels appear, the worst scenario is that *eventually* all the pixels in the whole sensor can be ruled out!

After all the above discussions, the conclusion we can draw up about is that the primitive solution to the noisy night scene pictures is only an less noisy sensor/imager by itself. This would be a hardware performance issue then, which, could by no means to be resolved by any *software* means - the reality.

Now, we have seen that the *only* way to go is to solve the problem is merely to improve the sensor noise performance, which just means that a larger photo site per pixel for receiving more light energy (e.g., Full Frame sensors) and better heat sink and drain are essential. All these are hardware improvement, particularly about the core technologies in making the sensor, instead of any software which can do no more magics.

In fact, I have not yet seen my Full Frame Canon 5D had any visible problem shown so far for long time exposure and of course no those similar troubles as reported by those K20 and K200 users in taking my night scene pictures, here is an example by me, taken in early 2008:-

And, if I have to use a DSLR like the K20D to take "real-time" night photography like the fireworks above, then it must be a big joke to me for having to wait another 10 seconds after I have just taken an 10 second shot and let those subsequent photo opportunities that may come to elapse during the waiting period! (not even to count the total delay as incurred in the additional processing time plus the final writing time to card). Well, remember, there is no way and *any* option for the K20D user to turn off the dark frame subtraction for this Pentax "flagship" DSLR!

Whilst many K20D users have been petiting for an firmware update for opening up the option of disabling the dark frame subtraction in their camera (do a quick search like this at large Pentax forums will return many of these and alike), I don't think Pentax/Hoya needed to do something particularly differently if they had the choice to do so. Afterall, it should be remembered that almost all DSLRs have the disabling option for the dark frame subtraction, even the K200D, why let alone for that "special arrangement" of the K20D? So, just go figure why. Indeed, the reason behind is trivial. Actually, Uncle Phil and his crew have already told us their findings, why we don't believe them?

Frankly, this very hot sensor issue (with considerably more typical background noise too (not just for long exposure) as found) of the K20D/GX-20 is really the first thing with the highest priority and upmost urgency which Samsung and Pentax/Hoya needed to tackle. Otherwise, they will have NO CHANCE to SUCCEED, but just FAILURE would result, I dare to say.

I really hope that the next generation of sensor used in the upcoming Pentax DSLRs can get rid of all those inferior sensor hardware weaknesses. If Samsung is still unable to make a better sensor, why Pentax not to source other better ones? At least Sony are still selling them sensors, aren't they? Of course, full frame is the utimate solution, why not trying to catch up faster in the game? Still, if Pentax and Samsung could really do no Full Frame, why not lower the sensor resolution, say, at 12MP, so that a better sensor should be made at the first place, for what they might to be able to do now, for a smaller sized APS-C sized sensor? My 6MP Pentax DSLRs used to have better noise performance, why so greedy even not capable?

Finally, an old example shot of mine with my old K100D is shown below:-

Anyway, the SAFOX VIII system performed very poorer for its (in)accuracy for shooting similar scenes. In fact, only the central AF point is marginally usable and all other 10 points of the SAFOX often yielded OOF photos (but then more re-composition actions are required as such). The photo below was taken using the AF:-

(With Auto Focus by SAFOX VIII)

After I noticed the AF error (despite the AF system told me that it was in-focus), I switched back to MF and use the somehow smaller viewfinder of my K100D to judge the "correct" focus:-

(With Manual Focus)

As an old man and Pentaxian, I have to say Pentax SAFOX VII and VI (in my MZ-S and MZ-30 respectively) both did a much better job for the same situation - the saddest thing there! :-(


Anonymous said...

most simple sollution is just to turn off your NR and let your RAW conversion program (Photoshop for instance) do the job with all the hot and dead pixels...

RiceHigh said...

This is surely NOT a "solution" when the NR CANNOT be turned OFF. Its *compulsory*!

Anonymous said...

It's great to see that you're back to the bitching and complaining we are all so used to. Welcome back RH!

PeterZheng said...

WOW! This Anonymous duck shows his own genitalia in the public location him was always so used to!

Whether Rice's review is right or error, then I agree or disagree with Rice's view, but I absolutely dislike that Anonymous uneducated duck in here!

In China, some uneducated FQs (know-nothing members) forbid that anyone to criticizes their emperor and gov, and then here this Anonymous uneducated duck want to forbid that someone else reviews Pentax?

PeterZheng said...

Since more peoples could freely criticizes United States gov, so it can became a strong country.

Because of more consumers could freely criticizes Canon DSLR, so that Canon's DSLR are powerful.

A frail person would fear the voices of criticism.

Anonymous said...

Funny Peter I think you think it's me... See the thing is there is more than one person who believes RH should keep his genitalia at home...;)
The thing that RH and Pentax have in common...
People tell both that they need to improve but neither listens...........
2 peas in a pod I say.
Peter at least your honestly rude....and whoever the other "anonymous" is his comment, though sharp, was really quite mild. Quick scan of th article shows nothing new or nothing that isn't a repeat of what he said somewhere else.
BTW: I put on my aluminum foil hat so my images in my head are clearer now thanks to less RF.

Anonymous said...

The "saddest thing there" is that you still cannot stop misinforming the public about the quality of Pentax products.

If things were as bad as you write, why does the K20D sell so well among Pentax users? Why does even dpreview think its 'AF works even in very low light', 'is also never intrusively slow for general shooting' and 'has been at least comparable with similarly-priced cameras, if not quite up to the standards of the more expensive, similarly-specified cameras from other brands.'?

Why is not everyone returning their K20D because of the 'oh so hot' sensor?

Dpreview once downgraded the K10D for its default settings. With other in-camera settings their criticism towards image quality would have been completely unfounded. To use the default settings against a DSLR (its not a P&S!) is just sad.

To, again, downgrade the image quality rating of the K20D because of a problem that many users do not have and that may well have been a problem of the copy dpreview had, is just as wrong.

But at least dpreview recognises the many, many good points about the K20D and doesn't exaggerate when the K20D doesn't quite reach the performance of cameras 2x and 3x as costly. Whereas your blog...

Anyone who has ever used the Pentax models you moan about knows your comments sometimes brink on the ridiculous. I'm just concerned about the innocent people that stumble across this blog and take your words for real.

Anonymous said...

The exif data of the last image says it was done with AF and shows the two AF points the camera selected. Why do you claim you used manual focus?

RiceHigh said...

Boy, could I use the Quick Shift Focusing when I found again the focus by the AF was in error? As I always told you, just believed it or NOT! And, I think NOTHING in my Blog is FOR YOU. Just forget about it. I have far more (pairs of) photos to show if you JUST want the EXIF showing that I put my camera in the MF mode (and the clear difference between AF and MF - where AF failed), but I think you are no way worth my time, neither you are needed to waste more of your time at my Blog, very frankly! People will judge and don't "misunderestimate" (funny new word newly created by Mr. Bush!) their wisdom. Your defenses are always looking very silly, IMVHO. Bye.

Anonymous said...

Why get so upset? I didn't know that quick shift focusing doesn't change the data to "manual focus". I think it should! Does it really not???
If it doesn't then your first shot may have been an accident. Camera focused and you accidentally put it out of focus? Anyway, shooting a night scene like this by letting the camera select the AF points appears to be a bit silly to me.

I don't get your rant against me and referring to my simple question as "your defenses". What's going on?

RiceHigh said...

If you have to believe everything I tell here are fake or just in errors, why bother?

My MZ cameras and Canon 5D have seldom failed in using the AF for shooting similar night scenes, why only those SAFOX VIII Pentax DSLRs are failing and cannot be used for such purpose? If you regard this is "silly", I think the most silly one is just YOU, and also those Pentax DSLRs which cannot do AF accurately and reliblably for night photography.

Post a Comment

Related Posts

Creative Commons License
RiceHigh's Pentax Blog by RiceHigh is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.