Web Analytics RiceHigh's Pentax Blog: When OIS Lenses are Put onto Sensor-shift IS Bodies..

Tuesday, September 08, 2009

When OIS Lenses are Put onto Sensor-shift IS Bodies..

Here is a very interesting latest experiment by the Japanese Impress DC Watch:-

http://dc.watch.impress.co.jp/docs/review/special/20090908_313599.html
(Article in Japanese, Google Translation in English Here)

The following are tested and compared:-

1. Sigma 50-200mm F4-5.6 DC OS HSM on K-7, and;
2. Panasonic Lumix G Vario 45-200mm F4-5.6 Mega OIS on Olympus E-P1.

And then both lenses are zoomed to the tele side, i.e., 200mm and a target at a distance of 5 metres was shot. 30 shots was taken for each test for each of the four combinations of enabling the lens-based and/or body-based IS for each combo. The amount of blurred shots against successful ones were then counted.

As the K-7 has an multiplying factor of 1.5X whereas that of the Oly is 2X. As such, the effective focal lengths, in traditional 135/film sense, are just 300mm and 400mm respectively for the K-7 and the E-P1 under this test. To make the test "fair" enough and meaningful, the tester then set the shutter speeds at 2 stops slower for both cameras, i.e., at 1/80s and 1/100s respectively for the two cameras.

One thing I think of a minor flaw of the test is that the focal length should be the right thing to adjust between the cameras, instead of adjusting the shutter speed. It is just because with different effective magnifications of the shot target, actually it is indeed somehow unfair to compare for the obtained results, for whether the shot is blurry or not. The design of this test favours more to the K-7 actually, as the magnification of the effective focal of 300mm is just lower than that of 400mm for the same target placed at the same distance of 5m away.

But anyway, let's look at the results. For the sake of easy reading and I think not many of my readers here do read Japanese (non the Google translation), I copy and re-type a summary table below for the obtained results:-

Condition
(Lens / Body IS On or Off)
K-7 Results (Number of Successful / Failed Shots)
Successful Rate (%)
E-P1 Results (Number of Successful / Failed Shots)Successful Rate (%)
On / Off
17 / 13
56.67
9 / 21
30
Off / On
11 / 19
36.67
20 / 10
66.67
On / On
16 / 14
53.33
5 / 25
16.67
Off / Off
5 / 25
16.67
17 / 13
56.67

I've also calculated and added the successful rate in percentage in my above table to give you a better idea on the actual performance.

Now the results are really very interesting..

1. Without any IS, the K-7 gives a lot of blur shots but the E-P1 is not that bad actually even with everything turned off. Well, one possible reason for this is because the E-P1 is mirrorless.

2. With body IS on but lens turned off, the performance of the K-7 is not much satisfactory and only marginal improvement is achieved. With solely body IS, the E-P1 has been able to improve the successful rate only by a very marginal factor, also.

3. With solely the lens IS, now the K-7 (but actually the Sigma!) performs the best, the successful rate is the same as what the user can get with the E-P1 with body IS. In contrast, with the Pana lens to do the OIS on the Oly, the result is really poor. The successful rate drops far below than what everything is turned off! Is that the Pana lens is incompatible with the Oly actually? Or is it that Pana lens is just a poor performer for its IS ability?

4. With both body and lens IS turned on (which is non-sense actually), the K-7 combo does still work. The successful rate is only just second to using the lens IS alone with a small difference and it is still far better than what the K-7 body IS/SR alone can do and of course better than when there is no IS of any at all. In fact, this result can be explained and it does tally with the previous findings on the effectiveness of the (Sigma's) lens against (Pentax's) body IS. The K-7's body IS is found to be ineffective in this test whilst the Sigma is shown to be working better. When both systems counter-react, the Sigma is still able to "overcome" and thus as a whole it does still work! On the other hand, the Oly/Pana combo gives the worst result when both IS systems are put into work, which is indeed not a big surprise as chaos has been created and blur is introduced.

So, after all these. The conclusions are clear. With the K-7, it is better to use the Sigma in-lens OIS technology than using the Pentax's SR! (Well..) With the Oly, it will do no big harm to forget about any IS completely as turning on the body IS only improves the successful rate by a small fraction, whereas using the lens IS could give worse results!

Comments (17)

Loading... Logging you in...
  • Logged in as
"To make the test "fair" enough and meaningful, the tester then set the shutter speeds at 2 stops slower for both cameras, i.e., at 1/80s and 1/200s respectively for the two cameras."

Let me get this straight, he set the K-7 to a much slower shutter speed and then - *gasp* - the K-7 missed more shots than the EP-1?? Is that really the comparison we are looking at?

I find it hard to believe that in an actual fair test and without any SR technology enabled any camera should get twice as many missed shots as another. There are two options - either Pentax actually purposefully shakes the sensor if SR is disabled, to make their SR technology look great when enabled, or the test is flawed.

Also, a doubling of the rate of "keeper" shots with in-body SR vs. no SR is hardly what you call a "marginal improvement". It is interesting, but not unexpected that the in lens stabilisation is better for a 200mm lens.

To make this test worthwhile, the chap really needs to retake all the shots. Let at least 2 different people shoot like 10 shots at each setting for a range of shutter speeds (say 5EV range, in 1EV increments), for each of the possible SR combinations. Then rate the shots on a scale of blurriness and tabulate the results. It's a lot of work (1200 shots to analyse if we have 3 people do the shooting :) ), but then you could actually get some meaningful data out of the whole thing - like what the shutter speed is where each camera performs fine without any SR and then how much lower you can go for a given rate of "keeper" shots.
4 replies · active 811 weeks ago
> "To make the test "fair" enough and meaningful, the tester then set the shutter speeds at 2 stops slower for both cameras, i.e., at 1/80s and 1/200s respectively for the two cameras."

2 stops is correct, 1/200s is a Typo, it states in the original Japanese article as 1/100s, which is exactly two stops slower than the safe handheld shutter speed. And, I have corrected the error in my article. Thanks for pointing it out.

> Also, a doubling of the rate of "keeper" shots with in-body SR vs. no SR is hardly what you call a "marginal improvement".

To me, 16% and 36% hitrate make no big difference, as both are just a failure per case. Especially the shutter speed is only lowered by 2 stops, which is far less than what the factory claims for its performance, i.e., 4 stops.

> It is interesting, but not unexpected that the in lens stabilisation is better for a 200mm lens.

Yes, but that's also the problem with Pentax's "solution". In fact, they have little effect for tele focals.

> To make this test worthwhile, the chap really needs to retake all the shots. Let at least 2 different people shoot like 10 shots at each setting for a range of shutter speeds (say 5EV range, in 1EV increments).. (snipped)

There is no need to do so as the test environment is already fixed and under control. Furthermore, control shots were also taken for comparison to judge the results. Other variables are not so important as at least in that particular set of testing parameters, the results are obtained scientifically and they can be meaningfully compared.

As I have stated, to make the test more fair, the same effective focal length and shutter speed should be used, e.g., 200mm for the Sigma lens on K-7 and 150mm for the Pana lens on the E-P1 and then both tested and shot at 1/80s, i.e., two stops slower.
> There is no need to do so as the test environment is already fixed and under control. Furthermore, control shots were also taken for comparison to judge the results. Other variables are not so important as at least in that particular set of testing parameters, the results are obtained scientifically and they can be meaningfully compared.

Well, no.
If the environment was properly controlled and the conditions accounted for differences in the two camera/lens combinations properly, then the OIS OFF/SR OFF test should come out even or close. At the moment, this tests shows that (with no SR technology at all) the K-7 is 3 times worse at taking non-blurred pictures than the EP-1. Anyone who has used any dSLR, will see that this is absurd. Such a strong claim requires meticulous evidence, which this test isn't and doesn't even come close to.
Your that assumption may not be valid. The K-7 is a traditional Reflex with a Mirror. The E-P1 is a digital rangefinder without any mirror. It will not be surprised if the E-P1 produces far less vibration than the K-7 during shutter release as such.
Oh come on, mirror shake does affect sharpness, yes, but it's probably more than an order of magnitude less than the kind of instability you get from shooting handheld in the first place.
Mirror slap might introduce some loss of fine sharpness to an image taken on a tripod with a stopped down telephoto prime, and it has a huge impact for astrophotography, but I refuse to believe it would make enough of a difference with a consumer telephoto, wide open and handheld.
This test was the worst SR/SSS/VR/IS test that I ever seen. It's complitly redicilous.

If you want to see a real pro test. Here is the real deals:

http://www.slrgear.com/articles/is_olympuse520/IS...

http://www.slrgear.com/articles/is_1how2read/read...

http://www.slrgear.com/articles/is_1iswp/iswp.htm
What a piece of sh*t ... on Pentax (and Sony) the Sigma OIS system is disabled during the triggering process, it is not possible to have sensor and lens stabilisation at the same time (it would correct twice the shake, so would give 99% blurry pictures). Conclusion : Pentax antishake can correct 33% to 55% at 200mm 1/80s.
2 replies · active 811 weeks ago
Is that so? I thought they had made some new lenses where the OIS was enabled even on the Pentax versions.
It does make sense to have OIS rather than in body SR for long telephoto lenses, where in order to compensate a small shift, the senor needs to move a lot. Of course the best system would be a hybrid, integrated one, with accelerometers and actuators both in the body and lens actually communicating with one another and cooperating.
The OIS is active on Pentax bodies, but not during the shoot, when the sensor shift becomes active.
J-M Fangio's avatar

J-M Fangio · 812 weeks ago

Wait, they tested stabilisation using a tripod ? Most ridiculous thing I've ever read...
2 replies · active 811 weeks ago
The tripod was used as a control so they could have a basis to compare blurry vs not-blurry. They have a picture of the test, a guy standing 5 meters away and taking the shots: http://dc.watch.impress.co.jp/img/dcw/docs/313/59...
Thanks for the clear explanation.
Manolo Paleto's avatar

Manolo Paleto · 811 weeks ago

why don't we talk about the virtues of the Canon 7D, another K7 wannabe??/
ah well seems that's yet another pentax failure to improve while brands like olympus who started under Pentax is now 4 to 5 times better in every way. What's new these days? Will we discover yet another Pentax failure again? The answer after the shot... ooooo
Real Engineer's avatar

Real Engineer · 810 weeks ago

rofl @ prestigious.
rofl @ this being a controlled test.
rofl @ ricehigh claiming to be an engineer and not understanding why.
Real Engineer's avatar

Real Engineer · 810 weeks ago

The results actually show that the Pentax has real improvment in its SR in every mode.
It even recognises an image stabilised lens and shuts off its internal stabilisation.
How freaking great is that!
Yet, where you want to call that a fault, I call that fantastic engineering.
but, let the results speak for themselves when correctly interpreted.
I have no idea how you mishandled this one so badly and the same goes for thet prestigious magazine.
They need better technical staff is all I can say.

The reference is the off/off results for both cameras.
This is pretty logical. This is the only way to eliminate as many varibles as possible to determine SR performance.
The other combination's need to be compared to the reference on a camera per camera basis. If not, the results are WORTHLESS.

Doing so shows the Pentax has real and significant improvement in all SR modes compared to the reference (no SR modes being activated).
The improvement is large, 100%, 200%, 300%. Its actually greater but I rounded down.
The Olympus shows 17% improvement in only one mode and a degradation in all the other modes. I did not round here like I did for the Pentax or it would show no improvement at all and that would be unfair to the Olympus since it does show a huge 17% (lots of sarcasm there)

YES, The OLYMPUS DEGRADES BY TURNING ON SR in 2/3 modes and improves only 17% for the one case where it actually helps.

And you say this is bad for Pentax, when the Pentax shows a minimum of 100% and going to 300% improvement.

If there is any proof one needs of blatant lying and deception regarding facts and figures, this is it.
Please keep your conclusions accurate.
People want to hear real information not hyped up blatant bad testing and bad interpretation of that testing.
All credibility for your measurebation skills has just been lost when something this simple and obvious can be twisted in such a deceiving manner that you convert fantastic results for a product against it competition and pervert it to make it appear bad.

BTW, your site does not allow posting under the opera browser.
I think its a hardware issue.
Firmware upgrades won't fix it. rofl
(that's sarcasm too, just so you are 100% in the know because I actually don't want to deceive you)
1 reply · active less than 1 minute ago
I may be merely an engineering student, but even I know that comparing results like this need to be normalized. Thanks for pointing that out!

I do find it interesting that the tester had so few good shots with the K-7 sans IS. I'm thinking that there could only be a few explanations for this. One, mirror slap is to blame, or two, he simply fails at shooting at 1/80th. These lead to a few conclusions; in either case, as Real Engineer pointed out, Pentax SR (and Sigma SR) really does a spectacular job. Whether he is just awful at shooting a DSLR or mirror slap shakes things up, >300% is a vasly significant improvement. One may argue that this proves that the K-7 is tragically flawed, but that's not really the point of this exercise, is it? The focus (lolz) isn't whether or not the K-7 has a fatal flaw (a mirror), but how well the IS works. Results clearly show, when analyzed correctly, that using ANY stabilization with the Pentax results in dramatic improvement. The Oly maintains good numbers without lens IS, but clearly (haha) doesn't do so well with OIS.

As for discussion regarding mirror slap, I'm not sure about it being the cause of such low numbers. The K-7 has (and is well regarded for) a delightfully soft and quiet mirror slap and shutter noise, meaning the mirror and shutter are well damped and smooth operators. Combining this with a heavy 200mm lens, even at 1/80th, I would guess that mirror slap is, at most, minimal.

In regards to this post: http://forums.dpreview.com/forums/read.asp?forum=...

It appears here that the Km, with its quieter (more damped) slap, doesn't adversely affect slow shutter speeds, whereas the K-x has significant blurring. I've heard (lolz) that the K-x has a particularly loud mirror slap sound, but I'd yet to see one in person, so I can't personally verify that. This would lead me to believe that the K-7, with its notoriously quietness, would act similarly to the K-x, that is, would not adversely affect most pictures by means of mirror vibration.

So.... user error? Hidden conspiracy? Perhaps there's an anti-pentax pirate holding the poor man at knife point!

And now for some non-serious arguments:

-Inspection of this picture: http://dc.watch.impress.co.jp/img/dcw/docs/313/59...
Clearly shows that the lens is in Manual mode, so the cause of poor non SR shots must be due to poor manual focusing skills.

-Or perhaps the tester has very sensitive hearing and fragile nerves, and the abrupt mirror slap is just too much for the poor guy.

-Anti-Pentax Pirate League. A-PPLe. (This is why we shouldn't buy Macs)

Post a new comment

Comments by