Terra Galleria Photography

Canon EF lenses sharpness measured with Imatest

9 Comments

Being a large format photographer, I tend to be a bit picky regarding image quality. When using 35mm cameras, since one needs to enlarge so much from the small image capture, everything has to be the best possible in order to yield a sharp print.

Of course, image quality is a minor aspect of photography, compared to all the other factors that go into making great photographs. However, to quote Norman Koren, “the technique you develop in testing cameras and lenses will spill over to your daily photography. Alfred Stieglitz tested film and developers extensively when he discovered photography as a student in Berlin. Ansel Adams performed extensive tests in the development of his zone system. Although nobody would claim that testing is responsible for their unique vision, it certainly contributed to the skill that transformed their vision into prints of transcendent beauty.”

Back in the early 1990s, I was doing quite a bit of lens testing in order to understand which lenses were sharper, and how the choice of focal lengths and f-stops affected image quality. One photographed targets (that had to be carefully labeled to identify the parameters at a later stage), had the film processed, and then squinted with a microscope, struggling to determine which groups of bars were “resolved”. This approach can yield very reliable measurements, including resolution in lpm (the best example of them today is found on William Castleman’s EOS lens reviews) but it was tedious, and I eventually lost interest.

This changed in 2004, when Norman Koren released his first version of Imatest, a software to automate digital image quality measurement. You still need to set up the target shots carefully, but now you just input sets of digital images into the software, and then read measurements. Compared to other solutions, Imatest distinguishes itself by the simplicity of the test targets it uses, consisting of two uniform regions separated by a straight border. Those targets are easy and inexpensive to make.

In late 2005, Canon released the 24-105/4, which was anything I wished for in an all-purpose lens. The only question mark was image quality. To find out for myself, in early 2006, I tested this lens against my 24-70/2.8 and other EOS lenses on a Canon 1Ds mark II. I have previously written a comparative review of the 24-105/4 and 24-70/2.8 lenses. This post summarizes my findings with respect to sharpness.

I won’t detail my experimental procedure here (this may be the subject of a future post) but let me say that tests were done very, very carefully and the results were repeatable. Imatest can provide a lot of information, including, in the current version, 3D graphs and a MTF graph similar to the ones published by Canon in EF Lens Work. This test was conducted with an older version of Imatest. For the sake of simplicity, I am using here a single number to represent image quality. This number is the MTF 50 calculated by Imatest, integrating several measurements over the image area. The larger the number, the higher the resolution/contrast.

Lenses tested:
24-70: Canon EF 24-70mm f/2.8L USM
24-105: three copies of Canon EF 24-105mm f/4L IS USM
12-24: Sigma 12-24mm f/4.5-5.6 EX DG IF HSM
17-40: Canon EF 17-40mm f/4L USM
70-300: Canon EF 70-300mm f/4-5.6 IS USM
70-200: Canon EF 70-200mm f/2.8 IS USM
24TSE: Canon TS-E 24mm f/3.5L
45TSE: Canon TS-E 45mm f/2.8
50: Canon EF 50mm f1.4 USM
85: Canon EF 85mm f/1.8 USM
90TSE: Canon TS-E 90mm f/2.8
100: Canon EF 100mm f/2.8 Macro USM



@24	24-70	24-105  	17-40	12-24		24TSE
2.8	26.2	na/na/na        na      na		na			      		    
4	29.8	28.2/34.8/35.0	28.3	na     		27  
5.6	31.6	33.6/38.4/35.8	33.3	23.6 		29.8
8	31.7	35.7/37.4/34.6	36.1	29   		31.1
11	30.2	33.2/33.4/32.3	34.6	29   		30.3
16	27.2	29.0/29.1/28.9	30.3	26.7 		26.9

@35	24-70	24-105		17-40
2.8	28.2    na/na/na        na
4	32.3	32.6/36.0/34.0	21.8
5.6	33.7	37.4/38.6/37.5	23.2
8	33.4	36.8/37.1/36.6	29.7
11	31.6	34.3/34.6/34.1	34.3
16	28	29.6/30.0/30.3	30

@50	24-70	24-105				50	45TSE
1.4	na      na				13.7    na
2	na	na				21.4    na
2.8	24	na				32.87	33
4	29.2	24.4/32.7/32.5			40.11	35.6
5.6	31.5	35.8/33.9/37.1			40.7	35.4
8	32	37.2/33.3/35.1			39.7	34.6
11	31.2	35.3/34.2/34.5			36.2	33.2
16	28.3	19.7/31.2/31.4			31.9	30

@70	24-70	24-105		70-300	70-200	85	90TSE
2	na      na		na      na	25.2	na
2.8	20.3	na		na	29.9	28.8	29
4	23.66	25.2/25.1/26.5	27.9	32.2	32.6	34.2
5.6	26.98	32.3/37.6/35.5	32.8	31.7	36	38.2
8	28.77	28.8/36.9/34.6	37.1	31.2	37.2	38.6
11	28.4	34.0/35.4/33.6	35.3	29.8	35.1	36
16	26.52	30.3/31.0/30.1	31.2	28	30.8	31.4

@105		24-105		70-300	70-200	100-400	100
2.8		na		na	32.6	na      29.8	
4		20.1/23.1/26.0	na	34.2	na      34.1	
5.6		26.0/33.3/34.9	34.3	34.1	28	34.6	
8		23.5/35.8/35.3	37.2	33	30.6	36	
11		18.8/34.4/33.3	36.4	31.2	31	35.4	
16		29.4/30.7/30.1	32.2	28.7	28.6	30.5
	

A wealth of information can be found in the table above, but here are a few general observations. Some will be well-known to some readers, but are worth repeating for others.
  • There is a significant performance variation between copies of the same lens. In general, Japanese lenses are not individually tested (unlike some German lenses, but they cost three times more). If you want to be sure to get a “good” copy of a lens, buy a few, test them, and keep the best.
    Amazon has free shipping both ways, and an easy 1 month return period, but beware that this does not apply to third-party sellers, which have their own policies. Rather than buying from a third party seller on Amazon, I prefer B&H. They have a 14-day return period and free shipping for some items only, however I have confirmed with their customer service that there will not be a problem to return lenses ordered for testing. Other vendors may charge restocking fees.
  • All lenses are sharpest at middle apertures (sharpness is limited by abberations at wide apertures and by diffraction at smaller apertures. In fact, by f16, all lenses perform almost the same, which is why I didn’t bother to make measurements at f22). If given the choice, use a middle aperture such as f8.
  • Zoom lenses can be sharper than fixed focal lenses.
  • Inexpensive lenses can be sharper than more expensive lenses.
  • Zoom lenses are in general weakest at longest focal. If you have the choice between two overlapping zooms, use the one which is less zoomed out.
Key findings about the 24-105:
  • 24-105 is sharper than 24-70 at all focal lengths and apertures. This comes as a surprise to some, as the 24-70 is the more “professional-looking” lens, with its 2.8 aperture, better construction and hood, and slightly higher price, but the conclusion is consistent with Chasseur d’Images tests.
  • 24-105 delivers excellent sharpness wide-open at wide focal lengths, but needs to be stopped down 1 f-stop at longer focal lenghts
Findings about other lenses:
  • The sharpest lens by far, the 50/1.4 is also the least expensive. Excellent lenses in this focal length are very easy to design and manufacture. There is a reason why those “normal” lenses were, in the past, bundled with SLR cameras bodies.
  • Of my three tele-zoom lenses, the sharpest one is also the least expensive, the 70-300, which in particular is sharper than the much-praised 70-200/2.8.
  • Of the three TSE lenses, the sharpest is the 90, followed by the 45, and then the 24. This is expected, due to the difficulty of designing a short lens with a large image circle.

9 Comments

  1. Tony says:

    Excellent work here. Thanks a bunch for sharing. Simple lines of resolution doesnt always mean sharpness. I find it hard to believe that the 24tse is not as sharp as the 24-105. Ill bet the edges are way sharper. My experience with the 24-105 (tripod mirror lock up etc) is not that great

  2. Michael Frye says:

    Did these tests measure edge sharpness? To me, a lens is only as sharp as it’s softest point, usually the edges, and I find that the standard “lenses are always sharpest at middle apertures” doesn’t hold up if you look at edges and corners, where small apertures may be better.

  3. QT Luong says:

    For each lens, I had a dozen measurements, including in the corners. The score number used is an aggregate. It is of course impossible to tell the story with a single number, but it makes it simpler to compare a large quantity of lenses. If there is enough interest, I may post more detailed results from my next test.

    What I have found is that in general, after the middle apertures, the sharpness doesn’t really improve that much in the corners, but it does drop in the center. This results in a sharpness that is more uniform across the field, something that one might prefer to a situation where the center is sharper, even though the lens “score” would be lower.

  4. Jerry Greer says:

    QT Luong,

    I really wish that you’ d gotten your hands on the new TS-E 24mm f/3.5L II. I have it and the results are fantastic. There’s NO comparison between it and any other WA lens that Canon offers. This lens is sharp from edge to edge and sharpness really doesn’t change much through the f/stops. Best of all, it is virtually CA free.

    As for the 24-105 being sharper than the old TS-E 24mm. Sure, it may be touch sharper f/stop to f/stop but in a true test using the tilts (as the TS-E’s were designed) to maximize DOF the TS-E will kill any of the fixed (non-tilting) lenses. Where you can use f/5.6 to f/8 with a TS-E in a near/far image you must use maximum f/stops, such as f/22, with the 24-105. I’ve witnessed a test between the TS-E and a Distagon 21mm. The Distagon killed the 24mm in f/stop to f/stop testing but when compared side by side in an extreme near/far DOF test the 24mm was sharper hands down. I always irritates me when the TS-E’s are tested as they are fixed lenses, which they are not. When using them as they are intended nothing comes close for landscape photography. And the new 17 & 24mm TS-E’s are by far the sharpest WA lenses available even tested f/stop to f/stop.

  5. Scott Bacon says:

    Many thanks for posting this. As you noted, its not the “end-all-be-all” for comparing/selecting lenses, but it certainly provides additional and very useful information.

  6. QT Luong says:

    Jerry, I certainly agree with your comment about the 24 TS-E. That’s why it’s in my camera bag, although both of my zooms are sharper. However, if I do not need the movements, I use the zooms.

    Those tests were made three years ago. I have three of the new 24mm TS-E on order, and will probably report. Did you compare the new TS-E’s to lenses such as the Distagon 21 or the Nikkor 12-24 ?

  7. Jerry Greer says:

    QT Long,

    No, I haven’t compared the new TS-E lens in a direct side by side test to any other lens, I just got mine the last week of Oct. I’m just comparing overall image quality between the old and new TS-E 24mm. There is no comparison! The crazy thing is that the new 24’s sharpness is really not affected by stopping down until you get to f/22 and at that point it is sharper than the old 24mm by a huge margin. It’s really pretty sharp wide open. I can say, I was fortunate to have had a very sharp version of the old TS-E 24mm but the new one is a huge step beyond. And, did I state that this lens is CA free! NO CA!

    Check out this review and test with side by side images at most used focal lengths http://tinyurl.com/q7wfwj

  8. Mike says:

    I had realy good luck with the 70-300 IS as well. However, I found the long end lacking. But from 70-200 it was amazing. I eventually grabbed a 300 prime to do the long end.

  9. i own several Canon L Lenses and they have the best quality. even better than Nikon.:,’

Leave a Comment