Lens Rating System

Photography is part art, part science; choosing and using lenses similarly blends subjective and objective views.

The ultimate merit of a lens is arguably best measured by its ability to become invisible, to deliver unadulterated reality to the film plane. However, lenses by definition offer the camera an interpretation of reality. They are designed according to constraints and balances – notably size, cost, and ease of manufacture. They are tuned in harmony with ‘house styles’ that represent the historically prefered compromises of Carl Zeiss, Yoshihisa Maitani, Lothar Kölsch et al.

From this we might say that there is no such thing as the ‘best’ lens: only what best suits your aesthetic preferences. On the other hand, lens designers seek shared goals of minimal aberration and distortion; and maximal resolution and accutance – measurable phenomena amenable to objective rating. This latter aspect of lens behaviour is well dealt with by testers such as CastlemanSchroff and (apparently) Jesus Christ, as well as in the independent MTF testing published by photodo.

However, sharpness is not the only factor. And numbers aren’t good at describing pictures to us. Is the gap between 1,850LW/PH and 1,925LW/PH enough to worry about? Will I see the difference between a 3.9 and a 4.1? That’s why 16:9 is based on complex real world images (not simple, unrevealing charts), demonstrating multiple aspects of lens behaviour using the only measuring system that matters: your eyes.

Having said all that, I felt it would be useful to supplement the pictures with a ready-reckoning system that grapples with this complexity. The final mark awarded to each lens is a summary of its performance over the broadest range of competence and is a fair measure of its overall efficacy. However, the percentage mark is itemised to allow you to make your own judgment according to your own criteria: if you shoot landscapes, observe the Bright Light Handling and M&S grades and ignore Geometric Distortion; if you shoot architecture, your priorities will likely be reversed.

‘Sharpness’ :

90 points
A maximum of five points is awarded for each area of the frame: Zone A (0–7mm from frame centre) , Zone B (8–16mm from frame centre) and Zone C (17-22mm from frame centre) . Apertures from f2.8 to f16 are tested. The grade takes into account resolution, accutance and vignetting, though the latter is not deemed critical because of the ease with which it is corrected. Naturally, sharpness is prized by all photographers: information in a scene can only be altered in post-production, never added. With this in mind, resolution is considered more valuable than accutance.

Chromatic Aberration :

10 points
Chromatic Aberration is allocated just 10 points out of a maximum of 150 because it can easily be remedied without deleterious side-effects. A single mark summarises the frequency and severity of its occurence across the frame.

Geometric Distortion :

15 points
Geometric Distortion impairs images and can be impossible to correct without reducing the effective resolving ability of the lens. It therefore merits 10% of the overall mark. Penalties are given for the severity and pattern of the effect: complex (waveform) distortion is particularly undesirable if maximum fidelity is required.

Bright Light Handling :

10 points
A single mark out of 10 is awarded for Bright Light Handling, assessing the lens’ resistance to flare, susceptibility to internal reflection (‘ghosting’) and its ‘hoshi’ – the type of star pattern generated by point light sources such as naked bulbs. Because this is not pertinent to all genres of photography, you may need to adjust the mark to better suit your priorities.

Magic & Sparkle :

15 points
As the label suggests, M&S is the most subjective category, but crucial nonetheless. The mark out of 15 covers all manner of mojo: plasticity (3D ‘etching’ of the image on the sensor), colour (neutrality and high saturation are favoured), bokeh (smooth, unfussy rendering free from double lines or other artefacts on defocused point light sources) and general ‘pop’: that special, intangible quality that makes an image sing.

Speed Rating :

10 points

Points are awarded as follows:
f1.2 – 10 points
f1.4 – 9 points
f1.8 – 8 points
f2 – 7 points
f2.4 – 6 points
f2.8 – 5 points
f3.2-5 – 4 points
f4 – 3 points
f4.5 – 2 points
f5+ – 1 point

Overall Rating (%) :

A final percentage grade is calculated on the basis of the marks out of 150 accrued in all areas of the lens’ performance.

Note that this system solely address lens performance at a given focal length. One of the knottier aspects of comparing lenses lies in the variable behaviour of zoom lenses across their range. As a general rule, aberrations are lowest in the middle of the zoom range, but resolution peaks at one extreme: typically the shortest. Most of the current crop of ultrawide zooms predictably follow a pattern of strong barrel distortion in the widest 20% of their output, rising to a sweet – but fleeting – moment of perfect projection, before reaching pincushion distortion at the long end. But midpoints vary . . . so, at 14-15mm, the Canon 11-24L is beautifully square, whereas at that focal length the Nikon 14-24mm is inflating it’s images with a 4% bulge. But it’s sharper there than at 18-20mm, where it reaches zero distortion. See the article assessing how lossy geometry correction is . . .

Lenses vary.

Sometimes the best prime is a zoom. But the devil is in the detail. For instance, you could make a case for the Sigma 24-35mm f2 as the finest 28mm available in 2018, because it peaks peculiarly at that focal length. When comparing the Olympus 7-14mm f2.8 and Panasonic 7-14mm f4 you have to remember that the Panasonic is strong on the wide side and weak on the long; whereas the Olympus is exactly the opposite. Primes are simple.

However, given the proliferation and excellence of current ultrawide zooms – and the apparently passé status of ultrawide prime – it might not be an extravagance to own two such zooms, mindful of their complementary skills: the Nikon 14-24mm is a fantastic 17mm prime, with some bonus coverage either side. The Canon 11-24mm is the best 14mm prime money can buy; not so competitive above 20mm.


Leave a Comment