I suppose it is in my nature to try and resolve certain observed phenomena down to the mathematical formulae that can reliably and logically explain them. This is surprising, given that I flunked Calculus 2 three times consecutively in college and eventually had to change majors because that dog just wasn’t going to hunt.

f(x)dx notwithstanding, I did eventually graduate and go on to work as a research and development chemist for one of the world’s largest producers of consumer products.

No one was more surprised than I when a sudden impulse arose in me at lunchtime today to calculate and chart the potential error in range estimates induced by the limits of visual acuity of both the reticle and the target. Let me restate that in simpler terms: I wanted to figure out just how screwed up reticle range estimates could get for someone with normal eyesight.

Excel to the rescue

Early in my career, I found out that I had special abilities with Microsoft Excel, and that I could make it do things that most of my coworkers could not. Well, this particular task didn’t exactly eactly require mad skillz, but Excel certainly made the task easier.

First, I had to determine the constants and variables so I could build the equations that would satisfy my curiosity.

Constant 1: is the normal human visual acuity or resolution of the naked human eye, based on the density of photoreceptor cones in the retina. The standard 20/20 line on a Snellen vision test uses .33 inch tall block letters viewed at a distance of 20 feet, which subtend 5 arc minutes, or for shooters, who are probably more comfortable with Minutes of Angle, 5 MOA. So this would equate to the ability to resolve the details of objects approximately 5 inches tall at a distance of 100 yards with the naked eye. (One true minute of angle is more like 1.05 inches at 100 yards, but we will use “shooter’s MOA” for the sake of argument, which is simply an angular unit of measure roughly equating to 1 inch per 100 yards.)

The 5 arc minute block letter E on the 20/20 line of an eye chart looks like a letter E (to someone with 20/20 vision), and not like a blurry black square, because of the ability to discriminate two contours separated by 1 arc minute, thus:

So what constant do we use? In all honesty I don’t know, and this now demands some experimentation. However, for the sake of argument we are going to use the baseline of 1 arc minute because that is the threshold of normal ability to separate contours. This also assumes perfect glass clarity and atmospheric conditions.

Constant 2: is the range constant to calculate the distance to a target given known height in inches and in MIL’s = 27.77

Variable 1: Target size in inches (a linear unit of measure)

Variable 2: Target size in milliradians or MIL’s (an angular unit of measure equal to .057° or 36 inches at 1000 yards.)

Variable 3: Magnification power of optic, measured in “X.” 5X = 5 times magnification.

Variable 4: Range to target in yards.

Once identified, I created my spreadsheet thus:

Let me explain:

As one changes the target size in inches and the magnification in “X,” the spreadsheet does the following:

1.) Takes the information from rows 1 & 2 to convert the theoretical size of a given target in inches (in this case 18) to MIL’s at the predetermined distances.

2.) Determines the size of target necessary to resolve detail at the specified distance with the specified naked eye visual acuity (in this case 1.0).

3.) Determines the size of target necessary to resolve detail at the specified distance and magnification (in this case 10X)

4.) Determines the minimum resolution of reticle details (assuming a first focal plane reticle) at the specified magnification, regardless of target distance. This is in MIL, based also on the given visual acuity and increases as magnification increases.

5.) Determines the maximum apparent target size by adding the target’s theoretical size in MILs (#1 above) with the magnified visual acuity (#3 above)

6.) Determines the minimum apparent target size by subtracting the magnified visual acuity (#3 above) from the target’s theoretical size in MIL’s

#5 and #6 therefore represent the potential extremes of target appearance based on normal visual acuity at the specified distance and magnification.

#7 and #8 do pretty much the same thing, but for the reticle, not the target.

The range error calculates the theoretical maximum estimated range to target using the standard ranging formula of Yards = Target Size in Inches X 27.77 / Target Size in MIL’s. Range overestimate assumes the largest apparent target size and minimum reticle appearance, while range underestimate assumes the smallest apparent target size with the largest apparent reticle appearance.

Stay with me… we’re almost there…

Using those numbers we calculate the theoretically possible overestimates and underestimates in yards and the average % error to tell us just how far off we can expect to be off if we measured correctly and to the given visual resolution.

As you can see by the chart above, the normal human eye just isn’t capable of doing very well with a 10 power optic, it is pretty easy to do everything correctly and still over or underestimate enough for a spectacular miss at any distance past 600 yards or so. So how do we do better? Well, there are a few tricks, rather than measure a small reference object, try to find something bigger.

What happens if we use a 30-inch reference object instead of an 18-inch reference object?

What else can we do? Well, how about upping the magnification:

Just by doubling the magnification, we reduced the potential error another 20% But, that still doesn’t help us very much past about 700 yards.

Realistically, using a standard IDPA silhouette target at 30-inches tall (not accounting for target angle that can make it appear shorter and also throw things off), and a fairly standard optic power of 24X:

We see that at 700 yards, it’s still more than possible, it’s likely, that even with perfect optical and environmental conditions we are able to over or underestimate the distance to the target by up to 24 yards. With a .308 launching a 175gr Sierra Match King at 2600 FPS, that is a miss. By switching to a 6.5 Creedmore with a 142gr SMK at standard pressures and velocities we can buy ourselves another 100 yards tops.

Now, let’s add realism: we know that even with expensive optics, there are limits to glass clarity and there will always be some defects such as chromatic aberration or vignetting that can distort the image. Add mirage, haze, pollution, and other environmentals and that will affect our ability to resolve the edges of a target considerably. Let’s say for the sake of argument that we take these affects into account and cut visual acuity in half – or the ability to resolve separate contours at only 2 arc minutes instead of 1:

That logically doubles our average percentage of error and reduces the reliability of our estimates by about 100 yards.

At longer ranges, even with the best glass, the human eye just has it’s limits. Standard deviation kicks in at some point, so some percentage of estimates will probably be close, but realistically, it is best to verify your ranges by measuring multiple target dimensions or multiple reference objects on the same plane as the targets and average them. That may reduce the error and while it may not get you on target, it may get you closer. Even so, you should have a few other options at your disposal if possible, such as a laser range finder (which also has a percentage of error) or terrain maps.

Other than that, your best bet for any unknown distance shooting past 500 or so yards is to have a good spotter. Knowing just how far off these estimates can be is helpful (for spotters), to estimate your corrections if you don’t have a mil or MOA reticle in your spotting scope.

Call the shot, run the bolt, make your corrections and send another one!