How to Calculate Precision

Precision and accuracy, though often used synonymously, are actually very different words in math and science. Precision means that a measurement gets similar results every single time it is used. For example, if you step on a scale five times in a row, a precise scale would give you the same weight each time. In math and science, calculating precision is essential to determine if your tools and measurements work well enough to get good data. Luckily, calculating precision is pretty easy.


  1. Image titled Calculate Precision Step 1
    Know the difference between precision and accuracy. Precision measures how well your tools are working, not what the tools are measuring. Accuracy checks how "right" your answer is. For example, if you weigh a 20 lb (9.1 kg) weight and your scale says 19.2 lbs (8.7 kg), then your scale is not accurate. If your scale says 19.2 (8.7 kg) every single time you weigh the weight, it is still precise, though not accurate.
    • Think of the two words in terms of archery: Accuracy is hitting a the bulls-eye every time. Precision is hitting the same place each time, even if it is not the place you aimed for.
  2. Image titled Calculate Precision Step 2
    Record a series of measurements. To calculate precision you need data on something. For example, if you want to check the precision of your scale, you could stand on it and record the weight reading 15 times.
    • You must take multiple measurements of the same thing under the same conditions to calculate precision. You cannot weigh 10 different people and compare the results.
  3. Image titled Calculate Precision Step 3
    Find the mean of your data. In order to make sense of the changes in precision, you need to compare your data to something. The mean, or the average, is the center point of your data and makes a good yardstick. To find the mean, add up all of the measurements you took and then divide it by the number of measurements.If, while weighing yourself, you recorded the weights: 12cm, 11cm, 14cm, 13cm, and 12cm, your mean would be:
    (12lb + 11lb + 14lb + 13lb + 12lb) / 5 = 62 / 5 = 12.4lb

    In other words, the average weight recorded was 12.4 lb. (10.8 kg).
    • You can also use known numbers instead of the mean. For example, you could use a 10 lb (4.5 kg) sack of potatoes and compare your numbers to that.
  4. Image titled Calculate Precision Step 4
    Use "standard range" for a simple precision calculation. The range is the simplest way to estimate precision. To calculate it, you just take your highest value, and subtract your lowest value from it. For the example above, it would be 14 lb (6.3 kg) - 11 lb (4.9 kg)= 3 lb (1.4 kg). So you could report that the object you measured was 12.4lb ± 3lb.
    • The "± 3lb (1.4 kg)" is your measure of precision. It means, "this scale is only precise within a 6 lb range," or 3 lb (1.4 kg) higher, 3 lb (1.4 kg) lower.
    • The "±" symbol should be read as "plus or minus."
    • This method, while simple, is the most general way calculate precision and is not, ironically, very precise.
  5. Image titled Calculate Precision Step 5
    Calculate the "mean absolute deviation," to find precision more carefully. Look again at the mean of our data: 12.4 lb (12.6 kg). Subtract each individual measurement from the mean to find out how far off each measurement was from the center. Make any negative number positive. For example:
    Data: 12, 11, 14, 13, 12. Mean/Average: 12.4

    Difference from the Mean: -.4, -1.4, 1.6, .6, -.4"

    Now, take the average of these numbers to find out, on average, how far away each measurement was from the center:
    (.4 + 1.4 + 1.6 + .6 + .4) / 5 = .88
    his means that, in general, every single measurement you make could be ± .88lbs different from what you see.
    • You must ignore the negatives, or else they could accidentally cancel out. Remember, both 11.4 and 13.4 are only 1lb away from the mean of 12.4, just on opposite sides.


  • If one of your trial values is much higher or lower than the rest of your values, do not exclude this number from your calculations. Even if it was a mistake, it is data and should be utilized for a proper calculation.
  • Perform more than five trials to achieve a more accurate calculation. The more trials you run, the closer you will get to a clear precision value.

Things You'll Need

  • Paper
  • Pen
  • Calculator
  • Tape measure
  • Pencil

Article Info

Categories: Mathematics