Equations 1-5, Charles Ghiliani, Error Ellipses, xyHt July 2017

Error Ellipses

This entry is part of 8 in the series July 2017

Part 2: Components and Increasing Probability

In the previous article (April 2017) I discuss the correlation of the coordinate values for a station and how an error ellipse is used to determine the direction of the largest error at a station. In this article I review the components of an error ellipse, the process that is used to compute the components of an error ellipse, and how to increase the probability of the standard error ellipse to the required 95%.


Figure 1, Charles Ghiliani, Error Ellipses, xyHt July 2017

Figure 1: Components of the standard error ellipse.

To review, the components of an error ellipse (Figure 1) are:

• the t angle, which is the amount of rotation required to remove the correlation between the two coordinate values for each station in a horizontal/plane survey,

Su which is the length of the semimajor axis of the ellipse and represents to greatest error at the station, and

Sv which is the smallest error at the station.

In regards to the ALTA/NSPS standards, we are only concerned about the size of Su at a 95% probability level for each station.

As discussed in the Part 1 of this article, to obtain the maximum error at a station, the correlation between the pair of coordinate values must be removed. This can be done using a two-dimensional rotation and error propagation. While the actual computations are not important, the resulting values used to compute the error ellipse are of importance to this discussion. In this process recall the covariance matrix shown in Part 1 and shown again in Equation 1.

Equations 1-5, Charles Ghiliani, Error Ellipses, xyHt July 2017To determine the direction of the maximum error at this station, which is the Su axis, the covariance matrix elements for this station must be rotated by the amount shown in Equation 2 where:

Sne is the covariance at the station, which is the off-diagonal element 0.021460,

See is the variance in the easting, which is 0.048736,

Snn is the variance in the northing, which is 0.022169, and

• C, which equals 180° in this case, was added to the results of the arctangent function, which was −58.243°, to obtain a positive rotation angle. Note that −58.243° is the amount of counter-clockwise rotation going from the N axis to the V axis.

Once the rotation angle is determined, the variances of the semimajor axis, Suu and semiminor axis, Svv can be computed as in Equation 3.

In this example Suu is 0.060691, whose square root is the length of the semimajor axis for the ellipse, Su, being ±0.246, and Svv is 0.010214, whose square root is the length of the semiminor axis for the ellipse, Sv, being ±0.101. Again the computations are not important since this is a standard component of most least-squares packages.

Error Ellipses of Higher Probability

The previous computations yield what is known as the standard error ellipse. The probability that the true coordinates for B are in the error ellipse are relatively low. That is, there is only a 35% to 39% probability that the true coordinates for station B are in the area bounded by the standard error ellipse.

Figure 2, Charles Ghiliani, Error Ellipses, xyHt July 2017

Figure 2: (a) A wire diagram of a bivariate distribution and (b) the isogonic lines of a contour plot of (a).

Typically, standards like the ALTA/NSPS standards want a higher confidence level on the region that contains the true coordinates. This simply means we need to select a higher probability from the bivariate distribution as shown in Figure 2.

This can be achieved with a multiplier obtained from the F distribution value (xyHt, July 2016), and is given as in Equation 4 where multiplier is dependent on the number of degrees of freedom, or redundancies, that are in the adjustment. The higher the number of redundant measurements, which means the higher the number of checks on the measurements, the lower the multiplier.

For example, suppose the adjustment had four redundant observations. To determine the size of the 95% error ellipse, we will need the critical value from the F distribution for F1−0.95,2,4 which is 6.94. This means the multiplier c will be √ 2(6.94)=3.73. Thus, using Sur, which was computed as ±0.246, the maximum 95% error at the station is ±0.918 (3.73×0.246).

Now assume that the state plane ne coordinates in units of feet for the two stations in this example are (391,043.29, 2,415,776.90) and (387,603.26, 2,416,892.70). The distance between these stations, which are property corners, is 3616.46 ft. This means that by ALTA/NSPS standards the maximum allowable error between these two stations is as in Equation 5.

Obviously, this station did not pass the ALTA/NSPS quality standards, which only allows the maximum error at any station to be ±0.07 ft + 50 ppm between any two corners, or ±0.19 ft in this instance.

Often the critical F values can be determined for any number of degrees of freedom from an adjustment in spreadsheets. In Microsoft Excel, the command is “=FINV(0.05,2,dof)” where dof is the number of degrees of freedom in the adjustment, which is the number of redundant observations.

Table 1, Charles Ghiliani, Error Ellipses, xyHt July 2017

Table 1

Table 1 list some critical F values along with their multiplier c for various degrees of freedom. Since a simple closed traverse only has three degrees of freedom, this means that standard practice traverses will always have multipliers of 4.371. This multiplier will make meeting the ALTA/NSPS accuracy standards extremely difficult to reach.

Since the number of degrees of freedom is critical to obtaining a low c multiplier, I suggest that you make a habit of always obtaining the backsight distance when you are performing a traverse. This process takes very little extra time since it is typically part of the setup in your controller software. These extra distance observations should be entered into the least-squares adjustment as observed and not averaged with their forward counterparts.

Additionally, you can always close the angular horizon at every traverse station to increase the number of redundant angle observations. This may seem like considerable extra work and time, but even for the beginning user this is not the case when you consider this against the time it takes to get to the station, set up your backsight and foresight targets, observe one angle, and tear down the setup.

I was once challenged by a second-year student who stated that the boss he had during the summer would never allow for this. After some repeated badgering by him, I stopped the class, got the same type of total station he had used during the summer, grabbed two stop watches, and timed the student while he observed the first and then the second angle.

To my pleasure he was extremely competent and quick with the procedure. In total it took him only two minutes to set up the instrument and observe the angles 1DR. Now, weigh this against the time it takes to set up, tear down, and travel between stations, and I will challenge anyone who thinks it will cost them their business.

To paraphrase a statement I have heard, it’s strange how a surveyor will cut corners when observing measurements to save time but does not see a problem with having to return to the field to uncover an error made the first time.

These practices allow field crews to catch their mistakes before they become a problem in the office. It also provides office personnel with checks on their field crew’s work. Ask yourself how many times a crew has had to return to the field to fix a mistake made earlier and compare this against the seconds it takes to measure an additional distance and angle once everything is setup. I hope this article gives you insight into the meaning and relevance of an error ellipse and how to check it against the ALTA/NSPS standards. Stated again, the error ellipse provides a region around your station where the true coordinates for the station reside at a particular level of probability.

The standard error ellipse has a probability of containing the true coordinates of the station only 35% to 39% of the time. However, using the F distribution, a multiplier can be determined to compute the size of the error ellipse for any percentage including 95%, which is required in the ALTA/NSPS standards.

For additional reading on statistics, least squares, and this subject one should turn to my book titled Adjustment Computations: Spatial Data Analysis.


National Society of Professional Surveyors. 2016 Minimum Standard Detail Requirements for ALTA/NSPS Land Title


Series NavigationThe Elusive Scope: Business Advice for Surveyors >>

Leave a Reply

Your email address will not be published. Required fields are marked *