Understanding the differences between ls algorithms and sequential filters

Understanding the Differences between LS Algorithms
and Sequential Filters
In order to perform meaningful comparisons between output from a least squares (LS)
orbit determination algorithm and orbit determination results generated using a sequential
filter, it is important to understand how the processes differ. In the most basic sense, LS
considers all measurements simultaneously to construct an orbit estimate at a single point
in time, while a filter processes one measurement at a time computing an update to the
orbit estimate with each processed measurement. One consequence of this difference is
that the state vector representing the result of the orbit determination run may have an
epoch at anytime within the fit span for a LS process, but it will always be at the time of
the last measurement for a filter. There are, however, several more subtle differences that
should be pointed out.
LS and sequential filter processes differ in the way in which data editing is performed.
LS processes may have a number of different editing criteria including gross data editors
and RMS based editing criteria. All of these methods involve examination of the
measurement residuals without regard for the uncertainty in the state. Filters have a
current estimate of the state and state error covariance available at each measurement
time and can therefore use this information to perform N sigma editing based on both
measurement noise and a mapping of the state error covariance into measurement space.
The purpose of data editing in the two processes is the same, to remove measurements
containing unmodeled errors to prevent those measurements from corrupting the orbit
determination solution. The difference in data editing techniques, however, leads to
slightly different data editing which results in the incorporation of differing information
into the orbit determination solution.
The construction of an ephemeris across a fit span also differs between LS and sequential
filter processes. In a LS process, ephemeris is constructed by simply propagating the
result of the orbit determination run using the same force modeling as was used during
the orbit determination. While this same technique is used for generating predicted
ephemeris from the filter result, the construction of historical ephemeris from the filter
result is done using a smoother. The filter runs forward in time, producing a
discontinuous ephemeris as it crosses the fit span. The filtered estimate at each point
during the fit span contains information about all earlier measurements, but no future
measurements are accounted for. The smoother starts with the filter output at the end of
the fit span and runs backwards in time mapping information related to later
measurements to earlier points in time. The result is a smooth ephemeris over the fit
span.
LS and sequential filter processes differ in the area of initialization. LS processes are
typically “cold started” with each run meaning that, other than the initial state,
information from prior runs is not used in the current run. If an a priori covariance is
specified, it is typically a very open covariance whose main function is to provide
additional stability to the solution of the normal equation. Filters are only initialized via a
“cold start” when necessary, otherwise they are started from restart records. Restart
records contain a complete description of the state and associated covariance at a specific point in time. A typical operational scenario would have each filter run starting with the ending conditions, state and covariance, of the prior filter run. Filters also go through an initialization phase during which the filter is distributing information throughout the covariance matrix. This initialization phase will require a varying amount of time depending on the orbit of the satellite. Additional information on the comparison between LS and filter can be found in Chapter 1 of the ODTK Orbit Determination: Theorems & Equations. Tracking System Calibration
In order to obtain optimal results from a filter, all of the inputs to the filter must be
realistic. It is often the case that measurement biases and variances used with LS
programs are not the optimal values for use with a filter. One way to determine optimal
biases and variances for use with a filter is by examining histograms of residuals. For the
filter to be performing optimally, the residuals should follow a zero mean Gaussian
distribution. A non-zero mean typically means that a bias adjustment is required while
adjusting the variance manipulates the spread of the residuals. The two images below
demonstrate the use of this technique to perform a calibration of range measurements
from a particular ground station. In the first image, the residuals are seen to have a
positive bias and are too bunched together. This bunching indicates that the variance on
the measurements is too high. The second image shows the post-calibration results.
Histograms of residuals may be created in ODTK by selecting the Residuals graph style
from the graph selection window on the Static Product Builder. The specific tracker and
measurement type of interest are then specified via data limiting on the Static Product
Builder input tab. These histograms do not provide any direct information as to the size
of the bias or the correct value for the variance since the residuals have all been
normalized prior to being plotted. To view the actual values of the residuals you can
create the “Measurement Residuals” report from the Static Product Builder.

A more complete description of tracking system calibration methods is available in the
document calibration.pdf available from the Printable (PDF) Manuals page of the ODTK
Help System.
Recommended Comparison Procedures for Real Data

There are a number of tests that can be performed to compare orbit determination results
based on real tracking data. First, let a definitive ephemeris be defined as an ephemeris
with very little uncertainty. Definitive ephemeredes are often constructed based on the
tracking of laser ranging or GPS data. If such an ephemeris is available, then comparisons
can be made to the definitive ephemeris. If such an ephemeris is not available, then
comparisons can be made between the outputs of the two orbit determination methods.
One method of performing ephemeris comparisons, through the use of STK, is described
later in this document. ODTK is capable of producing STK ephemeris files (.e) during
the simulation of tracking data, during filtering and during smoothing.
Post-Pass Comparisons using Definitive Ephemeris

1. Run filter/smoother and LS across the same tracking data interval 2. Difference filter-smoother results with definitive ephemeredes across fit span 3. Difference LS results with definitive ephemeredes across fit span 4. Compare differences 5. Compare smoother differences with smoother covariance 6. Compare LS differences with LS covariance Post-Pass Comparisons without Definitive Ephemeris

1. Run filter/smoother and LS across the same tracking data interval 2. Difference filter-smoother results with LS results across fit span 3. Compare differences with smoother covariance Prediction Comparisons using Definitive Ephemeris

1. Run filter and LS across the same tracking data interval 2. Propagate filter and LS results across prediction span 3. Difference filter propagations with definitive ephemeredes across prediction span 4. Difference LS propagations with definitive ephemeredes across prediction span 5. Compare differences 6. Compare filter prediction differences with filter predicted covariance 7. Compare LS prediction differences with LS predicted covariance Prediction Comparisons without Definitive Ephemeris

1. Run filter and LS across the same tracking data interval 2. Propagate filter and LS results across prediction span 3. Run filter/smoother and LS using additional tracking data over the prediction span a. Filter should be started using a restart record from the first run 4. Compare differences between filter predicts from first data interval and filter/smoother results from second data interval 5. Compare differences between LS predicts from first data interval and LS results 6. Compare filter prediction differences with filter predicted covariance 7. Compare LS prediction differences with LS predicted covariance Tracking Data Editing

Compare tracking data edited between the filter and LS. Use visual inspection of pre-
edited residuals to manually judge correct editing. Measurements are edited in ODTK
using the n-sigma editing criteria specified with the satellite object and the minimum
elevation angle associated with each tracking station. Edited pre-fit residuals are
annotated in the “Measurement Residuals” report in ODTK or can be viewed exclusive of
accepted measurements in the “Rejected Meas Residuals” report.
Residual Graphs

1. Graph measurement residuals from the filter 2. Graph measurement residuals from LS 3. Use whiteness test to compare results (Note: White noise contains no information Recommended Comparison Procedures for Simulated Data

ODTK can be used to generate simulated tracking data. The simulator is capable of:
A. Generating perturbed initial conditions for an orbit, consistent with the orbit state B. Generating an orbit trajectory using a perturbed force model, consistent with uncertainties on force model parameters. This trajectory may be written to a STK ephemeris file to facilitate ephemeris comparisons in STK. C. Generating observations of the perturbed trajectory which include white noise consistent with the measurement uncertainties associated with each tracking station. These simulated observations may be written to a file using a selected tracking data format. The orbit trajectory produced by the simulator may then be treated as a definitive ephemeris and comparisons may be performed using the procedures listed above for real tracking data. Performing comparisons in STK

Ephemeris comparisons can be performed in STK using the Ephemeris Difference and
RIC data providers associated with a Satellite object. Satellites can be created and
defined to import the ephemeris files as generated by ODTK and another orbit
determination program. In the case where a definitive ephemeris is available, a third
satellite defined using the truth ephemeris can be defined. In addition to the generation
of ephemeris differences, the RIC data provider can also provide a measure of the
ephemeris difference with respect to the position covariance. The Sigma Level data
element reports the sigma level boundary on which the computed ephemeris lies. This
information can be used to determine the validity of the covariance in the presence of a
definitive ephemeris and the equivalence of two trajectories within their stated
uncertainties. The trajectory of the satellites and an ellipsoidal representation of their
position covariance can also be displayed in STK/Advanced VO.
Covariance information may be input to STK via the STK external ephemeris file format
(.e). After the specification of the position and velocity information, a new data block is
added as follows:
CovarianceTimePos Time1 Pxx Pxy Pxz Pyy Pyz Pzz Time2 Pxx Pxy Pxz Pyy Pyz Pzz . . . The time values must match the time values associated with the ephemeris and the upper-triangular position covariance must be given relative to the J2000 reference frame. The elements of the position covariance matrix must be specified in units of m2. Examples of these files may be generated with ODTK using options on a satellite, filter or smoother. An important restriction applies when comparing ephemeris output from the filter to another ephemeris. The filter performs a state correction at each measurement which introduces a set of discontinuities into the ephemeris during the period of time when data was available. Interpolation of the ephemeris cannot span these discontinuities without introducing significant errors into the result of the interpolation. In light of this fact, comparisons against filtered ephemeris should be done at the nodes of the filtered ephemeris or by interpolating the other ephemeris. To follow this recommendation using the STK based ephemeris differencing process, select the satellite which has been defined to use the filtered ephemeris as the reference for the comparison. Position Covariance Visualization in STK/Advanced VO

An ellipsoidal representation of the n-sigma position covariance can be shown in
STK/Advanced VO for any satellite defined with an ephemeris file containing the
covariance data block described in the previous section. To enable visualization of the
covariance, go to the Covariance tab of the 3D Graphics properties of the satellite. The
Scale parameter is used to specify a multiplier, n, of dimension of the ellipsoid. The
default value is 1.0 meaning that the ellipsoid will be drawn to represent a 1 sigma
surface. Since the ellipsoid is a representation of the uncertainty in a three dimensional
random vector, the probability levels for various values of n are different than those
associated with a one dimensional random vector. The following table provides
probability levels for some values of n for one and three dimensional random vectors1.
A white paper, Position Covariance Visualization, describing the method used to visualize covariance ellipsoids is available on the Analytical Graphics web site, http:://. Update of Timely Data

The use of any OD program requires that certain timely data be present. In the case of
ODTK, this means that the Earth orientation data, solar flux and geomagnetic index and
leap second data files must cover the time period of interest. These files are updated
regularly on the Analytical Graphics ftp site, se files may be
downloaded using the ODTK LaunchPad which is available under the Help menu.
Reporting Problems

If you run into difficulties running ODTK or comparing output from ODTK to output
from your current OD software, package up all of your inputs and email them to
support@agi.com. Be sure to include your scenario (.sco) file, your tracking data file(s)
and a description of the problem. You can also call AGI technical support toll free at 1-
888-785-9973.
References

1. Gura, I.A., Gersten, R.H., “On Analysis of n-Dimensional Normal Probabilities,” Aerospace Report No. TR-0066(5129-01)-2.

Source: http://www.agi.com/resources/help/online/ODTK/ODTK/pdf/ODComparison.pdf