Standards and Data Formats

IGS files

The IGS standards define a number of files to be used as aids in the exchange of information for PNT. The RINEX standard defines three file types (observation, navigation and meteorological) but other RINEX like files (defined in different Standards but inspired by the original RINEX) define files for more specific activities stemming from GNSS use. These include clock monitoring, Ionospheric and Tropospheric data and Antenna characterisation, among others.

New naming convention for IGS files

From the beginning of GPS week 2238 (27 November 2022) the IGS will adopt for its operational products the same conventions and models as used in its third reprocessing campaign (repro3; see here). This includes in particular the adoption of new long filenames for the IGS products. Files previous to this data will not see any modifications to their names.

As part of the evolution of the IGS it was deemed necessary to adapt the product names to using longer designations rather than the 8.3 names inherited from initial OS naming limitations. In order to still be able to process IGS files users will have to modify their processing software to recognise files according to the new naming conventions. A good example of the changes comes from the clock files: up until now they consisted of an 8.3 name (e.g.jpl22280.clk). From now on they will have the format AAAVPPPTTT_YYYYDDDHHMM_LEN_SMP_[SSSSMRCCC_]CLK.CLK (see  the adoption of new long filenames for a detailed explanation)

This modification was launched at the same time as the change to a new reference frame and new calibration files. Please see Reference Frames section at the end for details.

RINEX files

The Receiver Independent Exchange format (RINEX) supports the exchange of key GNSS information for PNT such as broadcast ephemerides, code, carrier-phase and doppler measurements, signal-to-noise ratio, observation time and station meteorological data. This information is delivered in three ASCII file types, namely Observation Data File, Navigation Message File and Meteorological Data File. Since its initial definition, the RINEX format has evolved and improved alongside the evolution of GNSS. For instance, the last RINEX 4 version radically modernises the Navigation Message, breaking RINEX 3 backwards compatibility. Starting with RINEX 4, all RINEX file types also support FAIR data usage (Finding, Accessible, Interoperable and Reusable) through optional headers.

As RINEX files are uncompressed ASCII files, to facilitate storage and transmission without sacrificing readability Yuki Hatanaka developed an algorithm to compact RINEX files into a human readable ASCII format. A Unix compressed Hatanaka compacted file can reach 39% of the size of the Unix compressed file.

At present, GSSC ingestion processes support RINEX 2, 3 and 4 versions (as well as the compressed and Hatanaka compacted versions) associated with the following Resource Types ‘GNSS Observables’, ‘Broadcast Ephemerides’, ‘Meteorological Data’ and ‘Satellite and Station Clock Products’.

RINEX-like files

RINEX clocks files

RINEX Clock files contain the solutions for station receiver and satellite clocks. The solutions consist of the offsets due to clock synchronisation errors and contains the receiver clock offset estimated in the clock reference frame as well as the offsets for the satellite clocks including relativistic clock corrections. In addition to the offsets, the drifts and accelerations for the clocks may also be provided, although only the offset is mandatory. Together with each of these values the standard deviation must also be provided.

Most common sampling rate for clock data is 30s and 5 mins although other sampling rates are also possible. RINEX clock files are extremely important in POD and Genaral Relativity tests.

The GSSC is able to ingest all existing versions of the RINEX clock Standard (see a complete list of versions past and present here) as ‘Resource Format’ for the ‘Station Clock Products’ resource type.

IONosphere-map EXchange (IONEX) Files

As they travel through the ionosphere, GNSS signals get distorted by the effect of the ionised plasma the ionosphere is composed of. Roughly speaking, navigation signals will get delayed and their direction will change as they pass through the ionosphere. The magnitude of the effect depends on the elevation of the satellite above the horizon and the position of the Sun in the sky. It should therefore come as no surprise that information on the ionosphere can also be gleamed from GNSS signals. 

Analysis of the GNSS signals as they arrive at a station on the ground can therefore give us very important information on the state of the ionosphere. Being essentially a plasma, the important parameter here is the density of electrons. IONEX files contain the so called TEC (Total Electron Content). TEC is nothing but a measure of the number of free electrons in a column through the ionosphere with a cross-sectional area of 1 square meter. For reasons of convenience a density of 1016  is one unit of TEC.

IONEX files contain apart from TEC maps, RMS error maps for the TEC maps and also height maps.

The GSSC supports the current IONEX Standard IONEX v1.0 as ‘Resource type’ ‘Ionospheric products’.

ANThenna Exchange format (ANTEX) files

A precise characterisation of the antennas used to receive GNSS signals is a crucial element in the use of GNSS systems. Anthena manufactures always provide detailed information of their products (at least reputable manufacturers) but the formats used to convey the information are not always compatible from one manufacturer to another. This greatly complicates the comparison of different products and makes analysis a painful task. It is here that ANTEX files play an important role. 

ANTEX files provide characterisation both for the satellite antennas and receiver antennas. The most important pieces of information provided by the files are the phase centre offsets (PCO) and the phase centre variations (PCV). Any errors in the PCOs and PCVs will translate into positioning and timing errors for any navigation application. Importantly enough, ANTEX files also provide some information on how the calibration was performed, who did it (agency or lab) and the serial numbers for the antennas tested. In addition ANTEX files will also provide phase patterns for the antennas included in the file.

Extended Standard product (SP3) files

SP3 files (SP3 a, SP3 c, SP3 d) contain detailed orbit information for the satellites in different GNSS systems. The latest version of the Standard (SP3-d) is the evolution of the initial Standard (SP1) taking into account user feedback and the reality of more GNSS satellites in orbit. The information contained in the SP3 files consists of precise orbit and precise clock corrections with the associated errors for the satellites in the files. The current version allows all GNSS satellites to be included (maximum number of satellites per files was increased to a number allowing the inclusion of all GNSS satellites currently active) and also SBAS orbits.

The GSSC recognises all versions of the SP3 Standard as resource types ‘Station/Receiver Position and Velocities’ and ‘Precise GNSS orbits’.

Geostationary Satellite-based Augmentation Systems (GEO SBAS) files

The GEO SBAS file contains the binary data broadcast by the payload of one or more GEO satellites part of an SBAS system. The file does not contain any data on the GNSS part of the system which needs to be sought after in the other file types described here.

Solution (Software/Technique) INdependent Exchange (SINEX) files

The SINEX format was initially developed by the IGS for their weekly solutions for station position and velocity since around 1995. At some point both the International Laser Ranging (ILRS) and the International VLBI Service (IVS) started using it because the design of SINEX was modular enough to be easily adapted for techniques other than GPS. To meet all the requirements of these three techniques more elements were added and in 2002 anew version 2.0 finalised. The current version (2.02), resulted from the addition of a list of parameters prompted by the requirements of GALILEO.

All the versions of SINEX are supported by the GSSC as Resource Types ‘Station/Receiver Position and Velocities’.

SINEX_TRO files

It is not difficult to guess SINEX_TRO files contain tropospheric data in a format derived from SINEX. The first version of  SINEX_TRO (0.0.1) was developed in 1997 to allow the exchange of data containing the series of total zenith path delay transformed to precipitable water vapour. As made clear by the name these files inherited some of the structure of the plain SINEX files.

Due to the lack of Standardisation a few variations the original tropo SINEX coexisted with different fields and the impossibility of creating a single software package to deal with all flavours of TROPO SINEX files. The situation started to change when new types of date called for a more unified treatment: Parameters from different sources either than save geodetic techniques (e.g. numerical weather prediction models), products including slant tropospheric delays, long term series of individual stations and even formalities like long station names (9 characters) in agreement with the RINEX 3 Standard.

Initially SINEX_TRO was accompanied by a companion SINEX file. As of the new Standard v2.0, SINEX_TRO is a standalone file contained all the information required to accompany the tropospheric data.

SINEX_TRO is compatible with the GSSC as is processed as resource type ‘Tropospheric Products’

SINEX for GNSS BIASES V1.0 (SINEX_BIAS) files

SINEX_BIAS files provide GNSS differential code biases for satellites and stations. Its origins lay on a 2011 proposal to handle GNSS bias estimates for Galileo. The proposal (SINEX_BIAS V0.01) was similar to the SINEX_TRO one in the sense that it was also based on the SINEX format. After the format was modified by several additions and modifications only two blocks remain from the original SINEX inspired format (V0.01). The current version (V1.0), was finalised in 2016.

In addition to the biases and standard deviations slopes an their standard deviations may also be provided.

SINEX_BIAS files are ingested by the GSSC as ‘Differential Code Bias’ resource type.

Earth Rotation Parameters (ERP) files

The Earth Rotation parameters files contain the GNSS only daily estimation for the displacement of the North Pole and UT1 induced by the rotation of the Earth.

The GSSC ingests ERP files as ‘Earth Orientation Parameters’ resource type.

ILRS

The International Laser Ranging Service (ILRS) is responsible for the organisation of Satellite and lunar ranging campaigns resulting in data that is used in support of satellite orbit determination, geodesy, lunar science and fundamental physics (monitoring and determination of the fundamental constants).

The ILRS data available from the GSSC is provided in two formats:

CRD

This file format is an evolution from older formats taking into account the adoption of new technologies. The technology drivers are the increased adoption of kHz firing-rate laser and anticipated transponder missions. The former made the use of the older format with full rate data a bit cumbersome and the later had some problems with fields (some had to be created), and some were too small to accommodate then new data.

Instead of patching the old format to accommodate for the three types of data provided, the ILRS decided to go for a a completely new format to accommodate the three data types: full rate, sampled engineering and normal points.

The file contains three sections including the header with information on the station in use, target and session, information on configuration including detector configuration and clocks and, at last, the data itself.

The GSSC supports two resource types for CRD data: ‘SLR full rate data’ and ‘SLR Normal point data’.

The latest version of the Standard can be found here. This version (v2.01) superseded v1.01 in the 1st of August 2022.

Example Code to read/write and manipulate CRD files can be found here.

CPF

Again, for the same reasons mentioned above for then renewal of the predictions files, the ILRS realised the older format could no longer efficiently support then prediction data. Here another factor had to be taken into account: with the increase number of satellites in GNSS constellation the ILRS had to optimise orbit predictions so that a schedule that could accommodate the larger number satellites in need of observations could be easily accommodated. The result was the CPF files: the data contained in these files is the data used by the ILRS stations to schedule observations.

The file contains several headers with general information about the station, the clocks used, the lasers and of course also the predictions for satellite visibility for the relevant stations.

An old version (v1.0) for the CPF files format can be found here and the new version (v2.0) here. V2.0 supersedes V1.0 since the 1st of October 2021. Example code for file creation and edition is available here. File examples can also be found here. 

The GSSC uses resource type ‘Predicted Orbits’ to handle CPF files.

International Earth Rotation and Reference Systems Service (IERS) files

The IERS is responsible for providing accurate data on the earth rotation and also accurate definition of several Reference Frames essential for accurate positioning on the surface of the Earth. Because of its close interplay with GNSS data, it is perfectly natural to also include those in the GSSC.

Earth Orientation Parameters files

The Earth Orientation Parameters are the rotation part in the coordinate transformation from the International Terrestrial Reference Frame to the International Celestial Reference Frame. The orientation parameters provided are computed from a combination of data obtained though DORIS, VLBI, SLR and GNSS (available from the IGS produced Earth Rotation Parameter files as discussed above (Sec. 1.3)). It should be obvious for every reader who got this far in the GSSC that without a precise determination of the Earth Orientation Parameters, precision PNT is not possible.

Earth Orientation files contain displacements of the pole in arcsec, associated errors and nutation and precession of the pole and associated errors as well as UT fluctuations.

Different types of data can be provided quick look analysis results, the final daily values and predictions. Also, comparison is performed with different IAU1980 and IAU2000 models.

Details, formats and plots of the current time series can be found here. The explanatory supplement for the Earth Orientation parameters is available here. This document contain more technical details on the preparation of the Earth Orientation Parameters Files. 

Other Formats

NORAD Two Line Elements Sets (TLE) files

TLE files describe in an extremely compact way (only two lines) the approximated trajectories of any objects orbiting the Earth. They are designed to provide input to NORAD’s SGP, SGP4, SDP4, SGP8 and SDP8 simplified perturbations orbital models. Using the correct projection formula the orbital data provided in the files can provide the state (position and velocity) of the relevant object in the past or the future. 

Interestingly enough the format was designed to be input to a computing apparatus as a series of three punch cards.

The Two line element contains in fact three lines:

  • Line 1: Name of object (max 24 characters)
  • Line 2: Miscellaneous information on ephemeris and drag coefficients. The last field is a check sum
  • Line 3: Orbital elements. The last field is a check sum  

These models are widely used in not only in the prediction of orbital tracks as well as in the the prediction of future tracks of orbital debris and provide great help in characterising risks and plan collision avoidance manoeuvres.

Networked Transport of RTCM via Internet Protocol (NTRIP)

NTRIP is a protocol used by Real-Time Kinematic (RTK) positioning, a method used by an RTK-enabled GPS receiver to obtain extremely precise positions by using data from an RTK base station (a second GPS receiver) that is transmitted over the internet.

RTK uses a fixed base station broadcasting regularly on the internet accurate corrections to its position to any rover (e.g. a user’s mobile phone) listening in order to allow the rover to get its position to 1 cm accuracy. On the hardware side we have a GNSS system working in a normal GNSS configuration, a user’s device (for the purpose of NTRIP called the rover) and a fixed baseline station. On the software side we have a NTRIP server associated with the base station running the NTRIP caster broadcasting information from the base station and on the client side, associated with the rover we have the NTRIP client listening to the information broadcast by the server.

Satellite files

The GSSC offers not only GNSS data and products generated by several of the GNSS systems available to users worldwide but special datasets produced by a number of space missions with GNSS receivers on board. For these datasets it is important to note what you will not find here: this is not a duplicate of the archives for these missions and here you will only find datasets related to the PNT functionalities of the these missions.

Challenging Minisatellite Payload (CHAMP)

CHAMP was a German satellite launched July 15, 2000 from Plesetsk, Russia, finishing operations on 19 September 2010. The satellite was was used for atmospheric and ionospheric observations, as well as other geoscientific applications, such as GPS radio occultation, gravity field determination, and studying the Earth’s magnetic field.

At the GSSC we store the summary files and the Hatanaka compressed observation files (see above).

Gravity Field and Steady-State Ocean Circulation Explorer (GOCE)

GOCE was the first of ESA’s Living Planet Programme satellites. Its main mission was to map in unprecedented detail the Earth’s gravity field. For that end, GOCE’s primary instrumentation was a highly sensitive gravity gradiometer used to measure gravitational gradients along three orthogonal axes. In addition, GOCE also had a GPS receiver in its payload and generated some interesting navigation data available through the GSSC.

The GSSC distributes two Level 1b different file types belonging to the GOCE data set: 

  1. GO_CONS_SST_RIN_1b*.DBL This is simply a RINEX Observation file
  2. GO_CONS_EGG_NOM_1b*.EEF. This is an Earth Explorer Header

Note the CONS in then filename stands for consolidated (there are other types of data in GOCE).

The GOCE L1B products user handbook contains very useful information on formats and some technical information on their production and use.

RINEX Observation file

In the RINEX files GO_CONS_SST_RIN_1b*.DBL, the .DBL extension stands for data block. It should however be noted this file is fully compliant with the RINEX Standard v2.20, apart from the name.

SST refers to the satellite to satellite tracking a GPS-based high-low satellite-to-satellite tracking.

Electrostatic Gravity Gradiometer (EGG) file

The EGG stands for electrostatic gravity gradiometer, the main instrument of GOCE measuring gravity gradients in three orthogonal directions and the extension .EEF refers to the Earth Explorer Format. This is an xml file containing the raw measurements from the accelerometer, instrument-calibrated and corrected and the gravity gradients in the instrument reference frame.

Ice Cloud and land Elevation SATellite-1 (ICESAT-1) files

ICESAT was a NASA satellite mission for measuring ice sheet mass balance, cloud and aerosol heights, as well as land topography and vegetation characteristics. It was launched on 13 January 2003 and placed into a near-circular, near-polar orbit with an altitude of approximately 600 km. It was de-orbited with a series of manoeuvres between the end of July and middle of June 2010. Note there is also a follow up satellite ICESAT-2 launched in 2018. At the moment we do not keep any data from this satellite. From ICESAT-1 we store RINEX Observation files, position solution from receiver and ICESAT attitude as quaternions.

  • ice1xxx0.10o.Z (ICESat GPS data in standard RINEX format)
  • ice1xxx0.10pc.Z (ICESat position solution from receiver)
  • ICESAT_ATTITUDE_2010xxx.txt (ICESat attitude data as Quaternions)

XXX is the DOY associated to the files.

Note the files on the last bullet only cover the period from DOY 196 to 226 corresponding to the de-orbiting manoeuvres.

JASON

Jason is a series of satellites jointly operated by NASA and CNES for Jason 1 later joined by NOAA and EUMETSAT for Jason 2 and Jason 3.

The Jason missions are dedicated to oceanography, and in particular to oceanic altimetry ocean topography and circulation.

The files stored by the GSSC consist of the RINEX 2 observation files.

Links to Jason 2 and Jason 3 data archives and handbooks are available here.

Satelite de Aplicaciones Cientificas-C, or, in English, Satellite for scientific Applications (SAC-C)

SAC-C was an internacional colaboraron led by Argentina with a number of biosphere and environment goals including the provision of multispectral images of the Earth in order to monitor the condition and dynamics of the terrestrial and marine biosphere and environment, the development and use of new GPS-based techniques to globally measure atmospheric phenomena with a special interest in long-term climate change, study Earth’s magnetic field and the Sun-Earth interaction and, at last, measure the high energy radiation environement and correlate it with the degradation of electronic components in orbit.

SAC-C also carried a GNSS radio occultation receiver as a demonstration payload.

The SAC-C data stored by the GSSC consists of the summary files and the Hatanaka compressed observation files.

SWARM

SWARM is an ESA three-satellite mission dedicated to the study of Earth’s Magnetic field. The GSSC archives navigation and orbit related files produced by the mission. As the three SWARM S/C have GPS receivers on-board it makes sense to store this data in the GSSC.

The GSSC stores files containing dynamics products for SWARM as well as attitude files

  • S/C Dynamics Products (*SC_xDYN_1B*.CDF.ZIP) 
  • Attitude of S/C (*STRXATT_1B*.CDF.ZIP)

where x designates the S/C (A,B or C) and 1B means these are level 1B products. On top of these SWARM specific formats the GSSC also stores the observation and sp3 files for the three S/C. 

More details on SWARM data including formats descriptions can be found in the SWARM products handbook. In particular the level1B is documented here.

DYN files

DYN files contain auxiliary data for precise orbit determination and non-gravitational accelerations. The x stands for the satellite number (A,B or C) and 1B means this is a level 1B product.  Note this is a compressed CDF file. In fact the structure of the files is a bit more complicated. Inside the ZIP file there are two files: a .HDR file containing the headers for the product in xml and a Common Data Format File (CDF) containing the product proper. Note there is another cdf file called computable document format created by Wolfram inc. This last file is completely unrelated. Please do no confuse the two!

The DYN files are produced daily and the data rate is 1Hz. The total file size is approx 3.7 Mbytes/files.

ATT files

These files contain the attitude of the S/C. The structure of the content of the .ZIP file is the same as for the DYN files above. The files are also produced daily and and the data rate is 1Hz. In this case the approximate size of each files is 2.7 Mbytes files.

RINEX Navigation, Observation and SP3 files

The SWARM data distributed by GSSC also includes navigation, observation and sp3 files. These files are discussed under Sec. 1.2 above. Their content is the same as for normal GNSS files.

Datalabs

In addition to the access to GNSS data, the GSSC provides also access to datalabs. The concept of datalab is very simple. In an age of bigger and bigger datasets, it becomes unfeasible to transfer datasets of many hundreds (or more!) of GBytes across a network to the user’s computer where a processing application resides. In this case the solution is to move the application to where the data is. A datalab is nothing but an application that runs in close connection with a data repository in such a way that instead of moving data to the application, the application itself is moved to the repository. This concept is a simple but powerful one when one takes into account a large number of applications can can be transformed into a datalab.

All this requires a careful management of resources in the host but nothing that cannot be handled with modern technology. For special cases where users required unusually large resrouces (be it either CPU or CPU tieme, memory or disk space) it should be possible to provide these temporarily to users as long as their requests are well justified.

Although still in the Beta stage, the GSSC already has quite a few datalabs available, from simple applications to visualise RINEX files, to complex applications like gLab, a RINEX file processing application used frequently in learning contexts, or even full fledged programming languages such as Octave (a scripting language which works pretty well as a Matlab replacement in some contexts), Jupyter, or even Python.

One of the few limitations on making a piece of software a datalab is the license under which the software is released. For obvious reasons commercial software cannot the provided as a datalab. Apart from that there are few limitations to transforming a standalone piece of Software into a datalab.

In the near future, it is foreseen users of the GSSC will be able to install and share their own datalabs with collaborators either in a school or University project or for complex research. For especially important datalabs that could be useful the community at large, it should be possible to give them specially enhanced visibility so that the community can benefit from them. Of course this can only be done with the permission of the authors of the application.

It is also possible for users get their own datalabs installed in the GSSC. In order to do so, they should contact GSSC Support stating their use case and they will be guided through the process.

Reference frames

From the beginning of GPS week 2238 (27 November 2022) the IGS will use a new reference frame, IGS20, as the basis for its products. IGS20 is closely related with ITRF2020, released in April 2022. This will change be accompanied by a set of satellite and ground antenna calibration, igs20.itx, made effective at the same tine. Users are reminded this new calibration file should be used together with ITRF2020.

The announcement can be found here.

Other useful resources

A list of IGS file formats can be found here.

The Research group of Astronomy and GEomatics at University Politecnica de Catalunya is a very good source for many GNSS related topics, including a GNSS course with very useful examples, a very good software suite called gLab (also available as a fully functional Datalabs in the GSSC) accompanying the course and a number of hands-on examples of RINEX and RINEX like files cleverly commented in html. This is probably the best tools available to learn RINEX files.