OMBRO README FILE
Released: 16 November 2005
Date of this Document: 16 November 2005
Overview
This document provides a brief description of the OMBRO data product. OMBRO contains total column BrO and ancillary information retrieved from OMI global mode measurements using a retrieval algorithm that is based on non-linear least-squares fitting originally developed for GOME, and adapted for the OMI instrument. In global mode each file contains a single orbit of data covering a swath of approximately 2,600 km wide from pole to pole (sunlit portions only).
Average fitting precision is about 2*10-3 RMS, roughly 4-5 times what was achieved from GOME. Direct comparisons with GOME data products are not possible since BrO retrievals from GOME are no longer reliable due to the advanced degradation of the GOME instrument. However, retrieved vertical columns of ~2-4*1013 molecules/cm2 around the equator (where global BrO distributions are at a minimum) and ~1.5-2*1014 molecules/cm2 in isolated hot-spots like shelf-ice areas in polar regions during polar Spring, are in agreement with what has been observed from GOME in the past.
Release-Specific Information
| Software Version | 0.9.43 and higher |
| ECS Collection Number | 2 |
| Known Issue List | across-track striping in the data product (see OMSAO_DeStriping_README) |
Algorithm Description
The algorithm is based on the direct fitting of radiances and irradiances. In particular, and differing from what is commonly referred to as Differential Optical Absorption Spectroscopy (DOAS) fitting, radiances are not divided by irradiances, no logarithms are taken of the spectra, and no high-pass filtering is applied. The three main stages of the algorithm are (1) Solar wavelength calibration, in which the optimum wavelength registration of the solar irradiance measurements is determined and, unless pre-measured laboratory profiles are used, the instrument slit function is determined by fitting an (a)symmetric Gaussian (symmetric is the default); (2) Radiance wavelength calibration, which finds the optimum wavelength registration for a representative swath of radiance measurements (usually in the middle of the orbit) and determines a common wavelength grid for auxiliary data bases (molecular reference cross sections, etc.); and (3) Fitting of all swath lines in the OMI granule. In each stage, the calibration/fitting is performed individually for the 60 cross-track pixels1 of an OMI swath line. For improved numerical stability, radiances and irradiances are divided by their respective averages over the fitting window; in other words, they are "normalized" to values ~1.
BrO fitting is performed in the spectral window 323-347 nm, within the UV-2 channel of the OMI instrument. The model that is fitted to the measurements consists of the solar irradiance, attenuated by contributions from BrO (the target gas), inelastic (rotational Raman, or "Ring") scattering, and interferences from ozone and NO2; it also contains additive and multiplicative closure polynomials and parameters for spectral shift and squeeze (the latter is not used in nominal operations), as well as an undersampling spectrum that is computed on-line. The fit is mostly unconstrained, with the exception of selected parameters, including the spectral shift, which are constrained in order to prevent problems arising from .out of bounds. values.
The results from the spectral fitting are BrO slant columns. These are converted to vertical columns using a look-up table of stratospheric air mass factors (AMFs) that have been pre-computed using climatological BrO profiles. The AMFs used for the conversion are provided in the data product.
The algorithm employs a method to reduce cross-track striping of the BrO columns: fitting residuals from the radiance fits (Stage 3) are examined for outliers, which are defined as residual values larger than 0.005 absolute, for radiances normalized to ~1 over the fitting window. These values have been determined empirically from a statistically large number of BrO fits, and they represent the average upper limit of the spectral fitting residuals. If an outlier is identified in the fitting residual, the spectral fit is repeated with the CCD position corresponding to the outlier removed from the fit. The resulting BrO columns generally exhibit less cross-track striping, while average column values (across track for one or more swath lines) do not change significantly. The rationale for this outlier identification is as follows: CCD positions with associated fitting residuals that are significantly larger than the average residual over the fitting window are most likely due to bad pixels that were not correctly flagged and hence not excluded before the fitting process commenced. It is expected that improved bad pixel flagging will reduce - but not necessarily eliminate - the number of such outliers. Please see OMSAO_DeStriping_README for details on the destriping procedure.
In addition to the standard BrO columns as described above, the OMBRO product provides a smoothed BrO column that has been derived from the standard column by applying a cross-track smoothing filter. This product is highly experimental and should be used with extreme care. Again, please see OMSAO_DeStriping_README for details.
More details on algorithm specifics can be found in the OMI Algorithm Theoretical Basis Document Vol. 4 and in T.P. Kurosu, K. Chance, and C.E. Sioris, "Preliminary results for HCHO and BrO from the EOS-Aura Ozone Monitoring Instrument", in Passive Optical Remote Sensing of the Atmosphere and Clouds IV, Proc. of SPIE Vol. 5652 (2004), doi: 10.1117/12.578606, a PDF of which is available at http://www.cfa.harvard.edu/~tkurosu/Papers.
Data Quality Assessment
Across-track striping (systematically elevated or reduced column values at the same cross track position along the whole track) of the BrO columns is a presently outstanding issue. This is not unique to BrO but affects all OMI data products to a higher or lesser degree. Small absorbers like BrO, HCHO and OClO, however, are more strongly affected by striping since the column values are of a similar order of magnitude as the stripes, so that the effect is relatively stronger. Various efforts, both at Level 0-1 and 1-2 data processing, are under way to improve this situation, including the method of outlier identification in the fitting residual as employed in the BrO fit. A satisfactory solution remains still to be found, and users of the BrO columns provided here must be aware of this issue.
The BrO data product provides RMS and one standard deviation (1σ) fitting uncertainties, as derived from the fitting covariance matrix. These uncertainties do not include contributions from uncertainties in the measurements or the reference cross sections. In addition to the uncertainties, a fitting diagnostic flag (FitConvergenceFlag) provides information on (non-)convergence of the fitting process. This flag should be consulted for more details on the quality of a particular BrO column datum. For details see OMBRO.fs.
Which Data Should Be Used?
Each SAO data product (BrO, HCHO, OClO) contains an INTEGER data field MainDataQualityFlag that should aid the user in the selection of which data to use and which to avoid. Each ground pixel is assigned a value, the range and classification of which are as follows:
| Value | Classification | Rationale |
|---|---|---|
| 0 | Good | All quality checks passed; data may be used with confidence |
| 1 | Suspect | Data should be used with caution because one or more of the
following conditions are present:
|
| 2 | Bad | Data should not be used because one or more of the following
conditions are present:
|
| -1 | Missing | No column values have been computed; entries are missing |
Product Description
A 2600 km wide OMI swath contains 60 cross-track pixels, ranging in size from 13 km x 24 km (along x across track) in the center of the swath to about 40 km x 128 km at the edges of the swath. The pixels on the swath are not symmetrically aligned on the line perpendicular to the orbital plane. However, the latitude and longitude provided with each pixel represents the location of each pixel on the ground to a fraction of a pixel.
The OMBRO product is written as HDF-EOS5 swath file. (For a list of tools that read HDF-EOS5 data files, please visit http://disc.gsfc.nasa.gov/Aura/tools.shtml). A single OMBRO file contains information retrieved from each OMI pixel over the sun-lit portion of the orbit (a.k.a. an OMI granule). The information provided in these files include: Geodetic longitude and latitude, solar and line-of-sight zenith and azimuth angles, total column BrO with RMS and 1σ fitting uncertainties, longitude and latitude corner coordinates for each OMI pixel, and a range of ancillary parameters that provide information to assess data quality. Average values over an OMI granule for the BrO total column, uncertainties, and RMS, as well as the percent values of "good" (converged and columns positive within 2σ fitting uncertainties) and "bad" (failed convergence or truly negative columns) provide general, granule-based information on data quality. For a complete list of data fields and their description, please read the file specifications.
OMBRO data, as well as subsets of these data over many ground stations and along Aura validation aircraft flight paths are available through the Aura Validation Data Center (AVDC) website to those investigators who are associated with the various Aura science teams.
For questions and comments related to the OMBRO dataset please contact Thomas Kurosu (tkurosu@cfa.harvard.edu). Please send a copy of your e-mail to Kelly Chance (kchance@cfa.harvard.edu), who has the overall responsibility for this product.
1 Alternatively: 30 cross-track pixels in rebinned spatial zoom mode, occurring one day per month.
OMBRO README FILE
Released: 16 November 2005
Date of this Document: 16 November 2005
Overview
This document provides a brief description of the OMBRO data product. OMBRO contains total column BrO and ancillary information retrieved from OMI global mode measurements using a retrieval algorithm that is based on non-linear least-squares fitting originally developed for GOME, and adapted for the OMI instrument. In global mode each file contains a single orbit of data covering a swath of approximately 2,600 km wide from pole to pole (sunlit portions only).
Average fitting precision is about 2*10-3 RMS, roughly 4-5 times what was achieved from GOME. Direct comparisons with GOME data products are not possible since BrO retrievals from GOME are no longer reliable due to the advanced degradation of the GOME instrument. However, retrieved vertical columns of ~2-4*1013 molecules/cm2 around the equator (where global BrO distributions are at a minimum) and ~1.5-2*1014 molecules/cm2 in isolated hot-spots like shelf-ice areas in polar regions during polar Spring, are in agreement with what has been observed from GOME in the past.
Release-Specific Information
| Software Version | 0.9.43 and higher |
| ECS Collection Number | 2 |
| Known Issue List | across-track striping in the data product (see OMSAO_DeStriping_README) |
Algorithm Description
The algorithm is based on the direct fitting of radiances and irradiances. In particular, and differing from what is commonly referred to as Differential Optical Absorption Spectroscopy (DOAS) fitting, radiances are not divided by irradiances, no logarithms are taken of the spectra, and no high-pass filtering is applied. The three main stages of the algorithm are (1) Solar wavelength calibration, in which the optimum wavelength registration of the solar irradiance measurements is determined and, unless pre-measured laboratory profiles are used, the instrument slit function is determined by fitting an (a)symmetric Gaussian (symmetric is the default); (2) Radiance wavelength calibration, which finds the optimum wavelength registration for a representative swath of radiance measurements (usually in the middle of the orbit) and determines a common wavelength grid for auxiliary data bases (molecular reference cross sections, etc.); and (3) Fitting of all swath lines in the OMI granule. In each stage, the calibration/fitting is performed individually for the 60 cross-track pixels1 of an OMI swath line. For improved numerical stability, radiances and irradiances are divided by their respective averages over the fitting window; in other words, they are "normalized" to values ~1.
BrO fitting is performed in the spectral window 323-347 nm, within the UV-2 channel of the OMI instrument. The model that is fitted to the measurements consists of the solar irradiance, attenuated by contributions from BrO (the target gas), inelastic (rotational Raman, or "Ring") scattering, and interferences from ozone and NO2; it also contains additive and multiplicative closure polynomials and parameters for spectral shift and squeeze (the latter is not used in nominal operations), as well as an undersampling spectrum that is computed on-line. The fit is mostly unconstrained, with the exception of selected parameters, including the spectral shift, which are constrained in order to prevent problems arising from .out of bounds. values.
The results from the spectral fitting are BrO slant columns. These are converted to vertical columns using a look-up table of stratospheric air mass factors (AMFs) that have been pre-computed using climatological BrO profiles. The AMFs used for the conversion are provided in the data product.
The algorithm employs a method to reduce cross-track striping of the BrO columns: fitting residuals from the radiance fits (Stage 3) are examined for outliers, which are defined as residual values larger than 0.005 absolute, for radiances normalized to ~1 over the fitting window. These values have been determined empirically from a statistically large number of BrO fits, and they represent the average upper limit of the spectral fitting residuals. If an outlier is identified in the fitting residual, the spectral fit is repeated with the CCD position corresponding to the outlier removed from the fit. The resulting BrO columns generally exhibit less cross-track striping, while average column values (across track for one or more swath lines) do not change significantly. The rationale for this outlier identification is as follows: CCD positions with associated fitting residuals that are significantly larger than the average residual over the fitting window are most likely due to bad pixels that were not correctly flagged and hence not excluded before the fitting process commenced. It is expected that improved bad pixel flagging will reduce - but not necessarily eliminate - the number of such outliers. Please see OMSAO_DeStriping_README for details on the destriping procedure.
In addition to the standard BrO columns as described above, the OMBRO product provides a smoothed BrO column that has been derived from the standard column by applying a cross-track smoothing filter. This product is highly experimental and should be used with extreme care. Again, please see OMSAO_DeStriping_README for details.
More details on algorithm specifics can be found in the OMI Algorithm Theoretical Basis Document Vol. 4 and in T.P. Kurosu, K. Chance, and C.E. Sioris, "Preliminary results for HCHO and BrO from the EOS-Aura Ozone Monitoring Instrument", in Passive Optical Remote Sensing of the Atmosphere and Clouds IV, Proc. of SPIE Vol. 5652 (2004), doi: 10.1117/12.578606, a PDF of which is available at http://www.cfa.harvard.edu/~tkurosu/Papers.
Data Quality Assessment
Across-track striping (systematically elevated or reduced column values at the same cross track position along the whole track) of the BrO columns is a presently outstanding issue. This is not unique to BrO but affects all OMI data products to a higher or lesser degree. Small absorbers like BrO, HCHO and OClO, however, are more strongly affected by striping since the column values are of a similar order of magnitude as the stripes, so that the effect is relatively stronger. Various efforts, both at Level 0-1 and 1-2 data processing, are under way to improve this situation, including the method of outlier identification in the fitting residual as employed in the BrO fit. A satisfactory solution remains still to be found, and users of the BrO columns provided here must be aware of this issue.
The BrO data product provides RMS and one standard deviation (1σ) fitting uncertainties, as derived from the fitting covariance matrix. These uncertainties do not include contributions from uncertainties in the measurements or the reference cross sections. In addition to the uncertainties, a fitting diagnostic flag (FitConvergenceFlag) provides information on (non-)convergence of the fitting process. This flag should be consulted for more details on the quality of a particular BrO column datum. For details see OMBRO.fs.
Which Data Should Be Used?
Each SAO data product (BrO, HCHO, OClO) contains an INTEGER data field MainDataQualityFlag that should aid the user in the selection of which data to use and which to avoid. Each ground pixel is assigned a value, the range and classification of which are as follows:
| Value | Classification | Rationale |
|---|---|---|
| 0 | Good | All quality checks passed; data may be used with confidence |
| 1 | Suspect | Data should be used with caution because one or more of the
following conditions are present:
|
| 2 | Bad | Data should not be used because one or more of the following
conditions are present:
|
| -1 | Missing | No column values have been computed; entries are missing |
Product Description
A 2600 km wide OMI swath contains 60 cross-track pixels, ranging in size from 13 km x 24 km (along x across track) in the center of the swath to about 40 km x 128 km at the edges of the swath. The pixels on the swath are not symmetrically aligned on the line perpendicular to the orbital plane. However, the latitude and longitude provided with each pixel represents the location of each pixel on the ground to a fraction of a pixel.
The OMBRO product is written as HDF-EOS5 swath file. (For a list of tools that read HDF-EOS5 data files, please visit http://disc.gsfc.nasa.gov/Aura/tools.shtml). A single OMBRO file contains information retrieved from each OMI pixel over the sun-lit portion of the orbit (a.k.a. an OMI granule). The information provided in these files include: Geodetic longitude and latitude, solar and line-of-sight zenith and azimuth angles, total column BrO with RMS and 1σ fitting uncertainties, longitude and latitude corner coordinates for each OMI pixel, and a range of ancillary parameters that provide information to assess data quality. Average values over an OMI granule for the BrO total column, uncertainties, and RMS, as well as the percent values of "good" (converged and columns positive within 2σ fitting uncertainties) and "bad" (failed convergence or truly negative columns) provide general, granule-based information on data quality. For a complete list of data fields and their description, please read the file specifications.
OMBRO data, as well as subsets of these data over many ground stations and along Aura validation aircraft flight paths are available through the Aura Validation Data Center (AVDC) website to those investigators who are associated with the various Aura science teams.
For questions and comments related to the OMBRO dataset please contact Thomas Kurosu (tkurosu@cfa.harvard.edu). Please send a copy of your e-mail to Kelly Chance (kchance@cfa.harvard.edu), who has the overall responsibility for this product.
1 Alternatively: 30 cross-track pixels in rebinned spatial zoom mode, occurring one day per month.