gdal-async

<html>  <head>  <TITLE>Where is NetCDF Used?</TITLE>   <META NAME="UIINDEX" CONTENT="0"> <META NAME="BOOKMARK" CONTENT="NetCDF Usage"> <META NAME="AUTHOR" CONTENT="russ"> <META NAME="KEYWORDS" CONTENT="netcdf, use, usage, who, where"> <META NAME="DESCRIPTION" CONTENT="The netcdfgroup@unidata.ucar.edu mailing list has over 500 addresses in 32 countries. Several groups have adopted netCDF as a standard way to represent some forms of scientific data.">  </head> <body>  <A NAME="netcdf" ID="netcdf"></A> <h1>Where is NetCDF Used?</h1> Making the software available freely available without registration makes it difficult to know where and how netCDF is being used, but we have compiled a <a href="https://www.unidata.ucar.edu/committees/usercom/2013Apr/statusreports/metrics/downloads/session/netcdf/known_sites_list_per_year.html" >list</a> of over 1300 educational, research, and government sites that have recently made some use of netCDF, based on <ol> <li> downloads from the Unidata site (ftp and http) </li> <li> subscribers and posters to netCDF mailing lists </li> <li> email support requests </li> <li> participation in annual netCDF workshops </li> </ol> during the period from 2012-04-01 through 2013-04-01. Recent standardization efforts have resulted in <a href="docs/standards.html" >netCDF endorsements</a> by several standards bodies. Here is a partial list of organizations using netCDF for archiving and accessing some of their data: <ul> <li> NOAA: <ul> <li> GFDL </li> <li> NOAA NODC (National Oceanographic Data Center) </li> <li> NCEP NOMADS service </li> <li> PMEL </li> <li> FSL </li> <li> NCDC NOMADS service </li> <li> NDBC (National Buoy Data Center) </li> <li> NOS/COOPS </li> <li> NMFS/PFEL </li> <li> ESRL (formerly CDC) </li> </ul> </li> <li> EUMETSAT Data Centre </li> <li> GOES-R Ground Segment Products </li> <li> DOE/PCMDI CMIP5 climate model outputs </li> <li> NASA/JPL </li> <li> NASA/GSFC </li> <li> GHRSST-PP community </li> <li> US Navy/ Fleet Numerical/GODAE Data Server </li> <li> OceanSITES.org </li> <li> National Snow and Ice Data Center (NSIDC) </li> <li> USGS, Woods Hole Field Center (WHFC) </li> <li> National Center for Atmospheric Research (NCAR) </li> <li> Center for Ocean-Land-Atmosphere Studies (COLA) </li> <li> George Mason University (GMU) Center for Earth Observing and Space Research </li> <li> Asia-Pacific Data-Research Center (APDRC) </li> <li> (Australia) CSIRO Marine and Atmospheric Research (CMAR) Remote Sensing </li> <li> (Australia) Centre for Australian Weather and Climate Research (CAWCR) </li> <li> (France) AVISO (satellite data) </li> <li> (France) French Research Institute for Exploitation of the Sea (IFREMER) </li> <li> (Germany) Federal Waterways Engineering and Research Institute (BAW) </li> <li> (Germany) German Aerospace Agency (DLR) </li> <li> (Italy) International Centre for Theoretical Physics (ICTP) </li> <li> (South Korea) Seoul National University (SNU) </li> <li> (Netherlands) European Climate Assessment (ECA), Atmospheric Data Access for the Geospatial User Community (ADAGUC) product standard </li> <li> (Netherlands) Cabauw Experimental Site for Atmospheric Research (CESAR) </li> </ul> We list below brief descriptions of some of the projects and groups that have reported on their use of netCDF. <hr /> <ul> <li> Several commercial analysis and data visualization packages have been adapted to access netCDF data. For more information about these and about freely-available software packages that can be used to display, analyze, and manipulate netCDF data, see the <a href="/software/netcdf/software.html">https://www.unidata.ucar.edu/software/netcdf/software.html</a> document. </li> <li> NOAA's <a href="http://www.cdc.noaa.gov">Climate Analysis Branch (CAB)</a>, conducts diagnostic studies of climate variability on time scales of months to centuries. CAB climatological data is archived in netCDF format. This site gives access to metadata (information about these datasets) which can be searched by various keywords; actual data can be subset or plotted on-line and can be downloaded via ftp. </li> <li> Modeling groups contributing output to the Intergovernmental Panel on Climate Change (IPCC) model archives at the Program for Climate Model Diagnosis and Intercomparison (PCMDI) are <a href="http://www-pcmdi.llnl.gov/ipcc/IPCC_output_requirements.htm" >required</a> to provide files in netCDF conforming with the NetCDF Climate and Forecast (CF) Metadata Conventions. </li> <li> EUMETSAT has begun to offer netCDF as a common delivery format for <a href="http://www.eumetsat.int/Home/Main/DataAccess/EUMETSATDataCentre/SP_20111027152034196?l=en)" >popular datasets</a> ordered from their archive. The developed datasets use netCDF-4 classic model format. </li> <li> NASA's Halogen Occultation Experiment ( <a href="http://haloedata.larc.nasa.gov/">HALOE</a>) makes HALOE data available in netCDF form. The <a href="http://haloedata.larc.nasa.gov/haloe_data_2.html">HALOE Data Viewer</a> provides each data type with a menu driven interface to assist in locating files based on date, time, species, data version and mode. </li> <li> The global ocean modeling effort at Los Alamos National Laboratory (LANL), as part of the DOE <a href="http://www.esd.ornl.gov/programs/chammp/chammp.html">CHAMMP</a> effort and one of the DOE Grand Challenges, has selected netCDF as the archival format for its computational data. An effort to bring netCDF up on the parallel disks on the CM-5 is planned to begin shortly. </li> <li> The National Center for Supercomputing Applications has incorporated the netCDF 2.3 interfaces into the latest release of their <a href="http://hdf.ncsa.uiuc.edu/">HDF</a> software, permitting HDF tools that use this interface to be applied to netCDF datasets that are either XDR- or HDF-encoded. </li> <li> The Computer Planning Committee of NOAA's Pacific Marine Environmental Laboratory (PMEL) endorsed netCDF as the preferred data format for the Laboratory in early 1993. PMEL has developed two freely-distributed applications that utilize NetCDF -- the EPIC system (for observational data) and Ferret (gridded data). <a href="http://www.pmel.noaa.gov/epic">EPIC</a> is a system for management, display and analysis of oceanographic time series and hydrographic data. EPIC toolkits for netCDF include a data file I/O library, which is layered on top of the netCDF library, a netCDF calculator (nccalc) linked with a scientific graphics package (PPLUS), a suite of display and analysis programs for oceanographic data, and a <a href="http://www.pmel.noaa.gov/epic/mexeps.html">MATLAB interface</a> to netCDF. <a href="http://www.pmel.noaa.gov/ferret/home.html">Ferret</a> is a visualization and analysis program, also built upon the PPLUS program, that permits users to explore large and complex gridded data sets. New variables may be defined interactively as mathematical transformations. Complex analyses proceed as hierarchical variable definitions. Visit <a href="http://ferret.wrc.noaa.gov/fbin/climate_server">Live Access to Climate Data</a> for a better sense of the Ferret program. </li> <li> <a href="http://www.ldeo.columbia.edu/">Lamont-Doherty Earth Observatory</a> of Columbia University has converted all Marine Geophysics data (gravity, magnetics and bathymetry) acquired in the past 40 years by scientists at L-DEO as well as at other institutions to netCDF. The <a href="http://www.ldeo.columbia.edu/~mggdb/">Marine Geology & Geophysics Database</a> is a fundamental resource for Lamont scientists and students. </li> <li> The <a href="http://www.soest.hawaii.edu/gmt">Generic Mapping Tools (GMT)</a>, a set of command-line tools for data manipulation and display using PostScript, make use of netCDF for storage of 2-D gridded data sets. GMT is used worldwide by about 6000 scientists, according to the developers. </li> <li> <a href="http://www.epa.gov/asmdnerl/mod3.html">The Models-3 Project</a>, being cooperatively pursued by the EPA's Atmospheric Research Laboratory and by North Carolina Supercomputing Center, is using an environmental-modeling-specific applications programming interface on top of UCAR's netCDF as the means for persistent storage of both observational and model-output data, as well as for storing sets of data-file-structure definitions and (prototype, so far) data-dependency graphs for scheduling the sets of programs which constitute their environmental models. </li> <li> A group in the Atmospheric Chemistry Division at NCAR (the National Center for Atmospheric Research) that deals with UARS (Upper Atmospheric Research Satellite) data uses netCDF for their binary data format. Output from NCAR's High Altitude Observatory Division Thermospheric General Circulation Model (TGCM) and related models are converted to netCDF files for post-model visualization and diagnostic codes. NCAR's Research Aviation Facility has decided to use netCDF to distribute aircraft data. </li> <li> NCAR's Research Data Program uses netCDF as the primary file format for data archived and used in the <a href="http://www.atd.ucar.edu/rdp/zebra.html">Zebra</a> display and analysis system. Quick look data from various projects is distributed by RDP in netCDF. NetCDF is also the file format used by the (Zebra-based) Integrated Sounding System. </li> <li> The Cooperative Program for Operation Meteorology, Education, and Training (COMET), a program of UCAR, has created an extensive archive of meteorological case studies that contain observed and gridded data in netCDF. The netCDF definition in use was created by the Forecast Systems Laboratory (FSL), a NOAA agency, and files in this format are required for capatability with their PC-DARE workstations which are used in the COMET teaching classroom. COMET plans to create future case studies using the netCDF conventions currently being developed by a working group of Unidata, COMET, and FSL personnel. </li> <li> Climate data are archived at several of the NOAA Regional Climate Centers (RCCs) usuig netCDF files. The netCDF software provides rapid access to time-series data and is at the core of a newly developed distributed data access system that will be employed at all of the NOAA RCCs and will be linked to the NOAA Virtual Data System (NVDS). </li> <li> The Earth Scan Lab, an HRPT ground station at Coastal Studies Institute, is using both the Terascan TDF as well as the netCDF for ease of data exhange of AVHRR, TOVS and DCS data. Further, in conjunction with Woods Hole, Scripps and Texas A&M, CSI will be maintaining all oceanographic data in netCDF. </li> <li> DataHub from JPL, with funding from AISRP/NASA identifies and converts between a variety of data format, CDF, HDF, MMM/netCDF, FITS, PDS, ... Work is under way to support conversion from a variety of NASA data formats to netCDF used by the PolyPaint+ visualization system from NCAR's MMM division. (JPL Contact for DataHub: Tom Handley, thandley@spacemouse.jpl.nasa.gov) </li> <li> A major component of the US Climate and Global Change program is the TOGA-TAO Array in the tropical Pacific, which proposes to maintain approximately 70 moored ATLAS wind and thermistor chain and current meter buoys, spanning the Pacific ocean from 95W to 137E in the equatorial wave guide. The TAO Project Office, at PMEL, has developed distribution and display software for the real-time data from the TAO buoys, in a point-and-click UNIX workstation environment. This software is distributed nationally and internationally. All data is stored and distributed in netCDF format. All graphics displays and animations are produced with the EPIC tools for working with netCDF data files. </li> <li> The Woods Hole Field Center of the U.S.G.S. Marine and Coastal Geology Program uses netCDF to access a variety of scientific data sets, including output from circulation and sediment transport models, sonar imagery, digital elevation models, and environmental sensor data. They also make available the <a href="http://crusty.er.usgs.gov/~cdenham/MexCDF/nc4ml5.html">NetCDF Toolbox for Matlab-5</a> </li> <li> At the Woods Hole Oceanographic Institution, netCDF is used in several areas. Ships in the UNOLS fleet are recording measurements from the IMET systems in netCDF form. These data include wind, barometer, humidity, air and sea temperature, precipitation, short wave radiation, and GPS navigation. Data sets from these systems taken during the WOCE experiments in the Pacific have been archived recently at NCAR. Also, measurements from a diverse set of instruments deployed on buoys for the Subduction, TOGA/COARE, and several other experiments are translated into netCDF form for processing and archival. Reports that describe the software systems used for these processing activities are available from WHOI. </li> <li> Scripps Institution of Oceanography (SIO) and the University Corporation for Atmospheric Research (UCAR) conducted a multi-platform climate field project during March of 1994 based in Nadi, Fiji. All data from this experiment will be archived using Unidata's netCDF before release to the scientific community. </li> <li> The Oregon State University Oceanographic <a href="http://lubber.oce.orst.edu/Wecoma/WecomaHome.html">Research Vessel WECOMA</a> uses the netCDF library for primary scientific data logging. This includes navigational, meteorological, and other miscellaneous data. This logging is part of a client/server system for data distribution, display, and management known as XMIDAS. </li> <li> NOAA's <a href="http://www.fsl.noaa.gov/">Forecast System Laboratory</a> has adopted netCDF as a data access interface for some of their systems and applications. </li> <li> The <a href="http://www.its.csiro.au/">CSIRO</a> Division of Atmospheric Research in Australia uses netCDF to store all their GCM and ocean model results. </li> <li> The Cabauw Experimental Site for Atmospheric Research (<a href="http://www.cesar-database.nl/" >Cesar</a>) is a consortium of presently eight institutes and universities located in the Netherlands. The goal of the consortium is to operate and maintain at the Cabauw site a unique observational facility with a comprehensive set of remote sensing and in-situ equipment to characterize the state of the atmosphere, its radiative properties and the interaction with the land surface. All the data are stored in netCDF formatted files and comply with the CF-1.4 convention. </li> <li> Meteorological data from satellites is stored in netCDF form at CIRES (Cooperative Institute for Research in the Environmental Sciences) and several data analysis packages have been written to display and analyze the netCDF data. </li> <li> A general purpose finite element data model (referred to as EXODUS II) utilizing netCDF has been developed at Sandia National Laboratories. It consists of a C and FORTRAN application programming interface (API) to read and write geometry and results (including time varying data) for finite element analyses. For more information, contact Larry Schoof (laschoo@somnet.sandia.gov). </li> <li> <a href="http://www.aero.polimi.it/~mbdyn/" >MBDyn</a> is an open-source Multi-body dynamics analysis system that uses netCDF as its output format. MBDyn features the integrated multidisciplinary analysis of multibody, multiphysics systems, including nonlinear mechanics of rigid and flexible constrained bodies, smart materials, electric networks, active control, hydraulic networks, essential fixed-wing and rotorcraft aerodynamics. </li> <li> NetCDF is the vehicle adopted by the Analytical Instrument Association [AIA] to implement the Analytical Data Interchange Protocols [Andi Protocols] for chromatography [released in 1992] and mass spectrometry [released in 1994]. The Andi Protocols increase laboratory efficiency and productivity by facilitating the integration and use of data from multiple vendors' products. For more information about the Andi chromatography protocols, see the article <a href="http://pubs.acs.org/hotartcl/tcaw/98/may/stan.html">"Standards for Chromatography Data Systems: ASTM adopts protocols for analytical data interchange (Andi)"</a> from May 1998 issue of the American Chemical Society publication Today's Chemist at Work The resulting standard specifications and guides may be downloaded from <a href="http://www.astm.org/">ASTM</a>: <ul> <li> <a href="http://www.astm.org/cgi-bin/SoftCart.exe/DATABASE.CART/PAGES/E1947.htm?L+mystore+rpwd2606"> E1947-98 Standard Specification for Analytical Data Interchange Protocol for Chromatographic Data</a> </li> <li> <a href="http://www.astm.org/cgi-bin/SoftCart.exe/DATABASE.CART/PAGES/E1948.htm?L+mystore+rpwd2606"> E1948-98 Standard Guide for Analytical Data Interchange Protocol for Chromatographic Data</a> </li> <li> <a href="http://www.astm.org/cgi-bin/SoftCart.exe/DATABASE.CART/REDLINE_PAGES/E2077.htm?L+mystore+uvlt1856" > E2077-00 Standard Specification for Analytical Data Interchange Protocol for Mass Spectrometric Data</a> </li> </ul> Further development related to these standards now seems to be under the purview of an ASTM subcommittee E13.15 on analytical Data Management, at the Source Forge site <a href="http://andi.sourceforge.net/" >http://andi.sourceforge.net/</a>. </li> <li> The Positron Imaging Laboratories and the Neuro-Imaging Laboratory of the Montreal Neurological Institute have selected netCDF as the data format for their medical image files. Conventions for variable and attribute names and values have been established for the medical imaging context. These conventions, along with a package of routines to assist in handling image data, make up the MINC (Medical Image NetCDF) format. For more information, contact Peter Neelin (neelin@pet.mni.mcgill.ca) or see <a href="http://www.bic.mni.mcgill.ca/software/minc/" > http://www.bic.mni.mcgill.ca/software/minc/</a>. </li> <li> In the field of molecular dynamics and the simulation of biomolecules, <a href="http://amber.scripps.edu/formats.html#netcdf" >netCDF conventions</a> are now used for storing and accessing trajectory files associated with the <a href="http://amber.scripps.edu/index.html" >AMBER project</a>. </li> <li> In computational biology, specifically flow cytometry, netCDF/HDF5 format has been used in new OpenCyto software. See Finak G, Frelinger J, Jiang W, Newell EW, Ramey J, et al. (2014) OpenCyto: An Open Source Infrastructure for Scalable, Robust, Reproducible, and Automated, End-to-End Flow Cytometry Data Analysis. PLoS Comput Biol 10(8): e1003806. doi:10.1371/journal.pcbi.1003806. </li> <li> At the Federal Waterways Engineering and Research Institute (<a href="http://www.baw.de/EN/Home/home_node.html">BAW</a>), netCDF is used in several ways. In combination with the <a href="https://github.com/ugrid-conventions/ugrid-conventions">UGRID conventions</a> computational results from different hydrodynamic numerical models are stored for unstructured grids, e.g. water level, current velocity, and salinity. NetCDF is also used to store derived quantities like tidal high and low water, tidal range and many other similar characteristic numbers. In addition, measured data (tide gauge data, ADCP) are also stored in netCDF files based on NOAA's <a href="http://www.nodc.noaa.gov/data/formats/netcdf/v1.1/">NODC NetCDF templates</a>. </li> <li> <a href="http://www.dlr.de/">The German Aerospace Agency (DLR)</a>, located in Munich, are using netcdf-4 / CF standards for the provision of L2 data for S5p mission including L2 products such as Sulfur Dioxide, Ozone, Formaldehyde and Clouds (<a href="https://en.wikipedia.org/wiki/Sentinel-5_Precursor">https://en.wikipedia.org/wiki/Sentinel-5_Precursor</a> or <a href="https://sentinel.esa.int/web/sentinel/missions/sentinel-5p">https://sentinel.esa.int/web/sentinel/missions/sentinel-5p</a>). </li> </ul>  </body>