The ESDIM Program

A Brief History

by Eric Davis

ESDIM Program Manager
NOAA Environmental Information Services
NOAA/NESDIS

NOAA is responsible for the collection, management, and stewardship of a rapidly increasing amount of data and information. This data and information encompasses all of NOAA's activities and includes holdings of climatological, geophysical, oceanographic, marine fisheries, hydrographic, and cartographic records. Much of this data is held and archived in NOAA's National Data Centers: the National Climatic Data Center, the National Geophysical Data Center, and the National Oceanographic Data Center. Some of the data is stored by NOAA's Line Offices or by the office or scientist who originally collected it. Data is stored on a variety of media, but is generally considered to be of one of three types: paper, film, or digital.

A NOAA-wide program, the Environmental Services Information and Data Management (ESDIM) Program, was created in response to the NOAA Under Secretary for Oceans and Atmosphere's concern about data management in the organization. Data management in NOAA deals with acquiring, quality controlling, preserving, storing, and providing user access to its data holdings. A study by NOAA's Advisory Panel on Climate and Global Change was commissioned in 1989 to review NOAA's data management and to make recommendations for more effective information management. The panel's February 1990 report, along with a November 1990 GAO report, "Environmental Data: Major Effort Needed to Improve NOAA's Data Management and Archiving", were the catalysts that started NOAA's ESDIM Program in early 1991.

Originally part of the Office of the Chief Scientist, the ESDIM Program was soon transferred to NOAA's National Environmental Satellite, Data, and Information Service (NESDIS), Office of Environmental Information Services (EIS), where it is located today. The ESDIM Program is responsible for selecting and funding data management projects to be accomplished by NOAA. The ESDIM Program has a small permanent staff that runs the day to day operations and has a team of advisors from NOAA's Line and Program Offices, that reviews data management project proposals and selects the projects the ESDIM Program will support in the upcoming year.

During the early years, the ESDIM Program concentrated on data rescue. Data rescue is the saving or salvaging of data on paper, film, or digital media, that would otherwise be lost, converting it to a stable, useable media, and then archiving and/or making it available for access.

NOAA's data includes paper records from the beginning of our country's history through the present time. Paper deteriorates over time depending on the physical properties of the paper and the conditions under which they were stored. Microfilm and microfiche, used to rescue deteriorating paper records, are themselves susceptible to deterioration and must be rescued. Digital media is used to rescue paper and film records and to store newly acquired data. . Digital media is also susceptible to deterioration. Data stored on older, less dense magnetic tape are being rescued to more dense, smaller sized tape cartridges or to optical media. In time, data stored on today's best media will have to be rescued.

The amount of new data to be archived is rapidly increasing as new data collection systems are placed into service. These new systems include satellites and NEXRAD weather radars. To archive this data, it must be converted from the original media, particular to the collection system, to the data center's archival media. This conversion is also data rescue.

In recent years access to NOAA's data and information has become a significant area of concentration for the ESDIM Program. Access is the process of making available data and information held by an individual, office, or organization to a much larger audience. To access data it is necessary to know what data is available and where it is located. This is being accomplished by ESDIM with the online NOAA Environmental Services Data Directory (Earth System Monitor, December 1996, pp 6-8). The NOAA Directory catalogs the metadata, or data about the data, for NOAA's data holdings in Federal Geographic Data Committee (FGDC) Metadata Standard format. An online search for locating NOAA data, using any word in the data description, can be performed at the NOAA Directory web site,
http://www.esdim.noaa.gov/NOAA-Catalog/.

Providing web access to distributed NOAA data and information through a single web site is NOAAServer, an ESDIM supported project. Participants in the development of NOAAServer include representatives from all of the NOAA Line Offices. A limited but growing portion of NOAA's data and information is presently available through this system at http://www.esdim.noaa.gov/NOAAServer/.

ESDIM sponsors projects within NOAA that meet the goals of the program in the areas of access, rescue, continuity, and innovation. An example of the projects sponsored is NOAA's National Ocean Service(NOS) MapFinder. The NOS MapFinder is a major outreach effort, a one-stop Internet service, http://mapindex.nos.noaa.gov/, to deliver primary NOS products to public and private-sector coastal resource managers to assist their planning and management activities. NOS MapFinder will officially premiere in August 1997 and will provide direct Internet access to NOS imagery and data holdings including coastal photography, nautical charts, coastal survey maps, environmental sensitivity index maps, hydrographic surveys, water level stations, and geodetic control points.

Further information about the history of the ESDIM Program can be obtained by contacting the Program Manager: