WIGOS Metadata Guidelines for Data Providers - wmo-im/et-acdm GitHub Wiki
WIGOS Metadata Guidelines for Data Providers
Introduction and scope
Metadata for discovery and descriptive metadata
More specifically metadata for geospatial datasets; even more specifically metadata for atmospheric (composition?) data
Why is this needed? Finding data and linking data becomes more and more important; for example realtime applications are not feasible without automated data discovery and data ingestion. Setting this up requires automated and standardized provision of metadata on the side of the providers. Provision of metadata via open standard protocols also opens up new possibilities for integrated analyses across instrument or measurement types and across science and earth system domains.
During the past decade or so rapid developments took place both in terms of defining metadata standards and concepts and in terms of software stacks that support the development of open data services. However, most scientists are unaware of these developments, or if they develop an interest in these topics, they find it very hard and overwhelming to understand the concepts, what is needed to implement (or use) metadata services, and where to find reliable information about standards, how stable they are, etc.
In the domain of atmospheric data, the World Meteorological Organisation (WMO) has recently developed a metadata standard WMO Integrated Observing System (WIGOS), which provides a rather generic and flexible description of atmospheric observations, including the observing facility, instrument deployment conditions, instruments, data quality, and more.
This document first provides a general introduction to metadata including the formats and tools that can be used to represent and process them (section 2). It then (briefly?) describes the WIGOS metadata standard and the information flow and machine user roles (section 3). In section 4, a number of use cases and metadata examples are discussed.
Metadata basics: XML, XSD, UML, templates
The WIGOS metadata standard
Published as WMO No. 1192 Information on the normative WIGOS Metadata Representation (WMDR) is found at schemas.wmo.int/wmdr and documents referenced there.
Information flow between GAWSIS-OSCAR/Surface and the various data centres supporting GAW
GAWSIS-OSCAR/Surface (GAWSIS and OSCAR/Surface) share the same database but have slightly different user interfaces. The public API is provided by OSCAR/Surface, i.e., metadata upload using XML is only supported by OSCAR/Surface. The metadata in GAWSIS-OSCAR/Surface are organized in some main categories represented in a station report by collapsible blocks
- Station characteristics including program/network affiliations
- Observations including instruments used
- Contacts
In addition to these, documents and bibliographic references can be archived by the applications. Management of this information is not currently supported by the public API.
All information in GAWSIS-OSCAR/Surface is owned by the supervising organization of a station, i.e., the Permanent Representative of a WMO Member, often the head of the National Meteorological and Hydrological Service (NMHS). In order to facilitate regular updates of the information, the NMHSs can act as machine users and update all information on the stations in their country through the API. Members are ultimately responsible for the information in GAWSIS-OSCAR/Surface.
Registered data centres (DC) can also act as machine users. They are authorized to update information on the observations they host, including begin and end dates, instruments, methods, and responsible contacts.
In a practical sense, it is useful to consider who should be the single source of information, and which parts should only be consumed.
| Element(s) | Source(s) |
|---|---|
| Station characteristics | NMHS |
| Supervising organization | NMHS |
| Program/network affiliations | NMHS, DC |
| Observations / deployments including instruments used | NMHS, DC |
| Begin and end dates of deployments, declared operating status | NMHS |
| Begin and end dates of data at data center | DC |
| Contacts for observations | NMHS, DC |
Complications to be considered and agreed by ET-ACDM
- observations archived at multiple data centers --> requires agreements among data centers
- missing code list / vocabularity elements --> request additions at https://github.com/wmo-im/wmds
Use cases and XML examples
- PHP solution developed by Mustafa Sert