ET ACDM 2024‐3: 2‐May‐2024 - wmo-im/et-acdm GitHub Wiki

Date and Time

2 May 2024 12:00-13:30 UTC (13:00-14:30 Geneva time)

Venue

Telecon

Attendance: Alex Vermeulen, Lynn Hazan, Debra Kollonige, Kaisa Lakkala, Gao Chen, Jörg Klausen, Markus Fiebig, Kjetil Torseth, Judd Welton, Teruo Kawasaki, Tom Kralidis, Sara Basart, Anna Milan, Sergi Moreno

Notes

1. Agenda from last meeting approved ET-ACDM 2024-2

2. GAW updates from Secretariat:

  • New Head of GAW (Paolo Laj) started 1st of May.
  • G3W Implementation Plan approved and recommended for approval and adoption at the next EC-78. Concerns about costs involved and data sharing, however the WMO open data policy should prevail.
  • WIS 2.0 operational by Jan 2025, various appendixes added to WMO no 1060 for metadata and WIS 2.0 standards. Strong support from the INFCOM participants.

3. WIS 2.0 presentation and discussion

  • github issue #20: WIS 2.0 topic hierarchy

  • Topic presented by Tom Kralidis (cf. Powerpoint presentation Tom Kralidis and Jeremy Tandy, WIS 2.0 Metadata publishing)

    • Publisher needs to define a dataset and have some infrastructure: a web server and a broker.
    • Data is available as buffer data or net CDF or through an API or any other format.
    • If it's NRT data it's possible to provide notification messages.
    • User needs to discover the data: publisher provides metadata, which will be published in the Global discovery Catalog (GDC).
    • GDC will provide links to the data: API's links to archive data, zip file, additional documentation...links can be to a single station or a group of stations, that should be identified and detailed metadata would help to select the desired dataset.
    • Time would be another parameter associated to the dataset, it can have open end, for example NRT data has a starting date but it's ongoing.
    • Data policies should be also specified in the metadata.
    • Users will subscribe to NRT data notifications or to other type of data notifications.
    • For archived data there is not need to provide notifications.

Q&A:

  • Would it be possible to find in the discovery metadata a specific variable from one station? A: It is possible to encode and have a link for each station or a set of stations, however not for single variables. TT on Metadata is working to make that function available. See approved WIS 2.0 metadata standard: https://meetings.wmo.int/INFCOM-3/_layouts/15/WopiFrame.aspx?sourcedoc={55F833B0-2FF8-4440-B7FA-57D62B9A35B6}&file=INFCOM-3-d08-3(1)-AMENDMENTS-WIS-MANUAL-approved_en.docx&action=default

  • Is it possible to include a specific API to single files? A: Yes, it is possible. For example Sweden is doing minutely surface weather observations.

  • Can be the be notifications be overwritten? A: No, they have a unique identifier, however you can have an update of the same observation.

  • WIGOS metadata more detailed than WIS metadata, however WIS metadata more flexible, that means each community will come up with recipes for metadata based on the community requirements.

  • How can assistance be provided to data centres who want to join WIS 2.0? A: Contact Anna Milan [email protected] or Tom Kralidis.

  • Is there a connection to GEOSS in WIS 2.0? A: Yes, there is a concept of federated infrastructures, however WIS 2.0 won't recreate GEOSS.

Topic hierarchy proposal

Anna Milan explained an example of discovery metadata record for wcmp2 based on a MPLNET record: https://github.com/wmo-im/wcmp2/blob/mplnet-examples/examples/ac-network-mplnet-example.json

Q&A:

  • Does the system handle different granularity assets, what needs to be provided? One record for the whole data set or one per variable and instrument? A: Broader granularity is preferred, for example for WOUDC total column ozone observations would be a data set and ozonesondes data would be another data set.

Action: members to think about examples of datasets from their domains and bring them up for a group's discussion

  • In the example 'core' data is used, what are the differences between 'core' and 'recommended' data? A: 'Core' data identified in the WIGOS Manual or in the WIS Manual must be freely available and a copy of that data will be downloadable in the cache for 24 hours, the 'recommended' data won't be available from the cache. WIS 2.0 will pull the 'core' data and cache it.

  • Does it mean the data provider won't have an overview and feedback from the use of the shared data? A: Data providers can specify in the metadata they don't want the data to be cached by WIS 2.0 but data users to access to that data using the servers from the providers.

  • Will the notification provide a link to the cached file or to provider's API? A: Notifications will point to the provider web server, then once the data is cached the download link will change to the cached data, unless the provider specifies the data should not be cached.

  • Questions about WIS 2.0 nodes: they seem to be approved as supported by NMHS, what happens with infrastructures supported by other institutions or supranationals like ICOS? Different solutions can be identified and agreements should be sourced, for example ECMWF is one recognised node and ACTRIS and ICOS have operational data centers.

4. WIS 2.0 topic hierarchy:

Sara proposed 4 sub categories: surface-based observations, space-based observations, analysis-prediction and warnings. No strong opposition against them, however the topic will be followed up the next meeting. Github issue: https://github.com/wmo-im/et-acdm/issues/20

5. Open issues:

Action: members to assign themselves at least to two open topics and decide if they are still relevant and if it is the case resolve them.

5. Next Meeting

ET-ACDM-2024-4: 18 June 11-12:30 UTC

Topics:

  • WIS 2.0 topic hierarchy
  • GEOMS (?)
  • AoB (please annonce beforehand)