ET ACDM 2022 5 - wmo-im/et-acdm GitHub Wiki

2022 Meeting of the WMO Expert Team on Atmospheric Composition Data Management (ET-ACDM)

Location:

(Doc 1.4) PROVISIONAL AGENDA

Monday 7 Nov 2022 (09:00-18:00)

Opening Session

Time Item Topic Who Document
09:00 1.1 Welcome by the Director of Carbon Portal Vermeulen
09:15 1.2 Practical arrangements Vermeulen
09:25 1.3 Introduction of participants all Doc 1.3
09:40 1.4 Approve provisional agenda all this document
09:45 2.1 Report by chair, objectives of meeting Klausen Doc 2.1
10:00 2.2 Relevant outcomes of WMO: EC-75; SSC GAW IP meeting; INFCOM-2, SERCOM-2 WMO/Moreno Doc 2.2
10:30 Coffee break (30')

Emerging requirements and opportunities for atmospheric composition data management

Time Item Topic Who
11:00 WMO Greenhouse Gas Initiative Tarasova
11:30 WIS 2.0 data and metadata exchange to support WIGOS Fucile
12:00 Urban workshop outcomes and interaction Urban & AC Sokhi
12:30 Lunch break (90')

GAW WDCs: Achievements and plans w.r.t. to GAW ACDM (15' each)

Time Item Topic Who Document
14:00 4.1 WDCA Fiebig covered under 4.2
. 4.2 WDCRG/WDCA/EBAS Tørseth Doc 4.2
. 4.3 WDCGG Kinoshita Doc 4.3
. 4.4 WOUDC Kralidis Doc 4.4
. 4.5 WRDC Tsvetkov no contribution received
. 4.6 WDCPC Lehmann no contribution received
. 4.7 WDC-RSAT Goussev Doc 4.7
15:45 Coffee break (30')

Contributing/Regional programs/data archives: Achievements and plans w.r.t. GAW ACDM

Time Item Topic Who Document
16:15 GALION/MPLNET Welton Doc 4.8
16:30 Earth System Modeling, digital twins Peuch Doc 3.4
. TCCON Feist no contribution received
. SHADOZ Stauffer, Kollonige Doc 4.8
. NDACC De Mazière no contribution received
. (Other) Nasa data centres Chen no presentation, see minutes
. EANET Sato no contribution received
. SKYNET Nakashima no contribution received
. ICOS Vermeulen Doc 4.9
. EUBREWNET Redondas, Lakkala Doc 4.10
... ... ... ...
17:45 Wrap-up Klausen
18:00 Adjourn, informal dinner

Tuesday 8 Nov 2022 (09:00-18:00)

Making atmospheric composition data management fit for the future

Time Item Topic Who
09:00 ... ...
10:30 Coffee break (30')
11:00 Split into 2-3 sub-groups and work on 4 topics ...
12:30 Lunch break (90')
14:00 Work in sub-groups ...
15:30 Coffee break (30')
16:00 Briefing on status and plans from sub-groups Rapporteurs
16:30 Work in sub-groups ...
17:45 Wrap-up in plenary ...
18:00 Adjourn, informal dinner

Wednesday 9 Nov 2022 (09:00-11:00)

Making atmospheric composition data management fit for services

Time Item Topic Who Document
09:00 Reports from sub-groups, discussion ...
10:30 Wrap-up & Conclusions Klausen, all
11:00 Close of meeting

SUMMARY OF THE MEETING

Presentations: https://wmoomm-my.sharepoint.com/:f:/g/personal/smoreno_wmo_int/Eowq5m8099RKne9N_q56QDAB2GwifglHLZ_jkZmxuj8Wgg?e=TiWfqe

Alex Vermeulen welcomed the participants and introduced Lund University, a reputed education institution in Sweden, he also commented about the currently building ESS infrastructure.

Jörg presentation:

  • Changes in the membership to be confirmed (Schultz, Peuch, Goday...) and new members to be appointed with priority for Asian/African women. Sergi to contact Oylstein Goday about his commitment with ET-ACDM.
  • Concerns about the level of interaction with WRDC, WDCPC and WDC-RSAT. Discussions about the Global Energy Balance Archive in Switzerland, status of WRDC unclear at the moment. Action: Sergi to clarify the involvement of WDC-RSAT in GAW.
  • WMO web site is quite messy and the ET are not reflected, urgent action required.
  • The idea of a federated data management has not been achieved, open for discussion/alternatives.

Sergi’s presentation:

Oksana’s presentation: GHG Monitoring Infrastructure: https://meetings.wmo.int/INFCOM-2/English/2.%20PROVISIONAL%20REPORT%20(Approved%20documents)/INFCOM-2-d04-2-GHG-MONITORING-INFRASTRUCTURE-approved_en.docx?Web=1

  • WDCGG in JMA supports the calculation of global GG global averages from a climatological perspective.
  • IG3IS was set up to support policies and mitigation actions at national, subnational and facilities level.
  • The current approach to mitigation action does not take into account biogenic CO2, a global approach including emissions, sinks and net fluxes is needed.
  • WWW would be a paradigm for the new infrastructure. If the concept is approved by the WMO Congress in 2023 the idea is that observations and analysis are reflected in the first UNFCCC global stocktake.
  • GAW will have a strong participation in the research side, supporting long-term measurements for climatology.
  • The operational activities will be regulated with prescribed protocols.
  • The products would be tailored for each scale, focus on fluxes, 1° x 1° spatial resolution and one month time resolution. Comments:
  • For Data Quality would be good to have the data in 24 hr. 1 month is for fluxes average and data assimilation, so probably the QA/QC should be in half year. 1 month for inverse modelling product, which will be the final product for fluxes.
  • Satellite and in-situ data would be included, there is a priority list.
  • Groups from WMO like ET-Data, ET-Metadata, ET-ACDM would be included as far as the initiative advances. Sawa-san is included in the initiative.
  • Consultations in the IG3IS group with the private sector, they will make sure about the data sharing providing.

Ranjeet presentation:

  • GURME projects include not traditional sources of data, for example crowd-sourced data.
  • Impact of climate more important for cities. Also, different episodes like sandstorm, wildfires, etc... create different problems.
  • Short lived climate pollutants would be needed for urban monitoring.
  • Open AQ data is an option, but data from campaigns and other sources is useful because there is not a coordinated global database for urban data.
  • Data scarce areas would benefit from lower-cost sensors; however guidance is required.
  • There is a need for complex requirements for AC in urban areas.
  • It is possible to have some sets of data, but QA/QC could be an issue. Discussion about including more local stations within GAW, can Ranjeet contact them and ask them to be included, could it be discussed with UNEP?

Enrico’s presentation: Wis 2: WIS 2.0 Demo - YouTube

  • Data providers send data to a global cache and global broker via a WIS2 node (WIS2 BOX), users receive notifications of new data uploaded.
  • Core data can be downloaded directly from the global cache, otherwise recommended data can be downloaded from providers like EUMETSAT with the possibility of access control.
  • Integration of GTS and also WHOS and other systems within WIS 2.0. Still GAW needs to be included, discussion on how should be included open.
  • WIS 2.0 will be used to discover a data set and will offer a global catalogue to harvest metadata.
  • Descriptive metadata should be curated to be well described after several decades. Question about how the contributing networks will be notified about the use of their data. For discovery metadata there will be KPI to ensure quality, notifications can be provided for new data or for modifications in the existing data. Requirement that should be documented for WIS2 global services: • As a WDC, I would like to know the access and download metrics of my data as cached/served by the WMO Global Cache, so that I can include these metrics as part of my WDC reporting. Data archiving: as the cache storage won’t be eternal, data needs to be discussed case by case, for example satellite data can’t be stored permanently. The wide use of the data can be a good mechanism for getting more funding. Requirements to have metrics for the cached downloads would help to identify the providers and they can get more funding. WIS topic hierarchy: wmo-im/wis2-topic-hierarchy (github.com) wis2-topic-hierarchy/earth-system-discipline.csv at main · wmo-im/wis2-topic-hierarchy (github.com)

Kjetil: • EBAS database moved to Postgres platform, upgraded EBAS home. • ACTRIS European Research Infrastructure for aerosols, clouds and trace gases; costs: 100 M x all national facilities, 15M for the central facilities. • ENVRI-FAIR has surveyed participants on how the FAIR principles have been implemented. • Questionnaire about the FAIR principles. Matrix about the level of FAIR convergence. • Vocabulary server: Skosmos. I-adopt: how to build vocabularies. • ACTRIS also promotes data history: Provenance. • Data never delete it; any version of the data set from any time can be reproduced. • It is important to have a hierarchy for vocabulary: I-adopt as an example, WIS 2.0 will have also a vocabulary hierarchy, needs to be defined.

Atsuya: • WDCGG progressed assigning DOIs and adding OCO-2 data, still needs to integrate metadata in GAWSIS and uncertainty. • They do averages of data and monthly mean, published in the GHG bulletin. • Merged data gets DOI updated every year. • Visualization with concentration colour. • Discussion about sending an observation in order to register a station in OSCAR. • Concerns about having to ask the national representative to be able to become a member of GAWSIS.

Tom: • WOUDC corrects the metadata for QA/QC because providers sent wrong metadata on several occasions. Sometimes errors can’t be easily fixed and providers received an error report, however the errors are not fixed. • Difficulties moving to OGC, users reluctant to changes. • External data linkages: EUBREWNET and NDACC indexing (downloads still from their site) and full data conversion for SHADOZ. • Recommended a single source, one only representation of the same set of data. • Question about uploading pieces into OSCAR. • Analysis ready data, it needs to be self documented.

Oleg: • Services for Science approach: o Data on demand o Computing on demand o Service on demand o Operating on demand • DFD-ATM collaborates with ECMWF and ESA in different projects. • WDC-RSAT also works with infrared images. • Not possible to track users due to the European law of data protection. • Discussion on WDC-RSAT been a real WDC.

Vincent-Henri: • Digital twins help to: o have better simulations, o better ways to combine information (they could use data currently not in use) o and be more interactive than NWP • Systematic errors can led to wrong conclusions when comparing high resolution forecast • Uncertainty for short range forecast needs to be assessed and verification in research to operations should be implemented. • DT can help to reduce the dependence on expertise and use the data faster, an example could be to detect methane leaks. • Analysis ready data should be self documented (metadata). • Unknown data can be assessed considering time series or location for example. • In DT machine learning is needed in some parts both for modeling and for data assimilation.

Debra: • Shadoz is a gold standard for ozone measurements in tropics and southern hemisphere. • NRT data not practical, usually data delivered from weekly to quarterly year. • Around 400 ozone profiles each year. • New DOI for ozonesonde and uncertainty recently created. Next steps: DOI for each station, data delivery to WOUDC and core metadata updated. • Thanks to the Measurement Guidance ASOPOS 2.0 data is traceable to a single reference standard.

Judd: • Lidar network is very expensive, so it needs to be shared by different networks: GALION. • Different data products. • Partner portal in MPLNET: o Logging system o New data ordering tool o NRT data not available to download, only for partners • GALION has established standards for calibration and processing methods. • GALION data center in preparation, problems to report data in oscar: name of lidars, missing variables and aerosol variables. • GALION is trying to be the centre to upload data to oscar instead of every user doing it. • NASA Amazon web Services contract.

Kaisa: • Different products produced by EUBREWNET that can be downloaded in CSV • EUBREWNET is not officially linked to WMO? Perhaps SSC EPAC should decide. • Data is centrally gathered, and all the members follow the same procedure, then it can be downloaded from WOUDC. • Define architecture of participation to prevent double source of data. • Discussions happened about what and where the people submit data • Action for the next IP. Architect the data from EUBREWNET needs to be codesign, we can’t impose it. (similar issues with NDACC). • Similar status with UV radiation. European UV initiative led by Austria, it would be interesting to contact them.

Alex: • ICOS follows GAW calibration scale. Calibration labs used for atm & ocean data. In the repository and in the carbon portal you can find raw data, curated and calibrated data. • Machine to machine automated, open-source software. Highly FAIR data. Attribution of data providers, sponsor/funding. • Not suited for manual update. • Technical info in github, metadata mapped to ISO 19115 • Workflow for the ecosystems: • Direct upload to ETC, NRT processed by ETC. • Level 3 objects, jupyter service.PID registry API. • Python library created, direct access to the data. • Services: stilt: emission footprint viewer. Cities project. • Metadata unified for the whole network. Data can contain metadata, like headers.