DataCuration_Input4MIPs - ACCESS-NRI/CMIP7-Input GitHub Wiki
Data curation of the input4MIPs mirror
The existing input4MIPs local mirror within the NCI qv56 project should be expanded to include the CMIP7 forcing datasets.
Estimate of space needed for an ESGF input4MIPs mirror of CMIP7 datasets
Space needed for the CMIP7 mirror is approximately 10 TB, based on the space used by the /g/data/qv56/replicas/input4MIPs directories at NCI and the /gws/nopw/j04/cmip6_prep_vol1/input4MIPs directories on Jasmin:
qv56 at NCI
There is approximately 15.8 TiB headroom in qv56 for inclusion of CMIP7 forcing datasets.
[pcl851@gadi-login-06 ~]$ lquota | head -n3 && lquota | grep qv56
--------------------------------------------------------------------------
fs Usage Quota Limit iUsage iQuota iLimit
--------------------------------------------------------------------------
qv56 scratch 12.00 KiB 72.00 GiB 75.60 GiB 3 164000 172200
qv56 gdata 24.15 TiB 40.00 TiB 42.00 TiB 42451 3135000 3291750
Existing CMIP6 and CMIP6_plus mirrors consume 6.3 TiB:
- 5.3 TiB for
CMIP6 - 1.1 TiB for
CMIP6Plus
[pcl851@gadi-login-06 ~]$ ls /g/data/qv56/replicas/input4MIPs/
CMIP6 CMIP6Plus
[pcl851@gadi-login-06 ~]$ du -hsc /g/data/qv56/replicas/input4MIPs/*
5.3T /g/data/qv56/replicas/input4MIPs/CMIP6
1.1T /g/data/qv56/replicas/input4MIPs/CMIP6Plus
6.3T total
[pcl851@gadi-login-06 ~]$ du -hsc /g/data/qv56/replicas/input4MIPs/*/*
16G /g/data/qv56/replicas/input4MIPs/CMIP6/AerChemMIP
626M /g/data/qv56/replicas/input4MIPs/CMIP6/C4MIP
899G /g/data/qv56/replicas/input4MIPs/CMIP6/CMIP
43G /g/data/qv56/replicas/input4MIPs/CMIP6/DAMIP
1.7M /g/data/qv56/replicas/input4MIPs/CMIP6/DCPP
11M /g/data/qv56/replicas/input4MIPs/CMIP6/FAFMIP
398G /g/data/qv56/replicas/input4MIPs/CMIP6/HighResMIP
5.3G /g/data/qv56/replicas/input4MIPs/CMIP6/ISMIP6
3.7T /g/data/qv56/replicas/input4MIPs/CMIP6/OMIP
781M /g/data/qv56/replicas/input4MIPs/CMIP6/PAMIP
287G /g/data/qv56/replicas/input4MIPs/CMIP6Plus/CMIP
781G /g/data/qv56/replicas/input4MIPs/CMIP6Plus/OMIP
9.1M /g/data/qv56/replicas/input4MIPs/CMIP6/RFMIP
251G /g/data/qv56/replicas/input4MIPs/CMIP6/ScenarioMIP
6.3T total
j04/cmip6_prep_vol1 on Jasmin
- 2.5 TiB for
aims3.llnl.gov - 1.2 TiB for
esgf-data2.llnl.gov
[pcl900@sci-vm-01 ~]$ du -hsc /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/*
2.5T /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov
1.2T /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/esgf-data2.llnl.gov
341G /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/scratch
[pcl900@sci-vm-01 ~]$ ls /gws/nopw/j04/cmip6_prep_vol1/input4MIPs
aims3.llnl.gov esgf-data2.llnl.gov scratch
[pcl900@sci-vm-01 ~]$ du -hsc /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov/*
2.5T /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov/CMIP6
2.5T total
[pcl900@sci-vm-01 ~]$ du -hsc /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov/CMIP6/*
11G /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov/CMIP6/AerChemMIP
387M /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov/CMIP6/C4MIP
553G /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov/CMIP6/CMIP
25G /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov/CMIP6/DAMIP
494K /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov/CMIP6/DCPP
9.3M /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov/CMIP6/FAFMIP
234G /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov/CMIP6/HighResMIP
1.6T /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov/CMIP6/OMIP
504M /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov/CMIP6/PAMIP
7.3M /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov/CMIP6/RFMIP
154G /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/aims3.llnl.gov/CMIP6/ScenarioMIP
2.5T total
[pcl900@sci-vm-01 ~]$ ls /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/esgf-data2.llnl.gov/
CMIP6 CMIP6Plus
[pcl900@sci-vm-01 ~]$ du -hsc /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/esgf-data2.llnl.gov/*
877G /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/esgf-data2.llnl.gov/CMIP6
323G /gws/nopw/j04/cmip6_prep_vol1/input4MIPs/esgf-data2.llnl.gov/CMIP6Plus
1.2T total