Implementation Plan Table - ESGF/esgf.github.io GitHub Wiki

Task No. Task % Complete Status Day Started Day to be Completed Actual Date Completed Priority Milestone
1 CoG User Interface Working Team
1.1 Support CoG deployments 100 Completed 01/2016 12/2016 H
1.2 CoG with Globus API 80 On Schedule 02/2016 06/2016 H
1.3 Improve speed and responsiveness 0 Not Started NA 12/2016 M
1.4 Multiple virtual organizations 0 Not Started NA 12/2017 H
1.5 Integrate with computing analysis and visualization UI 0 Not Started NA 12/2019 H
2 Metadata and Search Working Team
2.1 Policy for index nodes 50 On Schedule 01/2016 05/2016 H
2.2 Atomic metadata updates 100 Completed 01/2016 05/2016 H
2.3 Enforce controlled vocabularies 10 Started 02/2016 12/2016 H
2.4 Transition harvesting to new Thredds API 0 Not Started NA 08/2016 H
2.5 Tag data with multiple activities 0 Not Started NA 08/2016 H
2.6 Support other data types 0 Not Started NA 12/2017 M
2.7 Solr cloud architecture 50 Ahead of Schedule 01/2016 12/2017 H
2.8 Search by location and time 0 Not Started NA 12/2017 M
3 Publication Working Team
3.1 Recommend changes to publisher tool 50 On Schedule 01/2016 12/2017 M
3.2 Test suite 75 On Schedule 11/2015 06/2016 L
3.3 New esgscan_directory 75 On Schedule 01/2016 06/2016 L
3.4 Drs_lite 0 Started 04/2016 06/2016 L
3.5 Support TDS new feature 0 Not Started NA NA L
3.6 Multiple facet values 75 Ahead of Schedule 02/2016 06/2016 M
3.7 CV integration/CMOR checker 0 Not Started 04/2016 06/2016 H
3.8 Support for new data services 95 Almost done 01/2016 05/2016 M
3.9 Best practices document 90 Almost done 04/2016 04/2016 L
3.10 Ingest Service developed and integrated with installer 80 On Schedule 7/2016 M
3.11 Update Ingest Service to replace publisher tool 0 Not Started N/A 12/17 L
4 Node Manager Working Team
4.1 Shard file generation 0 Not Started 04/2016 06/2016 M
4.2 Vet member nodes 0 Not Started 04/2016 05/2016 H
4.3 Integrate with desktop/dashboard 10 Design Proposed 01/2016 08/2016 H
4.4 Port to Flask 0 Not Started N/A 06/2017 L
4.5 Standby node mode 0 Not Started N/A 03/2017 M
4.6 Administrators’ console 0 Not Started N/A 08/2017 M
4a Tracking/Feedback (Notification) Working Team
4a.1 Proof of concept “Cron” job 0 Not Started 04/2016 09/2016 M
4a.2 Real-time tracking queries in GUI 0 Not Started N/A 12/2017 L
4a.3 Installer integration 0 Not Started N/A 04/2017 L
5 Identity Entitlement Access Management Working Team
5.1 Pilot integration of Live Access Service with CEDA ESGF OAuth 2.0 service 100 Done 02/2016 03/2016 03/2016 H
5.2 Implement service discovery mechanism for OAuth 2.0 5 Design Proposed 03/2016 05/2016 H
5.3 Deploy OAuth 2.0 and Online CA services operationally with ESGF Identity Providers 0 Not Started 06/2016 07/2016 H
5.4 Pilot integration of Globus with ESGF OpenID Connect service 5 Design Proposed 11/2016 12/2016 M
5.5 Retire MyProxyCA service (replaced by Online CA service) 0 Not Started 10/2016 10/2016 M
5.6 Implement and Integrate OpenID Connect into ESGF 0 Not Started 09/2016 12/2016 M
5.7 Retire OpenID 2.0 service 0 Not Started N/A 03/2017 H
5.8 Implement Attribute (role) registration web service interface 0 Not Started – may require reprioritization 01/2017 03/2017 H
5.9 Implement support for sign-in using other federation identities, e.g., InCommon and ESA identity federation 0 Not Started 02/2017 06/2017 L
5.10 Evaluate Assent for use with ESGF—enables integration of Eduroam with Single sign-on 0 Not Started 06/2017 09/2017 L
6 Compute Working Team
6.1 Finalize Version 1 of WPS API 50 On Schedule 01/2016 04/2016 H
6.2 Create and finalize API for computational kernels 10 On Schedule 03/2016 06/2016 M
6.3 Deploy proof of concept at LLNL 50 On Schedule 01/2016 07/2016 M
6.4 Define standard data set and unit tests 0 Not Started 04/2016 08/2016 M
6.5 Update to use cases 0 Not Started 04/2016 10/2016 M
6.6 Deploy second proof of concept at GSFC 40 On Schedule 01/2016 12/2016 M
7 Quality Control Working Team
7.1 Errata Service: Fix errata concept (WIP paper) 100 On schedule 04/2016 05/2016 H
7.2 Citation Service: Create development-operational instances 100 Completed 04/2016 05/2016 M
7.3 Errata Service: Fix issue information design + registration test on remote issue tracker and Handle Service + Django errata module development 60 Started 05/2016 06/2016 M
7.4 Citation Service: APEX GUI availability for modeling centers; integrate citation information in ESGF CoG; finalize citation landing page design and content 80 On Schedule 05/2016 06/2016 M
7.5 Errata Service: Register operable issues + integrate CoG errata module 0 Not Started 06/2016 09/2016 M
7.6 Citation Service: Add functionality to citation service (early citation is operational) 20 Started 06/2016 09/2016 L
7.7 Draft on recommendations for the integration of external information into ESGF 0 Not Started 06/2016 09/2016 L
7.8 Errata Service: full operability 0 Not Started 09/2016 12/2016 M
7.9 Citation Service: Integrate citation into LTA/IPCC-DDC and DataCite DOI process 0 Not Started 09/2016 12/2016 M
7.10 Final version of recommendations for the integration of external information into ESGF 0 Not Started 09/2016 12/2016 L
8 Installation Working Team
8.1 ESGF admin guide review— configuration part 0 Not Started N/A 04/2016 H
8.2 ESGF test suite review—CoG support 15 Test suite reviewed – CoG support ongoing N/A 05/2016 H
8.3 Integrate publication test suite written by PWT 0 Not Started N/A 06/2016 H
8.4 Make Globus Online credentials storage optional during installation 0 Not Started N/A 08/2016 H
8.5 RHEL/CentOS/SL 7 Support 0 Not Started N/A 10/2016 H
8.6 Migrate tomcat component installation to RPM 0 Not Started N/A 2017 M
8.7 Develop new Python based installation script 0 Not Started N/A 2017 M
8.8 Implement continuous integration based on existing continuous build infrastructure and apps 0 Not Started N/A 2017 L
8.9 Dockerize the data node stack 0 Not Started N/A 2017 L
9 Dashboard Working Team
9.1 Coarse-grained statistics system 95 Almost done 09/2015 04/2016 H
9.2 Project-specific statistics for Obs4MIPs and CMIP5 50 On Schedule 02/2016 05/2016 H
9.3 Design and first implementation of federated statistics views 20 On Schedule 02/2016 10/2016 H
9.4 Integration of perfSONAR statistics into the dashboard 0 Not Started N/A 12/2016 H
9.5 First implementation of new front-end presentation layer 0 Not Started N/A 12/2017 M
9.6 Extended set of statistics based on user requirements 0 Not Started 02/2016 12/2019 M
10 International Climate Network Working Group
10.1 Deploy perfSONAR at sites 10 Started 06/2015 12/2016 H
10.2 Publish data with GridFTP URLs 50 Started 01/2016 06/2016 H
10.3 Configure perfSONAR tests at selected sites 0 Not Started N/A M
10.4 Deploy DTNs 50 Started 06/2015 12/2016 H
10.5 Test Synda replication in collaboration with replication team 50 Started 02/2016 06/2016 H
10.6 Scale up DTNs 0 Not Started N/A 12/2018 H
10.7 Create agreed reference design for next-generation ESGF deployment 0 Not Started N/A 12/2018 M
11 Data Transfer Working Team
11.1 Deploy Globus Connect Server with automated certificate provisioning as the default installation option on all data nodes in ESGF 80 In Progress N/A 5/2016 H
11.2 Integrate Globus Transfer support for public data sets via CoG into the installer and deploy at various nodes 70 In Progress N/A 5/2016 H
11.3 Implement Globus Transfer support for restricted data sets. Integrate with the ESGF installer and make it available for deployment 15 In Progress N/A 8/2016 H
11.4 Integrate Globus Transfer as an option for data replication using Synda. Test and integrate with Synda. Participate in performance testing, in collaboration with ICNWG for optimization of performance replication 60 In Progress N/A 10/2016 H
11.5 Integrate use of OAuth for delegation of credentials to Globus, for improved user experience for browser-based flows using Globus for data transfer. Implement, test, and deploy this feature 0 Not Started N/A 11/2016 M
12 Software Security Working Team
12.1 Coordinate audits of the ESGF software suite release candidates using static and dynamic tools and code inspection as necessary 0 As-Needed N/A N/A H
12.2 Document findings and aid in issue resolution for all findings of moderate or higher impact 0 As-Needed N/A N/A H
12.3 Inform and assist the ESGF XC in assessing risks related to findings 0 As-Needed N/A N/A M
12.4 Support the ESGF Risk Executive in performing their function 0 As-Needed N/A N/A M
12.5 Document ESGF Site best practices regarding protective measures 0 Contained in Software Security Plan 4/2016 Goal: DRAFT approval + 3 mo. M
12.6 Participate in documenting the “as-is” state of the ESGF software suite build process 0 Not Started TBD (upon DRAFT approval) Goal: DRAFT approval + 6 mo. M
12.7 Maintain both the major and minor release procedures 0 On-Going N/A N/A L
13 Support Working Team
13.1 Create support documentation plan 0 Not Started N/A 4/2016 M
13.2 Revise documentation structure 0 Not Started N/A 5/2016 M
13.3 Revise content of ESGF User Guide 0 Not Started N/A 10/2016 H
14 Documentation Working Team
14.1 Upgrade appearance and design 0 Not Started N/A 09/2016 M
14.2 Refactor and reorganize documentation, splitting into user-, administrator-, and developer-centric hubs 0 Not Started N/A 11/2016 M
14.3 Coordinate with SWT to refactor and reorganize wiki 0 Not Started N/A 12/2016 M
15 Replication and Versioning Working Team
15.1 PID service performance and reliability tests 100 Completed 03/2016 04/2016 M
15.2 Install and test Synda installations at DTNs of core sites 5 Started 03/2016 06/2016 M
15.3 Initial deployment of PID infrastructure components and publication workflow tests at multiple sites 0 Not Started 05/2016 10/2016 H
15.4 Integrate and test Synda with Globus at core sites 0 Not Started N/A 07/2016 M
15.5 Define consistency requirements between replica sets at core sites and define policies ensuring these consistency requirements 0 Not Started N/A 08/2016 L
15.6 Complete full integration of PID services with web GUI 0 Not Started 07/2016 10/2016 M
15.7 Test large transfers between core sites based on Synda and work on optimizing end-to-end transfer bandwidth 0 Not Started N/A 11/2016 M
15.8 Complete initial integration of replica publication at core sites to the Synda-based replication workflow 0 Not Started N/A 11/2016 M
15.9 Integrate replication and versioning with PID infrastructure as part of ESGF CMIP6 publication workflow 0 Not Started 11/2016 M
16 Provenance Capture Working Team
16.1 Develop different language bindings for ProvEn Client API 0 Not Started N/A 08/2016 M
16.2 Requirement gathering for capturing existing sources of provenance information 0 Started 04/2016 06/2016 H
16.3 Performance reporting User Interface 0 Not Started 05/2016 07/2016 L