2025 Software Engineering meeting notes - NGEET/fates GitHub Wiki

May 05, 2025

Wins and Whimsy :tada:

  • E3SM API 39
  • Adrianna: personally defined PFTs working in nocomp-fbg mode
    • Potential presentation at FATES modeling meeting

Machine status and notices :desktop_computer:

Upcoming meetings and events :spiral_calendar:

  • FATES modeling meeting: 8 May, Qing Zhu
  • NGEE-Tropics Annual Meeting: 20-22 May
  • Jennifer: MRV talk tomorrow. Where to advertise?
    • Consensus is that its ok for Jennifer to advertise on the FATES modeling meeting list

[PR board status update](https://github.com/orgs/NGEET/projects/13/views/1

Integrated

On Hold

Testing

Prioritization

  • Ryan: Phenology change priotization relative to the CLM6-FATES code freeze?
    • Marcos: The science updates should come in after the code freeze.
  • Facilitate crops in SP mode (#817)
    • Adrianna: This should work.

Latest issues and discussions of note

  • Fuel consumed not zeroed when no fire (#1400)
    • Adrianna: this will impact diagnostics, but does not actually impact the internal spitfire calculations.
      • Should be a simple PR fix, but Xiulin noted there are other spitfire variables that have similar issues. Adrianna asked for list of these to wrap into one PR fix.

Calibration progress update

  • deferred

April 28, 2025

Wins and Whimsy :tada:

  • Ryan: spending hands-on time with OLMT

Machine status and notices :desktop_computer:

Upcoming meetings and events :spiral_calendar:

PR board status update

Integrated

Testing

Reviews

Latest issues and discussions of note

Code freeze coordination preview discussion

  • Code freeze release date: late summer 2025
  • Currently in code "chill"
    • B4B changes and parameter file updates for calibration
  • Will to send out a meeting invite

Calibration progress update

  • Adrianna: no update
  • Jessie and Rosie are out

Walk on topics

April 21, 2025

Wins and Whimsy :tada:

  • Successful ESS-PI and ESS-CIWG meetings

ESS-PI meeting recap

  • Ryan: Debjani Seehee (sp?) biogeochemical cycling work (lack of nutrient mineralize death spirals)
    • Jennifer: her U.S. digital soil maps are great
  • Jessica: Chongang showed results of stem respiration turned on with fates
  • Greg: Root phenology breakout presentation about decoupling root and leaf phenology
  • Jennifer: seperate disturbance timescales for above and belowground?
  • Ryan: anyone talking with Ben Sulman? Maybe invite him to talk.
    • Greg: He gave a lightning version of his ELM-PFLOTRAN alquimia talk

Machine status and notices :desktop_computer:

Upcoming meetings and events :spiral_calendar:

PR board status update

Integrated

Testing

Reviews

New

Latest issues and discussions of note

Calibration progress update

  • Jessie: tropical run with Rosie's parameter file with continent and pft specific allometries
  • Ryan: BCI calibration using OLMT on perlmutter

Walk on topics

April 14, 2025

Canceled due to ESS-PI meeting

March 31, 2025

Machine status and notices :desktop_computer:

Upcoming meetings and events :spiral_calendar:

PR board status update

Integrated

Testing

Reviews

  • Prescribed Fire (#1360)
    • Ryan: is there another way that we could capture these prescribed parameters?
      • Charlie: we could put it on the landuse file or make a specific prescribed fire file
        • Consensus is that this might be a good idea as prescribed fire is site/region specific
        • Greg: this would require new/updated infrastucture. What does Xiulin's timeline look like; can she wait?
    • Marcos: combine upper/lower along a fire dimension
    • Ryan: Worthwhile to invite Xiulin speak on this? Consensus is yes. Charlie will reach out to her and Lara.

New

Latest issues and discussions of note

Calibration progress update

  • Rosie: Working on hand tuning.
  • Adrianna: currently working on other deliverables not related to global calibration

Walk on topics

  • Adrianna: slamax, slatop being the same ok?
    • Marcos: this means there will be no plasticity.
      • Charlie: Typically for grasses/shrubs
  • Rosie: SLA unit conversion sheet and paper results comparison
    • Marcos: qualitatively the numbers look correct relative to his recent TRY analysis
      • Be mindful of where in the canopy the measurement was made, which can skew results.
    • Jennifer: recommending making a synthesis table of all the SLA/LMA results that group has

March 24, 2025

Machine status and notices :desktop_computer:

Upcoming meetings and events :spiral_calendar:

  • FATES modeling meeting: TBD
    • Qing Zhu moved to later date
    • April 10 Eastern regional DVGM overlap.
    • Greg to cancel this week.

PR board status update

Integrated

Testing

  • restart fixes for two-stream (#1321): Ryan running final tests
    • FATES two-stream restart fixes (ctsm#2949): associated ctsm pull request
    • Running into testing issues on izumi. We troubleshot this during standup today. Issue appeared to be license server needing a restart.
    • Ryan: clean up misleading text in future PR. Make comment
    • Charlie: make two-stream the default?
      • Ryan: Conduct a performance test. Greg made an issue #136?
  • NCL max (#1315): smoke tests run, but results need review

Reviews

New

Latest issues and discussions of note

  • Error calculating carbon flux due to fire mortality (#1365): Charlie to review and respond
  • Reconcile parameter and scales driving leaf phenology and mortality (#1363)
    • TL;DR: Difference in timescales for hydro mort and enviro mort could result in deciduous pfts dying in conditions that they should be protected against
      • From an ideal, simplified case deciduous should be nearly perfectly sheilded from hydraulic failure. Not so for evergreens that can "take more risk."
    • Ryan: would it be hard to kill deciduous plants?
    • Rosie: Hydraulic disassociation from the soil?
    • Marcos: calibration results for drought deciduous pfts?
      • Rosie seeing that they are dying, but not sure about the mortality type
    • Charlie:
    • Marcos: change deciduosness and see how this fairs
    • Aside: Marcos also seeing a lot of termination mortality. Likely unrelated.

Calibration progress update

  • deferred due to time

FATES specific surface dataset discussion

  • deferred until next week (missing to many folks)

March 17, 2025

Machine status and notices :desktop_computer:

  • CISL: none planned
  • NERSC: 26 March, perlmutter

Upcoming meetings and events :spiral_calendar:

PR board status update

New

Next

Updated

In-Progress

Latest issues and discussions of note

Calibration progress update

  • Rosie: no new update since Thursday modeling meeting updates. Kicking off new tests. Working on dry deposition as well.
  • Adrianna: getting ready to kick off another "one-at-a-time" calibration run using new fnps parameter
  • Jessie: Kasey (ORNL) different allometries for pfts depending on regional (i.e. continent)
    • Implementation idea: bounds on where seeds for specific pfts can not recruit. Potential ForestGEO collab.
    • Adrianna: this seems like it would benefit GPP calibration as well (seeing region balance shift of GPP for tropical evergreen trees, Asian vs Amazon)
    • Rosie: related paper: https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2024EF005225

Walk-on topics

  • Rosie: fates-specific surface dataset (to facilitate the above region specific pft discussion)
    • Charlie: move LAI data out of surface dataset and let fates use directly (similar to land use data)

March 10, 2025

Machine status and notices :desktop_computer:

  • CISL: none planned
  • NERSC: 26 March, perlmutter

Upcoming meetings and events :spiral_calendar:

PR board status update

Integrated

Updated

New

Latest issues and discussions of note

Calibration progress update

XML parameter file discussion

  • Pros:
    • CIME makes use of python XML base functionality
    • FOX as existing HLM fortran
    • fortran code on fates-side that reads in parameters is complicated (since we avoid HLM IO), would simplify any necessary maintenance
    • allows us to expand the features that we could control in the parameter file (e.g. metadata) Cons:
    • netcdf provides dimensionality fail-safes that we currently "get for free"
      • we'd have to take on this functionality ourselves
    • We'd need to pull FoX into FATES to make available to all HLMs
  • Questions
    • parameter ensembles?
      • xarray allows for modification of netcdf emsembles easily
    • Would the developer's experience in add parameters be simpler (i.e. RegisterParam)
  • ED2 (via Mike Dietz) implemented XML fortran

Walk-on topic

March 3, 2025

Wins and Whimsy :tada:

  • API38 integrated and tagged (still waiting on e3sm side)

Machine status and notices :desktop_computer:

  • CISL: none planned
  • NERSC: 26 March, perlmutter

Upcoming meetings and events :spiral_calendar:

PR board status update

Integrated

Updates

New

Latest issues and discussions of note

E3SM and LMWG report back

  • E3SM: focusing on V3
    • Utilizing E3SM Diags, the model diagnostics package more, in order to report performance progress of FATES. Can ask for tutorial from Jill or Xylar.
      • Includes benchmarking as well as diagnostics (includes ILAMB)
      • Related: NorESM looking at utilizing cupid.
    • Ryan: GPU push
      • E3SM is not concerned with FATES converting to C++. There currently exists multiple ways to integrate existing fortran code with C+
  • LMWG
    • FATES crop model work seen as necessary for full CLM adoption
      • Concerns about fates CPU performance cost

Calibration progress update

  • Ryan: centralized work tracking and data sharing repository?
    • Initially attempted to centralize, but diverged due to different focus.
  • Rosie: no-comp GPP with Medlyn in amazon issue discussion
    • Marcos: driest parts of the amazon seems to diverge the most

February 24, 2025

Wins and Whimsy :tada:

Machine status and notices :desktop_computer:

  • CISL: none planned
  • NERSC: 19 March, perlmutter

Upcoming meetings and events :spiral_calendar:

  • FATES modeling meeting: 13 March, project report back
    • Greg to send out reply-all to previous notice about this weeks usual meeting being canceled
    • Marcos: suggested to move Steven's presentation to April 24 to encourage attendance (avoid scheduleing conflicts with Eastern DVGM meeting)
    • Greg to send out notice for API 38 update. Bring this up during

PR board status update

Integrated

Update

New

Latest issues and discussions of note

Calibration progress update

Walk-on topic

February 17, 2025

Canceled due to Federal Holiday

February 10, 2025

Wins and Whimsy :tada:

  • Adrianna has been given expanded fates repo priviledges
  • New github issue templates

Machine status and notices :desktop_computer:

  • CISL: 11-12 February, derecho
  • NERSC: 19 February

Upcoming meetings and events :spiral_calendar:

  • FATES modeling meeting: 13 February, TBD
    • Consensus: cancel, Greg to send out notice
    • Rosie has reached out to Tom Buckley

PR board status update

Integrated

Update

New

Latest issues and discussions of note

Multi-layer canopy planning

  • Ryan: stakeholders?
    • Gordon
    • Gautam
    • Adrianna and Danica: grant funded work, connect Sam L's ctsm branch to FATES. Working on site-level data collection for validation.
      • Timeline: through to 2028
      • Includes NCAR NEON stuff as well
  • goal: FATES changes to be model agnostic
  • action: Ryan and Adrianna to meet to discuss

Calibration progress update

  • Adrianna: once SP bug fixed (via escomp/ctsm#2952), will rerun calibration
    • three paper ideas based on this work
  • Rosie: new post-doc to help with FATES calibration
    • Adrianna to share calibration script location

Walk-on topic

  • Rosie: Matvey thinks that Leung dust scheme (new scheme) might have been built on top of Meier2022
    • Adrianna: Zender is the old dust scheme
      • We should check in with Erik about Leung

February 03, 2025

Wins and Whimsy :tada:

  • Land use fixes integrated

Machine status and notices :desktop_computer:

  • UCAR HPC: izumi experiencing issues since Friday
  • CISL: 11-12 February, derecho
  • NERSC: 19 February

Upcoming meetings and events :spiral_calendar:

  • FATES modeling meeting: 13 February, TBD
    • Rosie to reach out to Tom Buckley
    • Ryan: project coordination meeting update?
      • Jennifer suggested doing this later as a report back from the CESM-LMWG and E3SM all hands
    • Charlie will be out next week

PR board status update

Integrated

Update

New

Latest issues and discussions of note

Calibration progress update

  • no updates

Walk-on topic

  • Rosie: Land use tool needs to wor for unstructed grids. Rosie has someone to work on this.

Janurary 27, 2025

Wins and Whimsy :tada:

Machine status and notices :desktop_computer:

  • CISL: 11-12 February
  • NERSC: 27-31 January, Full site power down

Upcoming meetings and events :spiral_calendar:

  • FATES modeling meeting: TBD
    • FATES tool poll results
      • modify_params had highest uptake, batchpatch, consistent with Ryan's expectations
      • immediate take away: don't change anything, look
    • office hours: Greg to send out notice
  • Mingjie and Jessie wind mortality meeting this Wednesday 11.30 - others welcome

PR board status update

Integrated

  • Crown depth checks (#1314)
    • Marcos: should we add more alllometry checks?
      • Greg: definitely worthwhile to add more checks for user conveience.
        • One option would be to move convienence checks (i.e. not things that could crash the model) upstream to fates tooling that changes the parameter file.
    • Charlie: sapwood allometry?

Update

New

Latest issues and discussions of note

Calibration progress update

  • Jessica: ran CLM calibration parameter file in ELM. At preliminary review, it looks good.
  • Adrianna: rerunning following htop issue fix (Adrianna to make formal issue).

Walk-on topic

  • Charlie: land use spin up
    • Default decomp value in ELM is very agressive, Charlie adjusted and spin up works much better
    • Might be due to of the decomp method "scheme"
    • Default value in CLM is more reasonable relative to updated value
  • Rosie: Meier2022
    • ELM
    • Ryan: making the code more "defensive"
      • Initializing fates code with "nan"
      • Don't allocate if not used
      • Adrianna: API that doesn't allow HLM to have access to data structures
      • Greg: look for API update issue/discussion

January 20, 2025

Canceled due to Federal Holiday

January 13, 2025

Wins and Whimsy :tada:

  • CTSM temp branch merged into master
  • ctsm5.0.18 has history time being on the middle of time bounds
    • Marcos: will this be reflected in e3sm?
      • Greg will reach out to e3sm folks
  • Rosie: New FATES post-doc hire. Will be working on dwarf shrub parameterization.

Machine status and notices :desktop_computer:

Upcoming meetings and events :spiral_calendar:

  • FATES modeling meeting: 16 January, Annemarie Eckes-Shephard
    • Ryan to conduct informal poll on using patches.

PR board status update

Integrated

Update

New

Latest issues and discussions of note

Synthetic patches for functional testing update (Adrianna)

  • insert_cohort not preserving sorted order inherently - deferred

Calibration progress update

  • deferred

Walk-on topic

  • Rosie: https://github.com/ESCOMP/CTSM/issues/2932
    • Boundary conditions are different between clm6 and fates
    • Displacement height needs to be fixed, biasing SP results. Is Meier2022 the default? Check in with Adrianna.
    • Charlie: ELM using ZengWang? Rosie: yes
    • Ryan: create quick PR to make sure ZengWang is default if fates is on? Greg to address.
    • Jessie will reach out to Adrianna for latest parameter file and create a new smoke test

January 6, 2025

Wins and Whimsy :tada:

Machine status and notices :desktop_computer:

Upcoming meetings and events :spiral_calendar:

PR board status update

Integrated

Update

Latest issues and discussions of note

  • FDI calculation (#1298)
    • Xuilin, NGEE-tropics phase 3 has a bunch of work planned (Lara)
    • Adrianna to bring this to the CA meeting
      • Jennifer will give Lara a heads up about discussing this at a future meeting
  • fire calculations regardless of event (#1299)
    • Adrianna: do we want to avoid calculating values when there is no fire?
      • Sam: not doing so will improve computation. Maybe improve history output.
      • Sam: some of the fire outputs were weighted by burned area, but not all outputs.
      • Rosie: you might want to know the potential effects regardless of fire active or not (are you looking "upstream" or "downstream") .
  • PVT run FAIL (CTSM#2919)

Synthetic (aka "dummy") patches for functional testing (Adrianna)

  • Adrianna developing "dummy" patches for use with funtional and unit tests
  • Idealized biomes with simplistic, well-defined values
  • Motivated by future work to conduct with Danica as well as creating unit tests for cohort sorting, patch insertion, etc.

Calibration progress update

  • no new updates due to break

Walk-on topic

  • Rosie: NorESM working on repo to keep recipes for simulations being run. Maybe preview next week.