July 29, 2024

• Wins (1 min)
• Machine status and notices (1 min)
• Lawrencium support (5 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Latest issues (5 min)
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

• Land use v2 E3SM API integrated

Machine status and notices :desktop_computer:

• CISL: August 12-15
• NERSC: July 31, Perlmutter

Lawrencium support

Lawrencium moved from CentOS to Rocky8 and the cime configuration to support this machine is broken

• Do we want to prioritize an effort to fix this in the long term?

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: August 15, Summer Office Hours

PR board status update

Latest issues and discussions of note

Calibration progress update

Tutorial update

---

July 22, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Photosynthesis Unit Tests (10 min)
• Latest issues (5 min)
  • Arctic Shrubs Update (5 min)
• Leaf Temperature Discussion (5 min)
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

• NGEE-T proposal review complete
• Land use v2 integrated (E3SM API to be integrated soon)

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: August 15, Summer Office Hours
  • Pick up regular cadence starting after the tutorial in September?

Machine outages :desktop_computer:

• CISL: August 12-15
• NERSC: July 31, Perlmutter

PR board status update

New

• nocomp patch cleanup
• refactor per-age-class outputs: drafted

Updates

• Dynamic patch arrays: investigating two test run fails

Latest issues and discussions of note

• fates_hydro_vg_alpha_node too small?
• photosynthesis check issue

Calibration progress update

Tutorial update

• Beta-testing scheduled for two-weeks starting Aug 12

---

July 15, 2024

Cancelled due to NGEE-Tropics proposal review

---

July 8th, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Latest issues (5 min)
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: July 18, Summer Office Hours

Machine outages :desktop_computer:

PR board status update

• No PRs integrated this last week

New

• Time integrated flux diag: needs review

Updates

• Land use v2: integration slated for today post discussion about fb+nocomp mode bareground calculation change
  • E3SM land use v2 API: Cleaning up and testing.
  • CTSM land use v2 API: Erik to review
• Grazing: reviewed by Sam with some comments/questions. Still needs build/run testing.

Latest issues and discussions of note

• Incorrect per-age-class outputs
  • Consensus is that the _AP should be weighted by the patch age class area, although we are not consistent currently
  • Sam also noting that #1170 looks like it may not have fully addressed the issue for FATES_LAI_AP.

Calibration progress update

• nothing of note, focus is on preparing for NGEE-T proposal

Tutorial update

• Greg shifting work post landuse v2 to tutorial infrastructure starting this week

---

July 1st, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Latest issues (5 min)
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

• LU V2 seems to be passing tests, ready for integration soon?

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: July 18, Summer Office Hours

Machine outages :desktop_computer:

PR board status update

• No PRs integrated this last week

New

• Integrated flux diagnostics - create an issue on the nutrient related bias
• prescribed uptake in the FATES PRT2 test

Reviewers

• Fire Weather Refactor
• History Flushing

Updates

• Land use v2: final testing with ctsm and e3sm
• Understory turnover rate: Approved with one minor comment. Needs testing. Has new parameters.
  • Plan is to have this be its own API update post land use API update.
  • Rosie would like to get this in soon for calibration efforts. As such, we will bring in #1206 and TBD nutrients switch default PR afterwards.
• Dynamic patch arrays: Approved with one minor question. Needs testing.
• Grass allometric equations: Check in with Marcos
• Growth Resp to daily timestep: Jessie provided review

Latest issues and discussions of note

No new issues or discussions

Calibration progress update

• Rosie: Planning to check in with Charlie about handing off efforts to him wrt land use spin up.

Tutorial update

• FATES tutorial will use FATES API 35

---

June 24, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Latest issues (5 min)
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

• Carbon budget closing thanks to Ryan and Jessie's efforts. PR forthcoming.

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: July 18, Summer Office Hours

Machine outages :desktop_computer:

• CISL: none planned
• NERSC: June 26, Perlmutter

PR board status update

Integrated

• FATES API 35 compat: Merge to next. Waiting integration to master.

Updates

• Land use v2: final testing with ctsm and e3sm
• Understory turnover rate: Approved with one minor comment. Needs testing. Has new parameters.
  • Plan is to have this be its own API update post land use API update.
  • Rosie would like to get this in soon for calibration efforts. As such, we will bring in #1206 and TBD nutrients switch default PR afterwards.
• Dynamic patch arrays: Approved with one minor question. Needs testing.
• Grass allometric equations: Check in with Xiulin. Needs testing. Updates param values.
• Growth Resp to daily timestep: Ask for another reviewer. Needs build and run testing.
  • Jessie will review as well.

Latest issues and discussions of note

No new issues or discussions

Calibration progress update

• Rosie: Planning to check in with Charlie about handing off efforts to him wrt land use spin up.

Tutorial update

• FATES tutorial will use FATES API 35

---

June 17, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Latest issues (5 min)
• Carbon balance closure issue
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: June 20, Summer Office Hours
  • Graduate student intern visiting NCAR this summer will present briefly on pyrogenic carbon
  • Related: Rosie noted that there is a NorESM PR to bring couple trace gasses back to HLM
  • Adrianna to send out email
• 2024 CESM workshop debrief
  • FATES presentations from Adrianna and Rosie

Machine outages :desktop_computer:

• CISL: none planned
• NERSC: June 26, Perlmutter

PR board status update

Integrated: History flush and zero fixes

Updates

• Land use v2: correcting precision error in patch splitting
• Understory turnover rate: Tagged Jenny.
• Grass allometric equations: Marcos reviewing. Needs followup?
• FATES API 35 compat: Merge to next. Waiting integration to master.
• Dynamic patch arrays: Greg to review.
• Grazing: Jennifer signed up to review this as well. Timeframe is out beyond land use v2 and may still needs build testing.

Latest issues and discussions of note

• seed production output bug: Jessie has a fix; she will point to her branch to be merged in at next earliest convience.

Calibration progress update

• Discussion of determine the uncertainty of the parameters

Tutorial update

Walk on topics

• Discussion of adapting damage model to fire processes
  • Damage model would likely slow down a global model quite a bit

---

June 10, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Latest issues (5 min)
• Carbon balance closure issue
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: June 20, Summer Office Hours
  • Marcos has a possible person to present
• 2024 CESM Workshop: Jun 10 - 13

Machine outages :desktop_computer:

• CISL: June 11, Derecho
• NERSC: June 26, Perlmutter

PR board status update

Updates

• History flush and zero fixes: in testing
• Land use v2: Refactoring split_patch to address precision issue in buffer reallocation
• Understory turnover rate: Ask Jenny to review
• Grass allometric equations: Needs review
• FATES API 35 compat: Update to include #1208 and retest
• CTSM Landuse v2 API: Waiting on review. Needs minor deconflict.
• Dynamic patch arrays: initial speed tests look fine, no difference
  • Tested with both old and new version to have cap of 3

Latest issues and discussions of note

• default fates_cnp_prescribed_* parameters: to solve, this needs to have prescribed p_uptake and n_uptake turned on for CESM CNP regression test

Carbon balance closure issue

• Marcos: attempt turning off reproduction to determine if that is the issue.
• Ryan/Charlie: NPP minus seed decay should equal allocation
• Ryan: we could also add another data structure to add this up over time

Calibration progress update

no updates

Tutorial update

• Google group for attendees setup
• Greg to send out email with "hello world" docker setup request

---

June 3, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Latest issues (5 min)
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: June 20, Summer Office Hours
• 2024 CESM Workshop: Jun 10 - 13

Machine outages :desktop_computer:

• CISL: none this week
• NERSC: June 26, perlmutter

PR board status update

Updates

• Land use v2: addressing rsnbl_math_prec issue during reallocation
  • Derecho is currently down, which is holding this up.
• History flush and zero fixes: ready for testing

Latest issues and discussions of note

Issues

• migrate fates "classic" projects: github sunsetting "classic" project boards
  • PR board migrated. We should triage the others.
• default fates_cnp_prescribed_* parameters: integrate this with #1136.

Discussions

• FATES invetory init: soil initialization followup question

Calibration progress update

Adrianna: updated emulator based on Jessie's suggestions which showed improvement in estimating average GPP.

Tutorial update

---

May 27, 2024

Cancelled due to Federal Holiday

---

May 20, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Latest issues (5 min)
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

• Greg accepted to ATPESC 2024

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: May 23, Shaoqing Liu
  • Greg to reach out to Marcos about sending emails
• 2024 CESM Workshop: Jun 10 - 13

Machine outages :desktop_computer:

• CISL: none this week
• NERSC: June 26, perlmutter

PR board status update

Integrated: photosytem 2 named constant

Updates

• Land use v2: addressing some run failures
  • Ryan: B4B expected
• Growth Resp to daily timestep: review underway
• Understood turnover rate: prioritization?
  • Rosie would like to see this in for calibration. Jessie has this on her calirbation work currently.
• Dynamic patch arrays
  • Looking for clarification on the consensus from last week
  • Agreement that ncl of two is too small.
  • Next steps: test performance of ncl of three against dynamic allocation PR

Latest issues and discussions of note

Issues

• arctic shrub pfts
  • from Yanlan's up coming paper
  • need to think about how to update the hlm-fates pft mapping
  • how many do we add for global simulations? And how do we split the fractions?
    • duplicate 11, split the fraction, don't use 9
  • Long term development: handling divergence in HLM pft data sets
• NaNs in climate driver warning: Erik responded
• carbon budget not closing: discussion deferred
• zero canopy layers no cohorts: discussion deferred

Discussions

• FATES invetory init: Marcos responded

Calibration progress update

• Rosie: need maintresp, allom_d2ca_coeff, allom_d2bl1 to be low to get LAI to observed
  • It's reasonable for this to be difficult for the model based on literature research
  • Ryan: is radiation model hampering things here? I.e. do we need some sort of explicit gap?
• Adrianna: shared latest updates from variation on results based on number of gridcells included

Tutorial update

---

May 13, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Latest issues (5 min)
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

Jennifer on Star Talk

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: May 23, Shaoqing Liu
  • No presenters scheduled beyond May 23
• 2024 CESM Workshop: Jun 10 - 13

Machine outages :desktop_computer:

• CISL: none this week
• NERSC: May 15, perlmutter

PR board status update

Integrated: Allometry unit test, Inventory input updates, e3sm unstructured grid tool

Updates

• Land use v2
• fraction energy named constant: to be tested ASAP

New

• Dynamic patch arrays
  • team formulated to discuss and assess impact of setting nclmax
  • termination mortality proposal for cohorts beyond nclmax sounds good
• Growth Resp to daily timestep: needs build testing
  • TL;DR: If plants aren't growing, they shouldn't be growth respiring. FATES only grows on daily timestep => only growth respiring on daily timestep
  • How does this compare to HLMs?
  • Don't use NEP for carbon budgeting; introduce a NBP flux for carbon budgeting.

Latest issues and discussions of note

• no new issues or discussions of note

Calibration progress update

• Adrianna: TRENDY plans discussion
  • TRENDY need: External pft-specific seed rain for extinct pfts (for spin up compensation)
  • Updated SP mode configurations forthcoming

Tutorial update

• No new updates

---

May 6, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Latest issues (5 min)
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

New group using FATES: Tau Liu at Michigan Tech. Teaming up with Chonggang. Interested in calibration and regional runs.

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: 09 May, Rachel Ward
• 2024 E3SM Tutorial: May 7 - 10
• 2024 CESM Workshop: Jun 10 - 13

Machine outages :desktop_computer:

• CISL: May 7-8, derecho
• NERSC: May 15, perlmutter

PR board status update

Integrated: CCH water xfer functions, LAI-by-patchage

Updates

• Small fire refactor: ready to test
• Allometry unit testing: testing discussion held on Friday. Minor org refactor to be done.
• Land use v2: Walkthrough held. Break out land use alignment from disturbance TBD.
  • CTSM API 36: drafted. run mode combo checks discussed and updated
  • How do we test spin up for land use?
    • Similar to the fact that we don't have AD testing
    • Sam Rabin knows how to add a new test in run_sys_test. Greg will tag up with him later.
    • Decision: make issue for testing spin up, but don't hold this up. Add standard regression tests for other landuse v2 modes.
  • Deadline: integrated Friday May 24

New

• Land Energy Balance Solve
  • what is a good absolute case? Look at distribution of conductance based on different values and come up with reasonable delta.
  • Marcos: Stomata have minimum when close so what about using residual as the absolute tolerance?
  • Ryan: check in with Gautam for E3SM-side reviewer and feedback

Latest issues and discussions of note

• no new issues or discussions of note

Calibration progress update

• Rosie: One at-a-time ensemble run last week
  • fates_phen_cold_size_threshold? See PR #554, grass phenology

Tutorial update

• No new updates

---

April 29, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Latest issues (5 min)
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

• NGEET Phase 3 proposal submitted
• First FATES newsletter sent out
• ELM User's guide PR with connecting FATES documentation

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: 09 May, Rachel Ward
• 2024 CESM Workshop

Machine outages :desktop_computer:

• CISL: none planned
• NERSC: May 15, perlmutter

PR board status update

• Integrated: E3SM API 33, fire edge case bug fix, day length factor, alternative vert scaling, CTSM API 35
• Updates
  • LAI-by-patch-age fix: ready to integrate
  • CCH water transfer functions
  • Understory turnover rate

Latest issues and discussions of note

• spin-up speed-up
  • Dan Ricutto might now
  • E3SM specific. Loads stuff into memory.
• growth respiration calc
  • Initial solution consideration (Marcos): calculate on a daily basis and apply with each timestep
    • Need to have code to account for edge cases given if function based on previous day's value
  • We want to get this done prior to the nocomp calibration release
  • How does this apply with HLMs (big leaf)?

Calibration progress update

• Adrianna: Daniel and Lydia discussion about setting up emulator. How much do we need for a good set for each pft?
• Rosie: Rerunning one at a time nocomp ensemble with latest main branch (daylength + alt vert scaling).
  • Charlie: High seed rain clarification: Rosie wants to dig into this a bit more, ideally with fully spun up case.
  • Is there a danger in including seed rain in an ensemble for global calibration?
  • Rosie will make global map of seed rain impact

Tutorial update

• Logistics: Organizing travel

Walk-on topics

• Should we re-review this meeting time?
  • Goverance meeting?

---

April 22, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• Newsletter discussion
• Metrics questions
• PR board status (5-10 min)
• Latest issues (5 min)
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

• Greg presented at SEA ISS for US-RSE working group

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: April 25, Huanyuan Zhang
  • Jessie got confirmation for this week
  • Marcos will need someone to host May 23

Newsletter discussion

• What sort of topics would people like to see?
  • Add request for fates users
• How to generate content?
  • Asynchoronous update + in meeting review
  • Dump "wins" from these meetings into the document live
• Ryan to setup calendar reminder for sending it out every quarter
• Rosie: Post-doc position - Rosie

Metrics questions

• How many countries and institutions use FATES?
  • Rosie has a pretty good handle on this. She will send out slide with this information.
    • US universities is an area of uncertainty.
    • Tutorial application could be used for metrics in this regard.
• How many members do we have?

Machine outages :desktop_computer:

• CISL: none planned
• NERSC: May 15, perlmutter

PR board status update

• API 35 integration and PR1128 is imminant
• API 33 integration needs confirmation of developer tests passing
• Landuse v2 walkthrough scheduling
  • Setup walkthrough meeting next Thursday in the timeslot for fates modeling
• Ryan to reach out to hydro folks for final review
• Adrianna looking for reviews for unit testing and fire refactor
  • Unit testing needs cime and shr
  • Greg and Charlie to review

Latest issues and discussions of note

• Minor cleanup issue submitted by Sam Rabin

Calibration progress update

• Rosie: shared parameter sensitivity results

Tutorial update

• No new update

Walk-on topics

• Carbon budget issue - DBEN - Jessie
• Post-doc position - Rosie

---

April 15, 2024

Cancelled

---

April 8, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Latest issues (5 min)
• Hydro / SP mode compatibility discussion (10 min)
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

• Eclipse today! :sun_with_face: :new_moon:
• Perlmutter speed up trick: DVS in read-only mode (see Jessie's email)
• Jennifer's birthday

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: 11 Apr 2024, Claire Zarakas
• ESS-Cyberinfrastructure meeting: 15 April 2024
• ESS-PI Meeting: 16-17 April 2024

Machine outages :desktop_computer:

• CISL: none planned
• NERSC: 17 Apr, perlmutter

PR board status update

• New allometric modes: Greg testing to check b4b issues Ryan seeing. Test reverting code in EDCanopyStructureMod
• CCH water xfer functions: CCH hydro improvements coming along. Ryan now tracking down issue with non-debug hydro resulting in carbon balance errors.

Latest issues and discussions of note

• duplicate respiration vars: need to remove FATES_AR and similar history outputs
• CSS/PSS file read issue: need a check for missing data and add graceful failure

Calibration progress update

• Rosie: Adrianna and Rosie discussing approach for next steps using Adrianna's pft files (one-at-a-time no comp simulations)
  • Rosie to integrate changes from Jessie's vert scaling PR as well
  • Workflow discussion
    • Jupyter notebooks are not great from a version control perspective, but nicely steps through ensembles updates

Tutorial update

• No new update

Hydro / SP mode compatibility (Ryan)

• Ryan noting that we need to add new features to make hydro and sp mode work together.
  • As such, we shouldn't consider failure in the mode to be a bug, but a feature not yet supported.
  • Greg to take the action to add graceful failure with both modes in API35 and API34 ctsm and e3sm PRs respectively.

Walk-on topics

• Rosie: Fire BVOC emissions
  • Getting MEGAN to work with FATES: https://github.com/NorESMhub/CTSM/issues/36
    • Keeping fire emission factors in fates parameter file and push calculations in FATES to HLM
    • Rosie currently has a new parameter to map from FATES to MEGAN. Rosie doesn't like that this makes the param file not HLM agnostic.
• Small conference rooms available in B74 (314 and 202)

---

April 1st, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Latest issues (5 min)
• Recap integration workflow (1 min)
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

• Fire trunk fuel bug fix
• Allometry parameters are in!
• Hydro restart bug fix
• FATES tutorial application notices sent out. We've gotten confirmation back from most of them.

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: 11 Apr 2024, Claire Zarakas
• Eastern Region Dynamic Global Vegetation Conference: 5-7 April 2024, Woods Hole

Machine outages :desktop_computer:

• CISL: 2-3 Apr, derecho
• NERSC: 17 Apr, perlmutter

PR board status update

• New allometric modes: Ryan testing, but running into unexpected non-b4b issues
• Fire refactor: Adrianna to add patch-level weather to this.

Latest issues and discussions of note

• fire trunk issue: fixed per #1180
  • Discussion of impacts to current work.
    • Team is following up with those that were impacted.
  • Adrianna to run coarse global 100 year run to get some assessment of impact. She will update the issue with results and close it out.
  • Future refactor is to not include trunk loading in fuel class
• SP restart crash: since there is no mort in SP mode, we will avoid these in fix

Calibration progress update

• Jessie: high biomass issue may be alleviated by SP parameters from Adrianna's work. Also recent fire fix may help as well.
• Adrianna: Planning on rerunning with new pft composition to address sparse grid limitations

Tutorial update

see wins

Walk-on topics

• Adrianna: Testing stand structure faster (testing initialization)
  • Another option: use the non comp mode with large trees. See https://github.com/NGEET/fates/pull/995 for reference
• SSH security heads up: backdoor found inxz-utils version 5.6.0
  • Fedora Linux 41 users should check security alert notice

---

March 25, 2024

• Wins (1 min)
• Upcoming meetings (5 min)
• PR board status (5-10 min)
• Latest issues (5 min)
• Improving PR integration workflow discussion
• Calibration progress update (5-10 min)
• Tutorial update (5 min)

Wins and Celebrations :tada:

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: 28 Mar 2024, Shaoqing Liu
  • Reschedule Rachel?
• Eastern Region Dynamic Global Vegetation Conference: 5-7 April 2024, Woods Hole

Machine outages :desktop_computer:

• CISL: 2-3 Apr, derecho
• NERSC: 17 Apr, perlmutter

PR board status update

• Integrated: User controlled history density, Carbon flux recruit history output, hydro ERS, default allom params
• New:
  • LAI-by-PA refactor
• Updates:
  • CCH water xfer functions: Ryan implementing function unit tests
  • day length factor: Refactored and tested

Fire refactor overview

Latest issues and discussions of note

• MPI 5x5_amazon resolution on derecho: one fates regression test needs update to avoid this
• normalize crown area weighted hist

Improving PR integration workflow

• Brainstorming ideas for reducing PR backlogs
• Reviews
  • Hold more regular code walk-throughs for large PRs
  • Break-up and stage large PRs if possible
• Testing
  • Empower core fates modelers to regression test PRs they author
    • need: maintaining derecho/perlmutter fates list parity
  • Leverage tutorial containers to automate subset of regression tests via github actions
    • Related: Automated unit testing
    • E3SM tests: https://github.com/E3SM-Project/E3SM/actions/workflows/e3sm-gh-ci-cime-tests.yml
• Other logistics
  • On-the-fly default parameter file build
  • parameter patch file auto-build/checking

Calibration progress update

deferred

Tutorial update

deferred

---

March 18, 2024

Cancelled

---

March 11, 2024

• Wins (1 min)
• Upcoming FATES meetings (5 min)
• Weekly synthesis of the PR Status Board (5-10 min)
• Latest issues (5-10 min)
• Calibration progress update (5-10 min)
• Hackathon for fire emissions

Wins and Celebrations :tada:

• Land use V1 integrated into E3SM master
• Landuse data tool hosted on anaconda.org: https://anaconda.org/ngeetropics/tools-fates-landusedata

Upcoming meetings and events :spiral_calendar:

• FATES modeling meeting: 14 Mar 2024, Rachel Ward
  • Marcos to host and send out the reminder email
• Eastern Region Dynamic Global Vegetation Conference: 5-7 April 2024, Woods Hole

Machine outages :desktop_computer:

• CISL: none planned
• NERSC: Wed 20 Mar, perlmutter
• OLCF: Tue 19 Mar, center-wide

PR board status update

• CCH water xfer functions: resolves #1163, but now seeing 1D hydro failure
  • Marcos/Ryan: capping psi_min and th_min discussion?
    • Discussion with Jenny last week (water balance conservation issue).
      • Ryan: Use linear model below zero similar to Van Genukten?
      • More discussion needed
  • Greg to track down regression test failure

Latest issues and discussions of note

• hydro error reporting: Greg to handle in the CCH PR

Calibration progress update

• Rosie: discussion of disconnect between tuning and diagnostics packages for land
  • Spectral elements runs versus clm diagnostic package
  • Jessie: E3SM-unified includes E3SM-diags and ILAMB. Also zppy.
    • zppy epic issue: https://github.com/E3SM-Project/zppy/issues/385
    • Charlie: are there separate land diagnostics packages?

Fire emission coupling hackathon (Rosie)

• Goal: get fire emissions out of fates and into atmosphere model
  • NorESM + Oslo atmo chemistry model
• Calculate biomass burned and pass to HLM to calculate emissions or fates calculate emissions directly
  • emissions are pft specific suggesting we should have fates calculate directly
    • would result in duplication of emissions factors from HLM
• Charlie: is emission a function of fire "type" (i.e. anthro versus wildfire)
  • Rosie: yes, but pft is a proxy of this
• Ryan: discussion or issue?
  • Rosie: https://github.com/NGEET/fates/issues/1088 also links to NorESM issue

Hydaulic failure discussion (Chonggang)

• Option: when leaf water potential high, zero stomatal conductance and not request water
  • May improve numerical stability
  • Chonggang hoping to attempt this at the end of the month
• Ryan: preference is to implement linear model between
  • Alternate (short term?): run case at much shorter time steps to get through snags
• Charlie: Develop dry condition test failures (e.g. single site at Atacama)
  • Ryan: functional unit testing
    • satphen + hydro?
  • Greg: being able to parameterize functional unit tests would be great

Tutorial update

• deferred

---

March 4, 2024

• Wins (1 min)
• Upcoming FATES meetings (5 min)
• Weekly synthesis of the PR Status Board (5-10 min)
• Latest issues (5-10 min)
• Calibration progress update (5-10 min)
• Global stomatal conductances (5-10 min)
• Tutorial update (5 min)
• Walk-on topics

Wins and Celebrations :tada:

• Ryan's paper was published in JAMES: Nutrient Dynamics in a Coupled Terrestrial Biosphere and Land Model
  • Reminder: send out LBL blurb:

Upcoming meetings and events :spiral_calendar:

• CESM Software Engineering Working Group Meeting: Today, 4 March
• FATES modeling meeting: 14 Mar 2024, Rachel Ward
  • Marcos to send reminder and will cover the announcment email for Charlie
• Eastern Region Dynamic Global Vegetation Conference: 5-7 April 2024, Woods Hole
  • Abstract submission deadline is today

Machine outages :desktop_computer:

• CISL: Tue 5 Mar, derecho
• NERSC: Wed 20 Mar, perlmutter
• OLCF: Tue 19 Mar, center-wide

PR board status update

• Stuck:
  • New category to visually communicate what PRs are held up by bugs found in testing or waiting on upstream fixes
  • fates param file autobuild (CTSM#2336): waiting on potential conda fix in #2385
  • Leaf phen. with ST3: Greg still investigating
• Update:
  • Default allom. params.: Greg to test and integrate. Will have API update with #1128. Make sure updates don't break tests.
  • User controlled history density: Ryan updated with requested review changes. To be tested next with API-side update next.
  • V2 Landuse Change: Marcos self-assigned review. Outside reviewers post-LMWG meeting?
• Discussion:
  • Vertical scaling: Discuss email thread on clarifying scalar definitions and migration to parameter file
  • Action:
    • Ryan to create new issue for discussion about sla scaling in the biophyscial rates routine
    • Jessie: add parameters Ryan suggested to PR to tee up for future implementation without needing to worry about future API update.

Latest issues and discussions of note

• No new issues or updates

Calibration progress update

• No new issues or updates

Global stomatal conductances (Ryan)

• Making coupling HLM-FATES faster. Honing in on the right stomatal conductance.
  • Noticing how wide range of values are globally. Why values in the sahara (full fates)?
  • Ryan will run an sp mode and share results

Tutorial update

• Application is closed: 73 applicants
  • 80% of applicants prefer R
    • Greg attending R intro course over the next two weeks
• tutorial container image build updated to API 31: https://hub.docker.com/r/ngeetropics/elm-builds/tags
  • Greg to run a test case this week to ensure current dependencies work with lastest commits
  • Related: E3SM forked into NGEET org
    • Tutorial build pipeline necessitates downloading HLM tar files associated with tags.
    • E3SM doesn't tag merge commits, so I've forked the repo to create our own tags associated with API updates

Walk-on topics

• Rosie: mini hackathon next Thursday/Friday to tackle the fire emissions coupling.
  • Add to discussion agenda for next Monday.

---

February 26, 2024

• Upcoming FATES meetings (5 min)
• FATES software meeting check-in (5 min)
• Weekly synthesis of the PR Status Board (5-10 min)
• Latest issues (5-10 min)
• Calibration progress update (5-10 min)
• Tutorial update (5 min)
• Walk-on topics

Upcoming meetings and events

• FATES modeling meeting: 29 Feb 2024 cancelled due to LMWG
• CESM Land Model & Biogeochem WG meeting: 27-29 Feb 2024

FATES software meeting check-in

• Is there interest in moving this meeting to an every-other week cadence?
  • Consensus: those present like the weekly cadence and the reporting back of PRs and issue status.
  • Greg: What about the notes format? Would people prefer to engage with the notes in another format?
    • Consensus: those present like the transparency of the wiki format and that the notes are co-located on the repo with everything else (issues, discussion, code, etc.)

Machine outages

• CISL: none planned
• NERSC: 20 Mar, perlmutter
• OLCF: none planned

PR board status update

Updates:

• phosphate fix (E3SM#6222): approved and tested. In the integration queue.
• Leaf phen. with ST3: Greg spent some time with this. Still investigating.
• fates param file autobuild (CTSM#2336): final testing. Next in ctsm queue.
• V2 Landuse Change: Charlie testing. Ready for review. New:
• Day length switch: Greg and Ryan quickly reviewed. Greg to make some minor refactors.
• CCH water transfer func protections: This might help with issue #1157.
• Fire edge-case bugfix: Needs review.

Latest issues and discussions of note

• #1163 and #1165 addressed by PRs #1164 and #1163, respectively

Calibration progress update

• Jessie: finishing up respiration work; will turn into maniscript in near future.
  • Considering adoption of Adrianna's ML method for next iteration (post SP-mode calibration updates)
  • Ryan: stem respiration for calibration?
    • Jessie: Not currently. Kolby keen to investigate this for next project phase. Discussions about field-to-model integration details on going.

Tutorial update

• Jennifer: what's the accessibility goal? Will we explain how to setup the container, etc?
  • Jessie: Ultimate goal is to make this available all online with a very simple BCI method.
    • Greg: We also have a tutorial setup guide repo in development to host a readthedocs style walkthrough of setting up on a laptop.
  • Greg: OLMT experience? We might use or compare the single site subsetting function for pre-processing.
    • Jennifer: Yes, two years ago. Current attempts to use were hampered however.
    • Ryan: Running pre-process all at once?
      • Jessie: yes, we could use the sparse grid NCL method.

---

February 12, 2024

• Upcoming FATES meetings (5 min)
• Weekly synthesis of the PR Status Board (5-10 min)
• Latest issues (5-10 min)
• FATES Speed Audit (10 min)
• Calibration progress update (5-10 min)
• Walk-on topics

Upcoming meetings and events

• FATES modeling meeting: 15 Feb 2024, TBD
• CESM Land Model & Biogeochem WG meeting: 27-29 Feb 2024

Machine outages

• CISL: none planned
• NERSC: 21 Feb, perlmutter
• OLCF: none planned

PR board status update

• Integrated: FATES API refactor (E3SM#6027)
• Updates
  • Landuse V1 API (E3SM#5760): Final testing underway.
• New
  • Phosphate fix (E3SM#6222)
  • Hydro drought decid. restart fix: Fixes #1151. Relation to #1157 below?
    • Greg/Ryan to check in with Jenny.

Latest issues and discussions of note

• Hydro restart fail: seems to occur only in drier regions
  • Encountered while fixing: #1151

FATES speed audit update (Ryan)

• Can we avoid leaf layer loop: Trimming routine forces us to iterate over leaf layer depth
• Current plan: investigate Ci iteration cap and threshold. Update as necessary along with remembering the previous value.
• Initial iteration analysis update suggests that we often hit the iteration cap.
  • Initial iteration testing with ST3 mode (audit was using full dynamics however).
• ED2 experience: What about trying Newton method using the derivitives from reduced analytical form? Review the ED2 approach and share with team.
• Investigate impact of making photosynthesis loosely coupled?

Calibration progress update

• Adrianna: CLM surface data does not seem to quite match ILAMB observation. This is causing issues with gridcell to gridcell comparison.
  • Ask Nate Collier for help.
• Calibration team to review Adrianna's brute-force method and will review selected parameter ranges.

---

February 5, 2024

• Upcoming FATES meetings (5 min)
• Weekly synthesis of the PR Status Board (5-10 min)
• Latest issues (5-10 min)
• Calibration progress update (5-10 min)
• Walk-on topics

Upcoming meetings and events

• FATES performance meeting: Thursday 8 Feb 2024, 1100 PST
• FATES modeling meeting: 15 Feb 2024, TBD
• CESM Land Model & Biogeochem WG meeting: 27-29 Feb 2024
• 2024 FATES tutorial: 9-13 Sept 2024, LBNL
  • Informational website up with application form

Machine outages

• CISL: 5-7 Feb, derecho
• NERSC: 21 Feb, perlmutter
• OLCF: 6 Feb, frontier

PR board status update

• Integrated: FATES testmod update (E3SM#6018), Carbon mort reporting, Disable hydro mort., Exp. carbon starv. mort.
• Updates
  • New allometric modes: Marcos and Ryan updated. Ready for regression testing.
  • User controlled history density: Functionality testing looks good. Ready for regression testing.
    • Question: should we split 2d and multiplexed into seperate subroutines? Should we put certain multiplexed into first level or have a combo level?
      • Consensus: no strong feelings. This is a good improvement, keep as is. Look at further reorganization as we use it.
• New
  • Carbon flux recruit history output, Patch insertion, hydro restart fix

Latest issues and discussions of note

• NaN/Inf for soil matrix

Calibration progress update

---

January 29, 2024

• Upcoming FATES meetings (5 min)
• Weekly synthesis of the PR Status Board (5-10 min)
• Latest issues (5-10 min)
• Calibration progress update (5-10 min)
• Walk-on topics

Upcoming meetings and events

• FATES modeling meeting: 01 Feb 2024, TBD
  • In lieu of agenda or scheduled speaker, cancel
• CESM Land Model & Biogeochem WG meeting: 27-29 Feb 2024
  • Adrianna, Charlie, Greg submitted abstracts. Ryan to submit 2-stream abstract.

Machine outages

• CISL: Mon 5 - Wed 7 Feb, derecho
• NERSC: none planned
• OLCF: none planned

PR board status update

• New
  • fates param file autobuild: Erik wants to see this come in as initial step.
• Updates
  • FATES testmod update (6018): Should be merged to next. Waiting to be integrated to master.
  • Carbon mort reporting: Greg to start testing today.
  • New allometric modes: Jessie has some quesitions. Ryan to test and coordinate paramfile updates with #1132.
  • Diable hydro mort.: Deconflict discussion against previous #1115
  • Leaf phen. with ST3: Not passing exact restart. Investigating.
  • User controlled history density: Greg to review today.

Latest issues and discussions of note

• LU data tool fails on derecho: probably taking too much memory for login node. This will need to be addressed eventually for CLM LU build on the fly.
  • This came up in the context of checking ctsm python environment on derecho, due to a dependency issue for xesmf.
  • Erik/Charlie: test this on the develop queue

Calibration progress update

• Adrianna: Ran Ball-Berry without temp acclimation. Observational mean is more easily achieved, but emulator does a worse job.
  • Temperture acclimation without very low vcmax
    • Charlie: Claire Zarackas is finding something similar in the absence of hydraulics. So there is a scientific rationale for excluding temp acclimation.
• Jessie: met with Ryan to talk about carbon budget issue; add recruitment carbon flux. Carbon budget closes now.
  • Not sure if recruitments that die in the same timestep are showing up in the diagnostics

Walk-on topics

• Rosie: Question: Matvey fates model focus? Take over Marius work? Where does it fall in the priority?
  • Jennifer: arctic-fire stuff coming up prior to arctic-hydro. Maybe hardening?
  • Greg: This is back on API 17, so bringing it up before it gets too stale would be good
  • Charlie/Ryan/Rosie: recent hydro issues do suggest that addressing hardening is a good idea.
  • Matvey: only calibrated to 2 local sites; will it work globally?

---

January 22, 2024

• Upcoming FATES meetings (5 min)
• Weekly synthesis of the PR Status Board (5-10 min)
• Building parameter files on the fly (5 min)
• Latest issues (5-10 min)
• Calibration progress update (5-10 min)
• Walk-on topics

Upcoming meetings and events

• FATES modeling meeting: 01 Feb 2024, TBD

Machine outages

• CISL: Mon 5 - Wed 7 Feb, derecho
• NERSC: none planned
• OLCF: none planned

PR board status update

• Integrated: Two-stream, stress/drought desc, crown area output, quad solver
• Updates
  • FATES testmod update (6018): Threading issue on chrysalis. Greg to update to single threading and restest for the short term.
  • Leaf phen. with ST3: Testing today
  • Landuse data tool: Refactored tool to merge V1 timeseries tool into V2 LU x pft map tool to share functions
  • Jennifer will cherry-pick her daylight factor switch to a new PR branch to be brought main
• Check-in
  • Default allom. params.: This is good to go at this point. There is still uncertainty in biomass, but this is a good next step.
  • hydro failure disable: Marcos to review Rosie's comments.
    • Aside: Rosie noted that Marius' frost work addressed some of this. She and others are writing a proposal to follow up on this.

Building parameter files on the fly

• Ryan to look at moving this forward this week

Latest issues and discussions of note

• hydro dry region failure: Marcos has been working with Jenny on this. Recommendation is that she not trace the issue since its out of scope for her work.
  • Ryan proposing to add a fail safe for extreme outlier cases to allow fates run to complete. Still need to make sure this is actually an outlier.

Calibration progress update

• Calibration branch
  • Jessie still needs this branch until 2-stream gets to E3SM (probably a few weeks at this point). The calibration branch is pretty close to FATES main for her purposes, so no need to merge main into calibration branch.

Walk-on topics

• Erik asking about status of Marius' frost branch

---

January 15, 2024

No meeting due to lab holiday

---

January 8, 2024

• Upcoming FATES meetings (5 min)
• Weekly synthesis of the PR Status Board (5-10 min)
• Latest issues (5-10 min)
• Calibration progress update (5-10 min)
• Walk-on topics

Upcoming meetings and events

• FATES modeling meeting: 18 Jan 2024, TBD

Machine outages

• CISL: Mon 5 - Wed 7 Feb, derecho
• NERSC: Wed 17 Jan, perlmutter
• OLCF: none planned

PR board status update

• Integrated: V1 Landuse Change, FatesPFTIndexSwapper update for landuse
• Updates
  • Two-stream: Clean up. Erik requesting additional testing. To be discussed on Wed.
  • FATES testmod update (6018): Final testing completed. Awaiting review and integration.
• New:
  • Carbon Starve Mort: Deconflicted. Needs review.
  • Quad eq sovler update: Approved. To be tested.

Latest issues and discussions of note

• spitfire fuel avg warning

Calibration progress update

• Jessie presented latest results: GPP and biomass bias improvements

---

January 1, 2024

No meeting due to lab holiday