Meeting notes: CTSM mizuRoute - ESCOMP/mizuRoute Wiki

Link to status of mizuRoute

Notes from (roughly) weekly meeting

September 30th, 2022

To-do lists

  • Setup 0.5 degree gridded network for mizuRoute with various CLM grids. Create alias

  • Update CTSM branch mizuRoute is now pointing to.

  • Merge pull request #313

  • (a month to few months goal) Pass precipitation and evaporation for lake, and also start looking at new river-lake data.

September 23th, 2022

initial code - direct_in_place, direct_to_outlet

Need to evaluate the simulation (what should I look at)

direct_to_outlet is slow. need to optimize the algorithm


After meeting with Dave, we decided to do the followings:

  • Run mizuRoute and MOSART with 0.5 degree gridded network (both MOSART and mizuRoute gridded network has the same flow direction, and land mask), and compare the simulations w/ and w/o irrigation and w/ and w/o irrigation threshold, and negative flow handling option.
  • Run the results with Sean. if qualitatively similar, move on to lake implementation.
  • Then maybe come up with better evaluation strategy for irrigation, and negative flow handling
  • Implement water budgets in mizuRoute.
  • Implement direct_all_outlet (bypass routing and send all the flow to the outlet). Sean's suggestion a few month ago

September 14th, 2022

Initial version of direct_to_outlet option is completed. mizuRoute domain decomposition can assign tributaries and their mainstem where outlets of the tributaries are located to different tasks, so MPI communication is used to send negative runoff to the outlet (from one task to another).

Tested with two LND-ROF grids: f19_f19_mg17_rHDMA and f19_f19_mg17_rMERIT.

Need to check the simulations, then will need to speed up this process (checked the timing of direct_to_outlet option. over 90% of total rouitng time is spent on sending fluxes to outlets).

Also, need to turn on clm option: limit_irrigation_if_rof_enabled = .true. and change irrig_threshold_fraction = ?? (check CLM namelist documentation)

August 24th, 2022

These are questions discussed:

  • Difference in CLM runoff (QDRAI+ QOVER+ QDRAI_PERCH) between runs with irrigation-off and irrigation-on (see image below- 2002-06 mean runoff, run w/ irrig minus run w/o irrig). It seems to be odd that CLM runoff with irrigation on increases runoff over CLM runoff with irrigation off. Need to turn on threshold in CLM (in last meeting with Dave and Sean, I misunderstood "threshold" (I thought threshold should be on in ROF option not CLM option)?? clm5_f09_f09_mg17_rHDMA_ro_irrig_comp_06

  • Concern about mismatch between mizuRoute HRU polygon and CLM grid along the coast line, especially the case where a CLM grid box is not overlapped by any mizuRoute HRU polygons at all. What will happen to the CLM runoff at those grid boxes?? Runoff is sent to the ocean directly (without passing mizuRoute)? The case where the mizuRoute polygon is not overlapped by any CLM grid boxes and the case where mizuRoute polygon is covered partially by CLM grid boxes are probably ok. But need to check. See Issues #301 184195558-02329528-9360-4aa8-a5ab-1c5d2e2da843

  • Need Greenland and Antarctica river network for fully-coupled run. Maybe patch the MOSART gridded network to HDMA and MERIT-hydro. MERIT-hydro has Greenland river network, not Antarctica.

  • What will happen to discharge (routed runoff by mizuRoute) that pours to the Endorheic basin (e.g., great basin). For now, mizuRoute routes the runoff to the outlet regardless of whether it's outlet to the ocean or Endorheic lake. Not sure if runoff at Endorheic outlet needs to be sent to the ocean somehow?

Other items:

  • codes for direct_in_place for negative runoff (qgwl and irrigation) are merged at PR #297.

  • now working on direct_to_oulet option.

August 1st, 2022

Discussion item for next meeting:

  • Talk with Dave about irrigation working now

Apply limit??

  • Talk with Dave about VOLR for headwaters top or bottom of HRU
  • mizuRoute puts runoff from CLM at an outlet of HRU (see overall mizuRoute procedure). Because of this, reach routing is not performed at headwater HRUs, and no volume is computed. This needs to be resolved.

Sean said, zero volume for headwater would be probably ok, and try testing.

  • Talk with Dave about QGWL and how it should be handled. See #164
  • Now Flrl_rofgwl is imported into mizuRoute, but not doing anything with it.

handling qgwl is essentially the same as irrigation. need to send negative q to ocean.

  • Talk with Dave about priorities:
  1. QGRWL
  2. Lake work (need to pass Precip and ET through coupler, Erik will help with this)
  3. Lake network data (need to talk with Shervan)
  4. Send back amount of water removed from Lakes? Or is VOLR sufficient see -- #170 (this is longer term issue correct?)

Finish up QGRWL and irrigation are higher priority and make sure that water balance is close. Implement all direct to ocean option to test all runoff from CLM == water going to ocean. For lake, will need to work out details....

  • Will asked me if I need LMWG CSL allocation for 2022-2024. if so need some short paragraph for the request by Friday.

This is development work, so no worry for this (LWMG allocation would be run more for science)

July 27th, 2022

June 12-18 and 19-25, 2022

No meeting due to other priority (Erik) and traveling (Naoki)

June 8th, 2022

Talked about how to turn on irrigation in CLM. Make sure if these are turned on, mizuRoute works.

CLM namelist

  • irrigate = T

Other related namelist variables:

  • irrig_river_volume_threshold

  • limit_irrigation_if_rof_enabled

  • use_groundwater_irrigation

June 1st, 2022


Looked over irrigation part in MOSART, a few questions came up:

  1. After irrigation demand is imported to ROF space from CLM, where does mosart take out the water from? Does it take water at the same location as irrigation demand (crop area)? or realistically taking it out elsewhere (e.g., somewhere upstream large water body or nearest river mainstem)?

  2. If irrigation demands are too much (more than water volume in river), actual water taken out is communicated back to CLM? (MOSART does not seem to do that)

Other minor discussion:

In fortran, is it ok to supply a portion of array (i.e., array(lower:upper)) into subroutine or function argument? does compiler make and copy temporary array in memory, which likely slows down.

May 13th, 2022

gauge meta.

Currently mizuRoute outputs discharge (and other variables) at all the river reaches in the network. I have a few global river discharge at gauges.

  • Dai, A. (2017). Dai and Trenberth global river flow and continental discharge dataset.

  • Beck, H. E., de Roo, A., & van Dijk, A. I. J. M. (2015). Global maps of streamflow characteristics based on observations from several thousand catchments. Journal of Hydrometeorology, 16(4), 1478–1501.

D17 data has ~900 sites (large river outlet) B15 data has ~21,000 sites (including inlands)

Does it make sense to output only at those sites? -> would be nice to have user output options - all reaches or selected reaches at gauges.

Wish list: Create diagnostic packages. look at examples and replicate this for mizuRoute ouptu, then build new hydrology specific. This is probably not high priority

higher priority is 1) irrigation, 2)enable lake, 3) tracer to route ice (snow cap in clm). then ocean coupling

Other important task is to send total water mass back to clm.

May 4th, 2022

Reviewed mizuRoute overall procedure and hillslope hydrology (See Apr 20th notes) for brainstorming how to couple both. If the spatial resolutions of mizuRoute and hillslope hydrology are the same, that would be ideal. if not, remapping could still work technically.

Reviewed coding guidelines (new wiki page)


Naoki: Merge #274. Work on incorporating gauge meta.

Erik: test on Izumi.

Apr 20th, 2022

With Sean

The hillslope hydrology option needs flood information from the river model. This may require mizuRoute to use a compound channel (or use river channel depth) and compute flood volume when routed flow depth exceed the depth. The river depth for each reach as a river parameter is required, or estimated with some empirical equation, just like river width computed as a function of total upstream area now.

Will have to handle negative flow and irrigation, which are imported from CLM. Currently mizuRoute is able to inject or extract water from any reaches. mizuRoute needs to know which reaches, how much and when (this can be from user - time series of water injection/abstraction or CLM through the coupler).

It would be good to create test case for a basin (e.g.,Columbia, Colorado River) for Sean to to be able to test CTSM-mizuRoute

overall mizuRoute procedure

From Sean. river-hillslope interaction concept

Apr 14th, 2022

With Dave L.

Naoki -- still trouble with the build? I did get the openmpi intel build fix PR to go into cime.

Naoki -- Tried the latest CTSM (cloned on Apr 14th) master branch, with SMS_D.f09_f09_mg17.I2000Clm50SpRs.cheyenne_intel.clm-default, and failed.

NLDAS-USGS case may require less CPUs. I tested with SMS_P360x1_Lm1.nldas2_rUSGS_mnldas2.I2000Clm50SpMizGs.cheyenne_intel.mizuroute-default and was able to run

It would be good to talk to Sean about hill-slope and river interaction. River model needs to track, discharge, volume and water depth. water depth may be needed for hill-slope and river interaction.

Mar 31th, 2022

NLDA-USGS issue. need to describe what happen in issue (Erik) and Naoki add the cause (need to look at the code)

Warning: external config change. mizuRoute points to the specific tag and the lates branch is commented out now in .gitmodule

gitHub auto close issue when merging the pull-request. Need to add close (fix, or resolve) #issue number in main comment section. Then if the pull-request is merged, the issue tagged will close.

Mar 17th, 2022

Funding for 25% of Naoki's time for CESM work will start after May 15th. We will normally always do these monthly meetings, but will often be short for the update to Dave. The agenda will be a normal kind of standup meeting to give an update of what we each are working on, as well as anything we are stuck on, that we need to get help with.

Erik should do things needed for Naoki to make progress in mid May. One issue we identified was for the lake fields to be passed in the coupler.

Sep 16th 2021

  • Dave tells me mizuRoute's importance is that we don't have in-house ROF component expertise. so hopefully mizuRoute can handle that. mizuRoute also allows us to run on HRU's which is likely important for some types of work for CTSM.

  • Ok, good to know.

  • cime issues and PR.

  • Included off-line mapping file in grid_config. can merge soon.

  • mizuRoute issues for coupling

  • mpi-serial issue

  • Running with different number of threads for different components? Does that work for nuopc now?

  • yes. should be able to do. need to try.

Sep 9th 2021

Items to talk about

  1. hybrid run CTSM
  2. Update on incorporating recent CTSM update

Aug 27th 2021

Identified next tasks

Next task items

  • 1. create hcru_hcru_HDMA and hcru_hcru_rMERIT grids.
  • 2. Incorporate another CTSM update.
  • 3. Remove time dimension from restart file, but write timing information.
  • 4. Check if MPI-serial works to compile without MPI library.
  • 5. Slowly work on tracer (maybe).

Aug 18th 2021

Meeting with Jim and Mariana

PIO history file writing was resolved at #203

pioSystem was initialized at CIME, and do not need to be initialized within mizuRoute. mizuRoute was confused between pioSystem initialized at CIME and the one initialized within mizuRoute

May 12th 2021

  1. PIO issue due to cime update in ctsm. cannot close netcdf.

  2. Area correction. issue #183. very low priority. this would be needed for inconsistent projection of the grids across the model components

April 29th 2021


  1. Erik continues on CTSM updates and close to finishing testing.
  2. Naoki makes mapping file between 0.5 LND grid and river network.

April 21th 2021


  • 1. ctsm update.


1. short-archive issue

Erik will work on this and let me know. Once it is done, 1. Pull ctsm from Erik. 2. run manage_external/checkout_externals. 3. pull cime.

April 14th 2021


  • 1. short archive issue.
  • 2. restart test with ERS issue.


1. short-archive issue

Erik fixed. ROF name needed to be mizuroute instead of mizuRoute. merged into pull-request NCAR#181

2. restart test issue

Fixing short-archive issue fixed restart test. restart test passed

April 6th 2021


  • 1. restart test with ERS issue. update from Erik.


1. restart test

Looks like cime issues and need further investigations

March 31st 2021


  • 1. restart test with ERS did not go successfully (though looks like it did restart)
  • 2. how to set new history frequency.


1. restart test

Just to be informative and compatible with other components. Include current time step to start in restart file, and read in and add one. Problem in restart test is due to the ROF short-archive problem Erik will fix and push it to mizuRoute cesm-coupling branch (in my personal). This would fix restart test.

2. history file frequency.

It is more complex than I thought- history file vs restart file timing. If restart happen in the middle of month and history file is monthly, restart run writes in the existing file not new file.

Use these namelist values to control how history file is written (common way across the ctsm components)

hist_nhtfrq: the frequency of writing hist_mfilt: how many of those time-samples go on the file before you finish writing.

e.g., if hist_mfilt = 1, only write one time sample per file. If hist_mfilt = 5 write 5 time samples on each file.

For example if frequency is daily. and hist_mfilt=365 then put daily values in a whole year worth on the file. if frequency is yearly, and hist_mfilt=1 then that would put the yearly average for each year on one file.

mid-term plan to do this.

March 24th 2021


  • 1. New mapping between lnd (1-degree) and ROF. 1) fv0.9x1.25 (lnd), MERIT_Hydro (rof), 2)fv0.9x1.25 (lnd), HDMA (rof). /glade/work/mizukami/py_mapping/mapping_conversion/data/
  • 2. How to activate restart writing?
  • 3. Variables sent back to CLM from mizuRoute. rtmCTL%volr, rtmCTL%discharge, and rtmCTL%flood. How are these used in CLM?


1. New mapping

Wait for Erik to correct names.

2. For restart test

Use ERS instead of SMS when making test case: e.g.,

./create_test ERS_Vnuopc_Lm5.f19_g17_rMERIT.I2000Clm50SpMizGs.cheyenne_intel.mizuroute-default -r . --no-build --queue premium --walltime 01:00:00

Need to implement rpointer.rof reading statement in code and assign restart file name in mizuRoute.

3. ROF to CLM variables

Don't worry about now. They are not used in CLM.