Coding workshop (London 15 17 Jan 2017) - multiply-org/multiply-core GitHub Wiki
Who are our users?
- Most users are "end users", although a couple want details on pre-processing. The latter might be done on a one to one basis
- We want show the end-results from the platform, demonstrating it over a couple of test sites, where all data is gathered and pre-processed previous to the workshop
- Our aim is to show the choices you can make as a user (eg how you process the data, SAR, optical, both?, choice of priors, etc)
RT models
- Optical: PROSAIL
- SAR: WCM + Oh model (possibly)
Infrastructure
One issue is how is the platform going to be deployed for the demo. Installing it on people's laptops is unlikely to work, as we'd need to cope with different OS and OS versions, as well as the limitations of the actual laptops in terms of storage and processing power. A better option would be to have this hosted on some cloud provider (commercial or otherwise), and to access it from a browser using e.g. Jupyter notebooks (see an earlier example running the earlier version of EO-LDAS). This simplifies things as all the data, code and so on are stored and managed in one single place. However this needs someone in charge of managing and setting everything up (and possibly paying for cloud services from Google or Amazon).
Proposed Setup:
- Hosting: 1) Run locally (see above), 2) Run cloud (who is going to pay?)
- Participants: 30 people: 10 groups of three.
- 10 nodes
- 32 Gb per node (RAM)
- 4 cores per node
- 1-2 Tb total disk space
- Virtual 'Machine' :-> Docker (available for Linux, also for 'Windows and Mac'). (graps updated version of the github on launch).
- Interfacing: 1) Standalone (electron) program with user-interface (not for the workshop), and 2) Jupiter Notebook (locally, or connect to cloud port). For individual case studies, 3) Command line interface. For interactive demonstration. (for higher API functionality with interface to config-file).
Test sites
- Munich Agricultural area (growing season March up to September 2017)
- Barrax Agricultural area (growing season May until September/October)
Later on, add some national park in East Africa (?)
Work groups for next days
- Inference Engine
- bug fixing: Jose, Tonio
- SAR RT model: Thomas W., Nicola, Jose
- state propagation: Jose, Nicola, Tonio
- Prior: Joris, Thomas R. (Jose if questions)
- Optical high resolution pre-processing: Feng, Tonio
- SAR pre-processing: Thomas W., Tonio
- Orchestrator: Tonio, ???
What can we show at the Workshop in Frascati
Case 1: Coarse resolution (Tanzania)
- optical data
- MODIS
- LAI
- faPAR
Case 2: High and coarse resolution (Barrax test site)
- optical data
- Show difference between coarse and high resolution data
Case 3: High resolution (LMU test site)
- switch observations (SAR, optical)
- put optical and SAR RT-model together (results will be crap)
- switch priors (use ground truth data as priors for LMU site)
- SAR RT-model
- set strong prior for soil moisture and retrieve LAI, and other way around
- set strong prior for SM and LAI then compare backscatter (how good are the retrieval results of the RT-model)
- SAR RT-model
- gap filling (clouds)
- Optical RT-model
- LMU site a lot of clouds, Barrax less clouds than LMU (who is the gap filling working)
- Optical RT-model
Further information for Workshop timetable
Tuesday 9:00 justification + background on limitations (platform, RT-model etc) 11:00: coarse resolution pre-processing optical data (atmospheric correction etc) 14:00: pre-processing S1/S2
What needs to be organized
- Wifi guest accouts (Peter)
- unblocking of cloud solution in Frascati (check if the setup is working)
responsibilities
- set up cloud system at Frascati (Tonio)
- download and pre-process S1 data for every test site (Thomas W)
- download and pre-process S2 data for every test site (Feng)