2020-04-20

RI presentations

ERIC-CERIC (A. Hafner)

- Oasys framework (based on orange)
- syned: abstract descriptor of widgets (source, optical elements)
- aljosa now expert to develop the code
- wise: raimondi et al (2015): numerical integration of huygens-fresnel (no FFT)
- -> very demanding in resources
    - ported wise-idl code to py
    - application: hard xray fel, grazing incidence
- remote access tool based on xpra and kubernetes
- runs in docker but users have access to their files
- access via VUO (elettra users)
- get access (RAFEC) to oasys canval in browser with screen sharing
- panosc objectives for oasys:
    - make available for other facilities
    - interoperability
    - documented beamlines
    - stregthen user communities

EuXFEL (J. E)

- SimEx: start-to-end simulation for advanced light sources
- current project:
    - hydrated protein
    - demonstrator for crystal diffraction
        - can generate diffraction patterns
        - WP4 tools can then analyze data
        - gauss source generator

- future projects:
    - pmi codes not well integrated
        - gromacs
        - cretin
        - refactor and improve documentation
        - need to update tutorials, currently too difficult to get started for new users

ELI-ALPS

Zsolt Lecz: Laser driven high reprate neutron sources at ELI-ALPS

- Transmutation project
    - generate ions, shoot on activated material → DD reaction, produce neutrons
    - HPL on Al foil
    - neutron prod. scheme: M. Roth et al. (2 stage scheme)
    - water droplets, 10-100 n/shot, 10^6..7 n/sec., isotropic emission at 2.45 MeV
    - converter script to produce neutron distributions in openpmd format

Mousumi Uphadhyay-Kahaly

- Presentation ELI-ALPS
- DFT: Ground state material properties ↔ strong field perturbations
- ASE: py environment for DFT sims (+TDDFT, quasiclass)
- Nomad-db: deposit material properties db

ESS (Mads Bertelsen)

- McStas dev team
- partly involved in WP8
- monolithic software
- new team, now more dedicated to panosc
- mcstas script:
    - py api to mcstas
    - pipy supported
    - used on beamtime, came out very flexible
    - integrated CI (travis)
    - HPC interface
    - lack internal review
    - API compatible with vinyl
    - sims can help understand background noise

- McStas:
    - move from phragmas to functions
    - gpu in beta (openacc)
        - large range of speedups

- openpmd extension:
    - now officially in beamph. extension
    - needs mpcl/openpmd conversion tool, ess showed interest

- WP8:
    - eneutrons migrated to ess servers (pan-learning)
    - has simple mcstas sim interface
    - no x-ray caps till now
    - will soon contact us

- wishlist:
    - vinyl server
    - CI
    - documented install procedure

Presentation of tasks

- Carsten presents and explains the tasks according to [gh issues]([github.com/PaNOSC-ViNYL/workshop2020/issues](https://github.com/PaNOSC-ViNYL/workshop2020/issues))

Teams:

Team 1: Mads + Zsolt
Team 2: Mousumi + Shervin
Team 3: Carsten + Juncheng + Aljosa

2020-04-21

Morning scrum meeting

Carsten:

- Started defining the vinyl base classes, relevant gh issue: https://github.com/PaNOSC-ViNYL/pyvinyl/issues/1

Mads:

- start work on abstract classes, work towards simex for now

Aljosa:

- study paper on beamline ID23 → paper is in issues
- use raytracing (faster)
- contact juncheng to coordinate interface to simex
    - use openpmd format ## Mousumi:
- work on presentation on usage of ASE
- need to discuss points 4- in relevant gh issue

Shervin:

- nomad not a solution for temp. workflow
- nomad can convert various qmd formats to own internal formats
- Files in the nomad repository not match the metadata systematically, finding files in Nomad is not an easy task

Juncheng:

- Make GAPD write pixelated images, so far only q,phi coordinates
- Consider treatment of coherence in GAPD

Zsolt:

- Finish pipeline to feed epoch data into mcstas

Afternoon scrum meeting

Shervin

- Is there a data sharing mechanism, so eg ILL staff can access data that
  was measured at EuXFEL or ESS without needing a serparate login. 
    - This may be part of WP3 or WP6 but details are not known.

Discussion about api design

- McStas is monolithic, so too much differentiation into calculators not
  needed or may even lead to problems. possible solution: make special

mcstas calculator, derived from abstract base calculator.

Juncheng

- GAPD pixelization, do it as in pysingfel
- calculate n photons from intensity:
    - possible issues in pysingfel

Shervin

- looks into powder component for mcstas

2020-04-22

Morning scrum meeting

Shervin

- Tutorial mongodb tutorial
- will continue next week to setup prototype sim metadata db
- talked to mads about ase output feed into mcstas
    - need more info from mousumi
    - will work on cif to hkl

Mads

- Reading simex source to understand calculator classes
- wanted to use simex base classes, problem with installation on mac
- suggest to install on a linux cluster

Aljosa

- suggest to install simex on cluster to mitigate dependency problems
- will continue on beamline id23

Juncheng

- went through pysingfel equations for diffracted intensity and started implementing pixelation in 

gapd

Zsolt

- finished example hdf file to be fed into mcstas
- needs more input on parametrization -> juncheng will show simex demo notebook to interested

Mousumi

- works on presentation
- wants to have analysis in same environment as sim
- wants to use tddft as rad-matter interaction
- presentation to come after review and comments from 
- works with shervin on nomad output and logs
- problem with connecting to maxwell

Carsten

- continue on vinyl-apis, start with top level class so mads can work against pyvinyl asap.

AoB

- Nicoletta (WP9) asked juncheng for content for social media postings, should be posted in the workshop slack first before sending out.
- Carsten will attend the PMC meeting this afternoon so not available for the afternoon scrum.

2020-04-23

Morning scrum

Carsten

* continue on pyvinyl, had to figure out how to do the type checking

Mads

* prototype for mcstas api using calculators and parameters

Juncheng

* Back on track to adjust GAPD code

Zsolt

* Trying to catch up with OO programming in py

Mousumi

* Conversion tool to convert QE output

Presentation by Mousumi

* DFT and TDDFT sims for s2e simulations
* Use ASE as interface to multiple DFT codes
* installation instructions 
-> can also use conda but DFT codes must be installed separately
* need a central pseudopotential repository
* crystal structures must come from somewhere -> eg COD
* QE has own file format
* <-> could use ase's converter (write) or openbabel
* perspectively also tddft can be run through ase

2020-04-24

Morning scrum

Carsten

* implemented dump, work on serialization
* will have to 

Mads

* will continue on simex/pyvinyl abstrac calcs
* asks for more details on features of pyvinyl to be able to take over some
  from carsten

Juncheng

* has finalized GAPD refactoring
* write interface to GAPD (GAPDCalculator)
* will try to catch up over weekend

Zsolt

* made progress in structuring code. shows current status. question about
  accessors in simex. will convert script to notebook and adopt simex style

and accessors.

Aljosa

* plan to have first prototype for simex in oasys with python scrip widget.
* original plan to pipe data directly but carsten recommends to dump to file
  between shadow and simex

Shervin

* played with mongo
* install ase on laptop, rewrite instructions in script
* use ase tools to QE <-> ASE
* create openpmd from zsolt's script to inject into mongodb

2020-04-27

Morning scrum

Carsten

* <++>

Mads

* Introduced basic checks of parameters in McStasParameter class
* Worked on introducing input_path in McStasScript as required by SimEx

Juncheng

* Discuss data format from ray-tracing with Aljosa
* Finish SimEx GAPD calculator unittest 

Zsolt

* Continues the implementation of Calculator and Parameter classes in SimEx needed for the demo laser-driven neutron source 

Mousumi

* <++>

Aljosa

* figure out output of Oasys objects
* has successfully installed SimEx and ran the test scripts with Juncheng's support

Shervin

* use Zsolt's script to produce an openPMD file to insert into MongoDB
* check how scheme validation works in Mongo
* find tools to convert from hd5 to json openPMD

2020-04-28

Morning scrum

Carsten

* <++>

Mads

* Uploaded repo for vinyl-mcstas prototype, had to implement input path
* needs more unit testing
* can take more work for the remainder

Juncheng

* working on bugs
* try finishing gapd calculator
* beam parameters in oasys, discussion with aljosa. suggested to make a
  polychromatic beam. 

Zsolt

* Uploaded files to neutrontools repo -> Mads will review, branch
  simex-compatible

Mousumi

* Tested ase converter back and forth, works ok
* calculate form factor: 
* need to explore interesting target for mcstas diffraction calculation

Aljosa

* will finish example on simex in oasys today
* can also do polychromatic diffraction with spectrum extracted from rays
* needs to figure out meaning of columns in oasys/shadow

Shervin

• went through ASE doc to provide only last iteration data when converting QE into CIF

  • tools for CIF -> QE -> CIF -> HKL are available to get the form factors for McStas

• openpmd insertion into mongo: more natural in json format, spent time on conversion. no conversion as of yet. Several issues on github on that:

  • https://github.com/openPMD/openPMD-api/issues/459
  • https://github.com/openPMD/openPMD-api/issues/290
  • https://github.com/openPMD/openPMD-api/issues/171

  can work around by saving directly into json.

• tried insertion into mongo with file. openPMD JSON too verbose, makes query difficult. Need scripts to convert openPMD json into slimmer BSON

• will spend time with mads, mousumi.

• need common compute resources -> juncheng will contact wp6p

2020-04-29

Morning scrum

Discussion on Format of Demo:

Shervin: - ask about repository github - jupyternotebook + documentation

Mads: - Set up a quick powder diffracttor instrument for Shervin - Remind the observers to attend the presentation - Carsten can send the invitation

Mads:

- Add more tests for McStats with new functions
- Review Zolts code
Today:
- Convert mcpo to openPMD
- Make Zolts output as the input of McStas

Shervin:

- mcpl into openPMD could be to much work?
- Mads: openPMD has its own API to write 

Ajosa:

- Needs help to write output conforming openopmd standard
- Will use Oasys to python code
- Oasys + SimEx docker to be prepared

Zsolt:

- Parameters as a dictionary

Shervin:

- Jupyternote book for Mousumi, to run ase. To find the right potential.
Today:
- HKL fileformat for McStats.

Carsten:

- Documentation for new pyvinyl API
- Short meeting at 9:00 AM

2020-04-30

Morning scrum

Carsten

* start demo notebook, 

Mads

* looked into hdf writer from mcpl (?)

Juncheng

* aljosa's environment works, prepare demo for mono and polychromatic source
  with oasys beamlines.

Zsolt

* jupyter notebook demo ready, will test some more

Mousumi

* absent today

Aljosa

* advanced with juncheng, has 2 interfaces: oasys gui and mock py script
  widgets: openpmd from shadow, gapd calculator & jupyter notebook bypassing
  oasys only shadow for raytracing, attached to GAPDCalculator. Marco maybe able to 

Shervin

* worked on demo for mousumi, include mads neutron instrument. uploaded to
  github. write readme with installation instructions. need mpich (mcstas

crashed with openmpi, works with mpich).