Commdyn weekly call

• Date: 2014-09-19
• Participants: Matt, Corinna, Chris, Andrew, Peter, Lauren

Agenda and discussion

• Run manager and analysis sharing update
  • use cases
  • worked on how to implement run manager in R
  • need to override functions and methods
  • thinking about organization of github repo and how classes would be laid out
    • planning to use S4 classes
    • Peter has a design document on his local machine
  • Run manager discussions overview doc (by Peter and Lauren W): https://docs.google.com/a/nceas.ucsb.edu/document/d/1ruXaqY_8hvBX6rTpyHXAUcHVkgM9KQUlz25nvqYHqrw/edit#heading=h.ouf209f6hgeh
  • Can we just use the ORE package as the serialization of a run for the time being?
    • And if so, do we need to also have an index/search mecahnism on the local disk?
    • including the ability to browse and search through past runs
  • Chris: have a listObjectsstyle method for locally listing all of the entities that are produced
• R package updates
• other items
• package creation

  • matt: d1 r client might be updated to write and read local datapackages

  • one approach is to have a package created automatically from all prov relationships and derived objects for a run
  • should we allow hierarchial data packages, multiple packages per run
• how best to share design & implementation ideas
  • put design documents in github, as markdown
• we should be able to trace provenance across different tools
  • e.g. derived product created in R, visualization from this dataset created in matlab
  • this is covered in paper by Bertram, et. all:
    • http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5671861
• use cases
  • from Corinna
    • scientist has 80 input datasets but they only want to make public the aggregated dataset
    • matt: can't publish a dataset that only has a local identifier
      • we may have to automatically assign d1 ids and archive local datasets that are included in a run

Action items