Parallelized Pipeline - danecross/SOM-photoz-BFD GitHub Wiki

As seen here, a basic pipeline looks like this:

  1. Generate GaussianSims using the full deep field catalog
  2. Train wide SOM (wide_SOM)
  3. Train deep SOM (deep_SOM)
  4. Classify Wide Catalog with the wide_SOM
  5. Classify Deep Catalog with the deep_SOM
  6. Classify simulations using wide_SOM
  7. Match simulations to deep catalog and get their deep classifications
  8. Make redshift map with PZC object
  9. Make tomographic bins with TomographicBins object

Many of these processes do not reply on each other and can be run in parallel. To see this, let's split them up into phases where the steps in each phase do not rely on each other.

Phase 1

  • 1.1 Generate GaussianSims using the full deep catalog
    • Cut deep catalog based on which deep galaxies make the signal-to-noise cut
  • 1.2 Train wide_SOM

Phase 2

  • 2.1 Train deep_SOM on cut deep fields
  • 2.2 Classify the wide wata
  • 2.3 Classify the generated wide fluxes from the simulations

Phase 3

  • 3.1 Classify the deep fields

Phase 4

  • 4.1 Match classified deep galaxies to simulations

Phase 5

  • Make P(z|c) and Tomographic Bins