Following the Development HEAD - toltec-astro/tolteca Wiki

Welcome to the tolteca wiki!

Following the HEAD

TolTEC is in the alpha-release and is still in active development.

The tolteca.simu and tolteca.reduce interface is not stable and this page keeps track the latest breaking changes which may require the user to re-clone the repos, re-compile the source code, or re-run the simulation/reduction. Please re-visit this page frequently or if you find any issue.

2022-01-14

citlali will not compile with the latest version of the Boost library (1.77) when using Clang13 due to c++-20 requirements. To compile, install boost via homebrew and add -DUSE_INSTALLED_BOOST=On when running cmake.

2022-01-12 (NEW VERSION, BREAKING CHANGE)

The tolteca version v1.0.1 is released. The changes includes:

To start using the new simulator, take a look at the example yaml files in https://github.com/toltec-astro/tolteca/tree/main/tolteca/data/examples

The new simulator is invoked with tolteca simu, while the old simulator still works, which is available with tolteca simu0

IMPORTANT: The toast atm is not enabled yet for the new simu. So if you need to use TOAST3 atm fluctuations, run simu0!

To see the full description of all config items, go to the doc folder of the workdir (created by tolteca setup), check 00_config_references.txt.

To migrate from old simulation config, the recommended way is to create new workdir, and copy over the old *_simu.yaml config, and edit it such that it works with the new config schema. Here is a list of items to be edited (not exhaust):

2021-11-09 (BREAKING HOTFIX)

The config to enable TOAST atm model is revised. The ??_simu.yaml needs to be updated accordingly for enabling the TOAST atm model.

To enable, edit in the ??_simu.yaml sources list, toltec_array_loading config dict, atm_model_name to toast:

simu:
    <other config entires ...>
    sources:
        - type: toltec_array_loading
          atm_model_name: toast
        - <other sources ...>

2021-11-08 (BREAKING CHANGE)

The tolteca version v0.3.0 is released. The API of the tolteca.reduce is re-worked to allow future extension and custom reduction steps and pipelining at the high level (Python based workflow). This version only support citlali >=1.0.

Also enabled in this version is the TOAST3 (https://github.com/hpc4cmb/toast/tree/toast3) atmosphere fluctuation model.

This changes breaks existing config.yaml files which are detailed below:

tolteca workdir

Old tolteca workdir can still be used. However, newly created workdir from tolteca setup will contain updated config entry schema documentation doc/00_config_dict_references.txt and updated example files doc/*.yaml.

In particular, the example file doc/70_redu_simulated_data.yaml contains extensive inline comments explaining the new reduce config dict.

reduce config dict

The old ??_reduce.yaml file needs to be updated following the schema defined in tolteca.reduce.ReduConfig. For those who reduce the simulated data, the example file doc/70_redu_simulated_data.yaml can be used, after modifying the reduce.inputs[0].path to the simulated data output path (i.e., the simu.jobkey in the ??_simu.yaml file).

TOAST3 ATM fluctuation model

The simulator can produce simulated data with TOAST3 atmosphere fluctuation model. To enable this feature, add in the simu config toast_atm: true under simu key:

simu:
    <other simu config entries>
    toast_atm: true

Note that toltec_array_loading source has to be disabled by commenting out the item in the simu.inputs list.

Python package toast3 has to be installed separately. As of now, we do not have sufficient documentation for end users to follow other than those provided in the its GitHub repo: https://github.com/hpc4cmb/toast/tree/toast3.

The TolTEC unity cluster shared TolTECA software environment is updated and can run the TOAST3 ATM simulation.

2021-10-28

Version v1.x of citlali has been released and is now public. This version is a complete refactor of citlali and integrates the refactored common_utils library now called tula, as well as the CONAN C++ package manager. Please review the README.rst file (https://github.com/toltec-astro/citlali/blob/v1.x/README.rst) in the citlali repository to see the changes to the install instructions and base requirements. Citlali v1.x uses C++-20 and an update to compiler versions (LLVM 13+ or GCC 10+) may be required.

This version also uses a completely new yaml config file, an example of which can be found at https://github.com/toltec-astro/citlali/blob/v1.x/data/config.yaml.

Many features of citlali v1.x are still in the debugging and deployment phase and will continue to be updated and released in the near future as they are completed and issues are fixed.

2021-09-21 (BREAKING CHANGE)

The version v0.2.0 is released with a complete rework of the config loading system and commandline API.

To update, reinstall both tollan and tolteca with --upgrade switch, if installed via pip.

For installation via git clone, do git pull in both tollan repo and tolteca repo.

The tolteca workdir created in previous version will stop work. An error message will be shown with instructions of how to proceed for the migration when you attempt run simu or reduce after updating to v0.2.0

You'll need to recreate the workdir with tolteca setup, and copy over any user config files (60_simu.yaml, 80_reduce.yaml, etc) to the new workdir.

Some of the config keys have been changed:

2021-08-25

The low level config has been changed in the latest version of citlali for handling multiple calibration items. Existing low level config files generated by tolteca reduce has to be regenerated.

2021-08-11

Compilation of citlali with llvm 12.01 failed due to an invalid option in llvm-ranlib. The option has been removed in the latest version of citlali.

The CCfits.cmake file in citlali has been updated to account for further issues preventing the compiler from finding cfitsio.h. Pull the latest citlali/common_utils/cmake version for this fix.

2021-08-09

The cmake installation step for CCFITS in citlali broke due to an out-of-date version being referenced. Do a git pull in citlali/common_utils/cmake to get the updated version of CCFITS.cmake.

2021-08-03

The tolteca simu command line interface now accepts arguments that overrides the entries set in the YAML config files.

In particular, the new option "exports_only" is added for the simulator to export the mapping pattern as LMT Observation Tool (LMTOT) scripts. Currently, only Lissajous and Raster map are supported. We are working on adding more.

To export, run the follows:

$ tolteca simu --exports_only

This will print the exported LMT OT script on screen, and also write the file named {jobkey}*_exported.lmtot in the output directory.

2021-07-21

Updates and fixes to citlali:

2021-07-20

Updates and fixes to citlali:

2021-07-13

Added a new mapping pattern "Rastajous" to the simulator. The example config can be found here: https://github.com/toltec-astro/tolteca/blob/aeba68d263d25254b65105951c9a0d6e06925759/tolteca/data/examples/60_simu.yaml#L35

Added a_fwhm and b_fwhm to the generated apt from tolteca.simu. Rerunning the simulation is required for citlali to make use of this info to generate more accurate kernel maps.

2021-07-10

Added tolteca command tolteca check that checks the installation and configuration and outputs a summary for potential issues and the fix.

Added a new mapping pattern "DoubleLissajous" to the simulator. The example config can be found here: https://github.com/toltec-astro/tolteca/blob/aeba68d263d25254b65105951c9a0d6e06925759/tolteca/data/examples/60_simu.yaml#L35

2021-06-29

The citlali has been updated in various places, including enable handling the proper conversion of timestream to physics unit internally. The apt_*_trimmed.ecsv files generated by the tolteca.reduce module need to be re-generated. This will happen automatically if you run citlali with tolteca reduce, but will not be done if run directly the citlali executable with the generated low-level config.

Some of the low level config has been updated, you'll need to revise the reduce YAML config (check the updated 80_reduce.yaml here: https://github.com/toltec-astro/tolteca/blob/main/tolteca/data/examples/80_reduce.yaml):

2021-06-28

The SPDLOG upstream has updated and breaks the citlali. If you found any compilation error, please run git pull && git submodule update --recursive to update the repo, delete the relevant build dependencies rm -r build/_deps/spdlog-* and re-run cmake.