[ver1.0.0]

Post 1st-beamformed baseband data are currently stored to enable optimization of the beam direction toward target sources. However, daily BURSTT trigger data of approximately 0.5 TB impose a heavy load on the storage systems at the Fushan main station and outrigger stations, including Nantou. To alleviate this issue, we developed a data-reduction pipeline, dedispersion pack (ddpack), which reduces file size by eliminating redundant time segments estimated from the incoherent dedispersion delay. The method was implemented and validated using Fushan-triggered data recorded between 2025 October 16 and 18. The architecture and workflow of the ddpack process are presented, demonstrating an effective approach for optimizing data storage in high-rate beamformed systems.

At this moment(2025/10/31), ddpack is available on burstt14. In this work, we use burstt14 because this server has enough free HDD storage space.

[terminal] ssh_burstt14

If you don’t have any access to BURSTT servers, please request to create your BURSTT account.

[<user_name>@burstt14 ~]$ cd /data/<user_name>/analysis/ddpacktrigger/bin

If there’s no directory of /data/<user_name>/analysis/ddpacktrigger, you can copy it from burstt13.

[<user_name>@burstt14 ~]$ rsync -avh <user_name>@burstt13:/data/hmasaoka/packages/ddpacktrigger /data/<user_name>/analysis

• bin; Basic scripts
• beamform; Chih-Yi's Fast Python code for reading data and 2nd beamforming

[<user_name>@burstt14 bin]$ source ./start_bashrc.example

[<user_name>@burstt14 bin]$ bda

(bursttda) [<user_name>@burstt14 bin]$

1. Select “Pulsar” and “Trigger sent”
2. Export Table to CSV

[terminal] rsync -avh <CSVfile_path> <user_name>@burstt14:/data/<user_name>/analysis/ddpacktrigger/bin

The auto_ddpack_ver1.3.py script implements flexible control options, including --tstart and --tend to define the time range of triggered events in UTC from a CSV file, and --station to specify a BURSTT station, automatically applying its associated parameters such as data format. In addition, a --dry-run mode enables users to preview the expected output without performing the actual data reduction, ensuring safe and transparent operation. This system also allows you to skip ddpack processes with --skip-ddpack.

(bursttda) [<user_name>@burstt14 bin]$ python auto_ddpack_ver1.3.py triggers_pulsar_events_202510300650.csv --tstart "2025-10-16T00:00:00" --tend "2025-10-18T00:00:00" --station "Fushan" [--skip-ddpack] [--dry-run]

(bursttda) [<user_name>@burstt14 bin]$ ls

20251017_171849Z 20251017_174214Z ... 20251017_205219Z

[Pipeline Completed] All steps finished successfully!!