2.2.1 loadSessions() - shunnnli/NeuroDAP GitHub Wiki

Overview

This functions reads in the path of session; extract, preprocess, and sync all the data; and save it as separate mat files inside the session folder. See also in "General overview: preprocessing phase" and "1.2 Phase: preprocessing" for info.

Inputs and options

Required inputs

  • sessionpath: string, points to the file path of the session

Options

  • Record params

    • withPhotometryNI: logical, whether there is photometry signal in NIDAQ acquisition system, default is false
    • photometryNI_mod: logical, whether photometry signal in NIDAQ acquisition system is amplitude modulated, default is false
    • photometryNI_modFreq: double, what is the modulation frequency of photometry signal in NIDAQ

  • Labjack concat options

    • labjackSetup: string, signals which specific labjack setup will be used in analysis. This will determine which specific version of concatLabjack_setupName() function will be used. Default is 'Shun'.

  • Reload options (unfinished)

    • reloadAll: logical, default is false. If false, skip the whole function if sync_.mat is found (recommend using this one)
    • reloadNI: logical, default is false. If false, skip loading NI and use previous loaded NI data (not recommend)
    • reloadLJ: logical, default is false. If false, skip loading LJ and use previous loaded Labjack data (not recommend)
    • reloadCam: logical, default is false. If false, skip loading Cam and use previous loaded Cam data (not recommend)
    • reloadImec: logical, default is false. If false, skip loading Imec and use previous loaded Imec data (not recommend)

  • Stimulation digital line params

    • invertStim: logical, default is true. Invert signal from arduino for stimulation. Very specific to Shun's task

  • Photometry related params

    • recordLJ: double, default is [1,1,0]. If labjack-based photometry recording is used, this indicates which channels in Labjack contains signal. This value should be already defined by the time of recording by running run_labjack.mat.
    • nSampPerDemodBin: double, default is 1. This determines how many sample will be in for each demodulation bin. This will be automatically calculated in demodulateSignals() based on desired downsample frequency. See demodulateSignals() for more details.
    • rollingWindowTime: double, default is 60. This determines the length of rolling z-score window in seconds
    • LPFreq: double, default is 0. This determines the low pass frequency for photometry processing. If the value is 0, then no low pass will be performed
    • downsampleFs: double, default is 50. Determines the final sample frequency that each acquisition system will be downsampled to. This is a very important parameter.
    • dsMethod: string, default is 'resample'. If after downsample/demodulation, the final frequency still does NOT equal to downsampleFs, this determines which method is used to achieve (ie "force") the sample rate to be the same as downsampleFs. dsMethod='resample' will utilize the resample() function in matlab. See demodulateSignals() for more details.
    • movingAvergeFilter: logical, default is false. If true, perform moving average after downsampling.
    • movingAverageWindowSize: double, default is 2. This determines the moving average window size after downsampling.
    • removeTwoEnds: logical, default is false. See demodulateSignals() for more details.
    • plotPhotometry: logical, default is true. This determines whether the function plot sample photometry signals (to visualize raw and processed photometry data) or not.

  • Sync related params

    • syncPulseWindow: double, default is 5000. This determines the number of sync pulses used to calculate inter-pulse interval for later cross-correlation.

Outputs

  • The function does not have any direct outputs. Rather it save the following four files inside the session folder.
  • sync_sessionName.mat
    • This file is crucial and indispensable for single session analysis, as it contains synchronization information between multiple acquisition system involved.
    • params struct in sync_sessionName.mat will be the struct that stores ALL the key params related to synchronization, acquisition, and behavior type of the session. Following session analysis phase will heavily utilize and refer to this structure.
  • timeseries_sessionName.mat
  • data_sessionName.mat
  • behavior_sessionName.mat

Details

loadSessions() contains following steps

  1. Detects how many acquisition system exists within the session folder by searching for following files.
    • NIDAQ and Neuropixel: .ap.bin or .imec.bin
    • Labjack photometry: /Photometry folder
    • Camera: cam1_XXX.avi and times_cam1_XXX.csv
  2. Run signal preprocessing for each individual acquisition system
    • For NIDAQ:
      • digital: assigns each digital channel to corresponding events (packaged into function later)
      • analog: threshold and extracts rising edge if licking is recorded, or perform downsample (to options.downsampleFs) if photometry/movement traces are recorded
    • For labjack:
      • run concatLabjack_setupName.mat. Within it, assign digital/analog channels
      • For analog&recorded channels (stored in rows of labjack.raw), perform demodulation (demodulatePhotometry()) if corresponding labjack.mod(i) == true, or downsample (downsamplePhotometry()) otherwise. Save corresponding result in timeSeries.
      • The timeSeries struct will contain following fields and will be compatible with sabatini lab datajoint pipeline:
        • timeSeries.name: name of each signal, same as labjack.name
        • timeSeries.finalFs: final sample frequency (should be equal to options.downsampleFs)
        • timeSeries.system: acquisition system (can be 'NI','LJ', or 'Cam')
        • timeSeries.demux_freq:
        • timeSeries.behavior_offset:
        • timeSeries.system:
        • timeSeries.system:
    • For camera:
      • read times_cam1_XXX.csv file. Extract sync pulse and camera readings from Bonsai
      • If there is Bonsai-analyzed data like pupil/eye area, perform downsample (downsamplePhotometry()) and optional detrending based on user input
  3. Synchronization
    • Calculate inter-pulse interval of sync pulse recorded in each acquisition system
    • Perform cross-correlation to find the first common sync pulse
    • Assign each sample of each acquisition system with a common timestamp. This timestamp will be used to cross-reference between different acquisition system.
      • For example: water is delivered at NIDAQ sample 10003240, which corresponding to common time 10.1s. To find the photometry traces in Labjack aligned to this event, I can find the closest timestamp in labjack system (eg. 10.1003s) and plot the traces
    • Calculate time offset comparing to behavior (used in sabatini lab datajoint pipeline)
    • Save everything in params.sync fields and save params struct in sync_sessionName.mat

Examples

loadSessions(sessionList{idx},reloadAll=analysisParams(idx).reloadAll,...
            invertStim=true,...
            withPhotometryNI=false,photometryNI_mod=false,...
            recordLJ=[1,1,0],...
            rollingWindowTime=180)