Seti_Spec_Restricted - david-macmahon/wiki_convert_test GitHub Wiki

Error / Diagnostic Logs

SERENDIP-error Archives

Configuration Scripts

Current config scripts are here:

obs@beecourageous:/home/tfiliba/bin$ ls -al conf*sh
-rwxr-xr-x 1 tfiliba admin   87 Aug 26 03:55 conf_chip.sh
-rwxr-xr-x 1 tfiliba admin  776 Aug 25 14:47 conf_chip_ct.sh
-rwxr-xr-x 1 tfiliba admin  113 Aug 25 14:47 conf_chip_fft.sh
-rwxr-xr-x 1 tfiliba admin  116 Aug 25 14:47 conf_chip_pfb.sh
-rwxr-xr-x 1 tfiliba admin 1057 Aug 26 04:06 conf_chip_thr.sh

data storage and mysql: mattl-setup.ssl (login through casper.berkeley) l: root p: 1+usual

initial copy (oct 29) sent 20865186939 bytes received 3654 bytes 2307203.03 bytes/sec total size is 51036913030 speedup is 2.45

Beam switcher mapping

An analog beam switcher cycles through each of ALFA's 7-beam / 2-pol combinations and is controlled by the iBOB. Here is the mapping:

Here is the mapping from analog input to logical switcher value as defined by the code that programs the iBOB (beam_switcher.c). (I've added the parenthetical beam names based off my notes.)

#define IN00 0xEE
#define IN01 0xED (beam0a)
#define IN02 0xEB (beam0b)
#define IN03 0xE7 (beam1a)
#define IN04 0xDE (beam1b)
#define IN05 0xDD (beam2a)
#define IN06 0xDB (beam2b)
#define IN07 0xD7 (beam3a)
#define IN08 0xBE (beam3b)
#define IN09 0xBD (beam4a)
#define IN10 0xBB (beam4b)
#define IN11 0xB7 (beam5a)
#define IN12 0x7E (beam5b)
#define IN13 0x7D (beam6a)
#define IN14 0x7B (beam6b)
#define IN15 0x77

Note that we do not use IN00 or IN15.

In the data extraction code the beams are numbered 0-13 as defined in setiread.c.

    switch((int)beam_no)
    {
        case 238:
            beam_no = 0;
            break;
        case 237:
            beam_no = 0;
            break;
        case 235:
            beam_no = 1;
            break;
        case 231:
            beam_no = 2;
            break;
        case 222:
            beam_no = 3;
            break;
        case 221:
            beam_no = 4;
            break;
        case 219:
            beam_no = 5;
            break;
        case 215:
            beam_no = 6;
            break;
        case 190:
            beam_no = 7;
            break;
        case 189:
            beam_no = 8;
            break;
        case 187:
            beam_no = 9;
            break;
        case 183:
            beam_no = 10;
            break;
        case 126:
            beam_no = 11;
            break;
        case 125:
            beam_no = 12;
            break;
        case 123:
            beam_no = 13;
            break;
        case 119:
            beam_no = 14;
            break;
        default:
            beam_no = 255;
            break;
    }

    return beam_no;
}

Note: The first and last cases of the switch should never occur as no input is connected to them.

Observing Log

Feb 21, 2009

restarted ssh tunnel again.

Nov 7, 2009

Dead ssh tunnel... restarted with:

ssh -L 3133:lotf.ssl.berkeley.edu:22 [email protected] -N -f
  • edited Jan 2010 to reflect new hostname for lotf

Nov 3, 2009

Noticed too frequent file writing:

>>> See rsync output below:
>>> building file list ... done
>>> diskbuf/
>>> diskbuf/largefile_Nov022009_1013
>>> diskbuf/largefile_Nov032009_1538
>>> diskbuf/largefile_Nov032009_1549
>>> diskbuf/largefile_Nov032009_1600
>>> diskbuf/largefile_Nov032009_1608
>>> diskbuf/largefile_Nov032009_1617
>>> diskbuf/largefile_Nov032009_1625
>>> diskbuf/largefile_Nov032009_1634

Looks like levels might be set wrong?  Maybe engineering tests at AO?

counter: 500 numhits: 97059 beamnum: 3 filter: 0 headersize: 4096
size of spectra 97145
counter: 501 numhits: 97145 beamnum: 3 filter: 0 headersize: 4096
size of spectra 98290
counter: 502 numhits: 98290 beamnum: 3 filter: 0 headersize: 4096
size of spectra 98294
counter: 503 numhits: 98294 beamnum: 3 filter: 0 headersize: 4096
size of spectra 98304
counter: 504 numhits: 98304 beamnum: 3 filter: 0 headersize: 4096
size of spectra 98300
counter: 505 numhits: 98300 beamnum: 3 filter: 0 headersize: 4096
size of spectra 531
counter: 506 numhits: 531 beamnum: 3 filter: 0 headersize: 4096
size of spectra 455
counter: 507 numhits: 455 beamnum: 3 filter: 0 headersize: 4096
size of spectra 328
counter: 508 numhits: 328 beamnum: 3 filter: 0 headersize: 4096
size of spectra 979
counter: 509 numhits: 979 beamnum: 3 filter: 0 headersize: 4096

We also had a problem w/ receiving updated beam switcher values - we did a full restart to see if the problem went away.

Looks like everything came back up ok... We hadn't restarted the ibob in a while, maybe it got into a funky state. ALFA should be back in tomorrow afternoon, at which point we can take another look at the beam switcher problem.

If indeed we have a level issue, perhaps we can ask Luis to add/remove some pads?

Nov 30, 2009

Laura lowered thresholder scale in file bee2.config on beecourageous:/home/tfiliba/bin/bee2.config:

> thresholder_scale 80

This corresponds to 20 times the average power. Will go into affect the next time the beecourageous is rebooted.

Jan 2, 2010

hacked up and compiled scram_peek.C:

siemion@pattern:~/server_software/datarecorder2$ gcc -g -D_REENTRANT -I./EDTpcd -IAO/_home_phil_vw_pnt_exportCode -IAO/_home_phil_vw_h -IAO/_home_cima_Wapp_Bin_Sources_Includes scram_peek.C -lstdc++  error.o ao_utils.o -L. AO/lib/scram.o AO/lib/flip.o AO/lib/swapdata.o -LAO/lib -lazzatoradec -lm -I../siren/include -L../siren/lib -o scram_peek -lrt -I/usr/include/tk -I./setispec -IAO/_home_phil_vw_pnt_exportCode -IAO/_home_phil_vw_h -IAO/_home_cima_Wapp_Bin_Sources_Includes -I/usr/include/tk -I./setispec -IAO/_home_phil_vw_pnt_exportCode -IAO/_home_phil_vw_h -IAO/_home_cima_Wapp_Bin_Sources_Includes
scram_peek.C: In function ‘int main(int, char**)’:
scram_peek.C:67: warning: deprecated conversion from string constant to ‘char*’
siemion@pattern:~/server_software/datarecorder2$

To check if alfa is in:

siemion@pattern:~/server_software/datarecorder2$ ./scram_peek -i 1 1 2>&1 | grep useAlfa
Sat Jan  2 19:43:51 2010 IF2 from observer.naic.edu, useAlfa 1

Status Check Script

#!/usr/bin/php
<?
$ps_check_string = "setispec_dr";
$disk_to_check = "/dev/md1";
$ping_check_string = "1 received";
$recipient_list = "[email protected], [email protected], [email protected], [email protected], [email protected]";

$mail_message = "SERENDIP V.5 CRITICAL ERROR REPORT\n\n";

echo "pinging bee\n";
$output = shell_exec("/bin/ping -c 1 beecourageous");

if( stristr($output, $ping_check_string) ) {
        $mail_message .= "The BEE2 is responding to pings:\n".$output;
} else {
        echo "bee is dead\n";
        $mail_message .= "The BEE2 is **NON-RESPONSIVE**:\n".$output;
        $mail_message .= "Attempting to restart the BEE... \n";
        $output = shell_exec("/usr/bin/perl /home/siemion/UU.pl 192.168.100.65 admin:ibmthinkpad 5off");
        $mail_message .= "\n Power off command output: ".$output."\n";
    sleep(5);
        $output = shell_exec("/usr/bin/perl /home/siemion/UU.pl 192.168.100.65 admin:ibmthinkpad 5on");
        $mail_message .= "\n Power on command output: ".$output."\n";
    sleep(2);
    $output = shell_exec("/usr/bin/perl /home/siemion/UU.pl 192.168.100.65 admin:ibmthinkpad status");
        $mail_message .= "\n Current Status: ".$output."\n";
        sleep(60);
        $mail_message .= "Sleeping 60 seconds to wait for BEE2 to come back up...\n";
        $mail_message .= "\n";

        $output = shell_exec("/bin/ping -c 1 beecourageous");
                if( stristr($output, $ping_check_string) ) {
                        $mail_message .= "!!Success!! The BEE2 lives!\n";
                        $mail_message .= "Sleeping 5 minutes to let the BEE2 boot...\n";
                        sleep(300);
                        $mail_message .= "Attempting to restart the whole shebang...\n";
                        $output = shell_exec("sudo /home/siemion/dorun2.sh > /tmp/blah  2>&1 &");
                        do {
                                sleep(2);
                                $output = shell_exec("cat /tmp/blah");
                        } while(!stristr($output, "imdonenow"));

                        $mail_message .= "Check output below: \n".$output;

                } else {
                        $mail_message .= "Ugh... Looks like the BEE2 is still dead.  Better take a look at this.\n";
                }
        $send_message = 1;
}


$output = shell_exec("/bin/ps aux");

if( stristr($output, $ps_check_string) ) {
    $mail_message .= "The data collection process is running as of ".date ("l dS F Y h:i:s A")."\n";
} else {
    $mail_message .= "\n\n*****The data collection process is **NOT RUNNING** as of ".date ("l dS F Y h:i:s A")."\n";
    $output = shell_exec("/usr/bin/perl /home/siemion/UU.pl 192.168.100.65 admin:ibmthinkpad status");
    $mail_message .= "\n Current Status: ".$output."\n";
            $mail_message .= "Attempting to restart the whole shebang...\n";
            $output = shell_exec("sudo rm -rf /tmp/blah");
            $output = shell_exec("sudo /home/siemion/dorun2.sh > /tmp/blah  2>&1 &");
                        do {
            sleep(2);
            $output = shell_exec("cat /tmp/blah");
            } while(!stristr($output, "imdonenow"));
            $mail_message .= "Check output below: \n".$output;
    $send_message = 1;
}

//echo $mail_message;

$output = shell_exec("/bin/df -h");
$mail_message .= "\n\nDisk Usage: \n".$output."\n";

$output = shell_exec("/bin/df -h | grep ".$disk_to_check);

$tokens = explode(" ", $output);
for($i = 0;$i < count($tokens); $i++){
   if(stristr($tokens[$i], "%")){
       $moretokens = explode("%", $tokens[$i]);
       if($moretokens[0] > 95) {
           $mail_message .= "*** WARNING *** WARNING *** DISK USAGE CRITICAL ON DISK: ".$disk_to_check." - ".$tokens[$i]."\n\n";
           $send_message = 1;
       }
   }
}


if($send_message == 1){
mail($recipient_list, "SERENDIP V.5 Critical Error", $mail_message, "From: [email protected]\r\n");
}
?>