old Server First Aid - ndexbio/ndex-rest GitHub Wiki

The purpose of this document is to provide general guidelines and a step by step procedure in case of a server failure.

STEP 1 - Use Putty to log on to the server in "SSH".

For Test server: test.ndexbio.org
For Production Server: ftp.ndexbio.org
For AWS instance: Make sure you have they KEY FILE (.pem) on your computer and Follow this procedure

STEP 2 - Examine log file

The log will provide useful info about the problems that caused the server failure. To view the log file, navigate to the logs directory and open the file called catalina.out

cd /opt/ndex/tomcat/logs tail -100 catalina.out | less


### STEP 3 - Verify Tomcat status
To check whether Tomcat is running or not, use the following command:
>````
ps -ef | grep tomcat

If Tomcat is not running, nothing should be displayed; however sometimes the database takes a long time to clean up and it might fail shutting down processes... in that case, a message could be returned even if Tomcat is not running (idle) and it should display the PID of the process that failed.

If Tomcat is running, a message similar to the following one will be displayed:

rudi 13680 13483 0 11:39 pts/0 00:00:00 grep tomcat ndex 32310 1 0 Oct30 ? 00:09:37 /usr/java/jdk1.7.0_67/bin/java -Djava.util.logging.config.file=/opt/ndex/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/opt/ndex/tomcat/endorsed -classpath /opt/ndex/tomcat/bin/bootstrap.jar:/opt/ndex/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/ndex/tomcat -Dcatalina.home=/opt/ndex/tomcat -Djava.io.tmpdir=/opt/ndex/tomcat/temp org.apache.catalina.startup.Bootstrap start


To distinguish between these scenarios, use the TOP command... if Tomcat is running, you should see a java process (PID 32310 in this example) at the very top that is owned by user "ndex" and uses a substantial % of memory
>````
 PID  USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 1207 root      20   0  184m 2144 1776 S  0.3  0.1  10:55.78 vmtoolsd
13798 rudi      20   0 15036 1248  948 R  0.3  0.0   0:00.05 top
32310 ndex      20   0 2466m 918m  11m S  0.3 24.0   9:38.88 java

STEP 4 - Restart Tomcat

Before restarting Tomcat, make sure to kill any processes that are still active (see STEP 3) Once the grep tomcat command returns an empty message, you can restart Tomcat with the following command:

sudo service tomcat7 start

If you are unable to restart Tomcat, proceed to STEP 5.

### STEP 5 - Call Jing!!!