Connecting to the linux server from Windows Spyder - jamiefogel/Networks GitHub Wiki

Connecting Spyder on Windows to a Jupyter Kernel on Linux

This guide details how to connect Spyder running on a Windows machine to a Jupyter kernel on a remote Linux server using SSH tunneling and manual configuration.

Prerequisites

  • PuTTY installed on Windows (for SSH tunneling).
  • Spyder installed on Windows.
  • Jupyter Notebook installed on the Linux server.
  • Access to a shared directory (optional) if you need to access kernel connection files.

Step 1: Set Up the SSH Tunnel with PuTTY

  1. Open PuTTY and configure the connection to your Linux server:

    • In the Session section, enter the hostname or IP address of your Linux server (e.g., 10.1.1.226) and the port (usually 22 for SSH).

  2. Set Up Port Forwarding:

    • In PuTTY, go to Connection > SSH > Tunnels.
    • Under Source port, enter 8888. This is the local port on your Windows machine that will be used to access the Jupyter Notebook server.
    • Under Destination, enter localhost:8888 (or 127.0.0.1:8888 if localhost doesn't work). This should match the port and IP address used by Jupyter on the Linux server.
    • Click Add to add this port forwarding rule.

  3. Open the SSH Connection:

    • Go back to Session and click Open to initiate the SSH connection.
    • Log in with your username and password for the Linux server.
    • Leave this PuTTY window open for the duration of your Spyder session, as it keeps the SSH tunnel active.

Step 2: Start the Jupyter Notebook Server on the Linux Server

  1. Activate the Conda Environment (if required):

    • Run the following command to activate the labor_gt environment (or whichever environment you are using): 
      conda activate labor_gt

  2. Start the Jupyter Notebook Server:

    • Run the following command to start the Jupyter Notebook server on port 8888: 
      jupyter notebook --no-browser --port=8888 --ip=127.0.0.1
    • This binds Jupyter to 127.0.0.1, meaning it will be accessible only on the server itself. However, because we set up an SSH tunnel, you’ll be able to access it through the local port 8888 on your Windows machine.
    • Make a note of the URL and token that Jupyter prints in the terminal. You may need this token later.

  3. Create a New Kernel by Starting a Notebook or Console:

    • Open a browser on Windows and go to http://127.0.0.1:8888 to access the Jupyter Notebook server (use the token if prompted).
    • Start a new Python 3 notebook or console. This will create a new kernel file in the ~/.local/share/jupyter/runtime/ directory on the Linux server.

  4. Move the Kernel Connection File to /NetworksGit/jupyter_kernels :

    • First launch new PuTTY connection to 10.1.1.226. This is currently necessary because the previously-opened terminal will not return control to the keyboard (maybe I can figure out how to fix this)
    • In the PuTTY terminal on the Linux server, run: 
      mv "$(ls -t ~/.local/share/jupyter/runtime/kernel* | head -n 1)" /home/DLIPEA/p13861161/labormkt/labormkt_rafaelpereira/NetworksGit/jupyter_kernels/kernel.json
    • Look for a file named kernel.json. This file contains the necessary connection information for Spyder.
    • I renamed the file to kernel.json so that it will always have the same name. This allows me to save the kernel filename in the connection file in Spyder.

  5. Copy the Kernel Connection File to a location accessible by Windows:

Step 3: Configure Spyder to Use the Remote Kernel

  1. Open Spyder on Windows.

  2. Open the Kernel Connection File:

    • In Spyder, go to Console > Connect to an existing kernel (or click on the three horizontal lines in the Console pane).
    • Click Browse and navigate to the location of the kernel-xxxx.json file. Select this file. (If needed, copy it to a shared folder that’s accessible from both Linux and Windows.)

  3. Enter SSH Details:

    • In the Connect to an existing kernel dialog in Spyder, check the box This is a remote kernel (via SSH).
    • Set the following:
      • Hostname: Enter the IP address of your Linux server (e.g., 10.1.1.226).
      • Port: 22 (the SSH port).
      • Username: Enter your username for the Linux server.
      • Password: Enter your password for the Linux server.

  4. Click OK to Connect:

    • Spyder should now connect to the remote kernel using the SSH tunnel.

Troubleshooting Tips

  • If Spyder Hangs on “Connecting to Kernel”:

    • Make sure the PuTTY SSH session is open and active.
    • Verify that the Jupyter Notebook server is running on the Linux server.
    • Double-check that the correct kernel-xxxx.json file is being used and points to the right IP (127.0.0.1) and ports.

  • Testing the SSH Tunnel:

    • Open a browser on your Windows machine and go to http://127.0.0.1:8888. You should see the Jupyter Notebook interface if the tunnel is set up correctly.

This document should help you establish a connection between Spyder on Windows and a Jupyter kernel on a Linux server manually. Follow these steps each time you want to connect without relying on automated scripts.