It is possible to connect to a running Python session via IPython by setting up ssh tunnels and initializing connections between local and remote clients. This post assumes:

  1. The ‘local’ client is a Windows host with the Anaconda platform installed. The local client will be referred to as localhost for the remainder of the post.

  2. The ‘remote’ host is a Linux server, also with the Anaconda platform installed and is accessed via ssh. The remote server will be referred to as remotehost for the remainder of the post.

Locate Runtime Directories

We need to obtain the location of the Jupyter runtime directory for both localhost and remotehost. This is the location to which the IPython session kernels are saved. We can locate the runtime directory using the following command:

localhost> jupyter --runtime-dir
C:\Users\username\AppData\Roaming\jupyter\runtime

Similarily, for the remote host:

remotehost> jupyter --runtime-dir
/home/username/.local/share/jupyter/runtime

So on localhost, our IPython runtime directory is C:\Users\username\AppData\Roaming\jupyter\runtime and for remotehost is /home/username/.local/share/jupyter/runtime.

Initializing IPython Kernel


Now that we’ve obtained the locations of localhost and remotehost’s runtime directories, we can initialize the IPython kernel on remotehost. Once initialized, the command will return the name of the kernel file specific to this latest session, and we will copy this .json file from remotehost’s runtime directory to localhost’s runtime directory:

remotehost> ipython kernel
To connect another client to this kernel, use:
    --existing kernel-32627.json

Since Windows doesn’t include an ssh server by default, we need to manual copy the kernel file from from remotehost to localhost. WinSCP makes it easy to view local and remote file systems side-by-side, and facilitates the transfer of file system objects across hosts. With winSCP, we can navigate to each host’s respective runtime directory, and copy the kernel-32627.json file from /home/username/.local/share/jupyter/runtime to C:\Users\username\AppData\Roaming\jupyter\runtime.

Tunneling

After copy the file from remotehost to localhost, we need to open it and execute ssh command that reference the ports specified in the json. The file used for this example, kernel-32627.json, contains the following:

{
  "iopub_port": 39736,
  "control_port": 59725,
  "transport": "tcp",
  "shell_port": 51963,
  "key": "1fcd997c-ef64-4322-8762-c034af6095e1",
  "stdin_port": 59714,
  "signature_scheme": "hmac-sha256",
  "hb_port": 41128,
  "ip": "127.0.0.1"
}



The ssh commands, which will be executed on localhost, will be of the form:

localhost> ssh -p <port_nbr> username@remotehost.com -f -N -L <port>:127.0.0.1:<port>


The -p flag is optional, and is used for instances in which the remote sever runs ssh from a non-standard port. If ssh on your remote server is bound to the standard port 22, you may ignore the -p option. Then the command would reduce to:

localhost> ssh username@remotehost.com -f -N -L <port>:127.0.0.1:<port>


We need to execute this command for each port specified in kernel-32627.json: iopub_port, control_port, shell_port, stdin_port and hb_port.

A brief description of the -f, -N and -L flags:

  • -f - Requests ssh to go to background just before command execution.

  • -N - Do not execute a remote command (useful for only forwarding ports).

  • -L - Forward the given port on localhost to the given port on remotehost.

We execute the command five times, with only the port number changing between calls. Using kernel-32627.json as our reference, we enter and execute:

localhost> ssh -p 2245 username@remotehost.com -f -N -L 39736:127.0.0.1:39736

localhost> ssh -p 2245 username@remotehost.com -f -N -L 59725:127.0.0.1:59725

localhost> ssh -p 2245 username@remotehost.com -f -N -L 51963:127.0.0.1:51963

localhost> ssh -p 2245 username@remotehost.com -f -N -L 59714:127.0.0.1:59714

localhost> ssh -p 2245 username@remotehost.com -f -N -L 41128:127.0.0.1:41128



In this case, no news is good news. If, after typing each command and hitting enter nothing is returned except the prompt, the command was executed successfully.

As above, if ssh is bound to the standard port 22, the commands reduce to:

localhost> ssh username@remotehost.com -f -N -L 39736:127.0.0.1:39736

localhost> ssh username@remotehost.com -f -N -L 59725:127.0.0.1:59725

localhost> ssh username@remotehost.com -f -N -L 51963:127.0.0.1:51963

localhost> ssh username@remotehost.com -f -N -L 59714:127.0.0.1:59714

localhost> ssh username@remotehost.com -f -N -L 41128:127.0.0.1:41128



All that remains is to open Qt Console, but with an extra argument that references kernel-32627.json. This way, IPython will connect to the already-running session on remotehost instead of creating a new session on localhost.


From localhost, open a new command prompt (Windows key + R, type ‘cmd’ (no quotes)), then enter:

localhost> jupyter qtconsole --existing C:/Users/username/AppData/Roaming/jupyter/runtime/kernel-32627.json

Be sure to use the forward slash as your path delimiter. The Qt Console that renders will look like your Qt Console from localhost, but you can easily verify that it’s running the remote kernel as follows:

>>> import sys
>>> sys.executable
remotehost.com


If connecting to remote kernels through Qt Console is something you’ll be doing with regular frequency, it makes sense to encapsulate the above commands in a script that parses the kernel file and automatically binds the necessary ports via ssh. This is a fairly straightforward task, and example of which is available in this GitHub gist. Until next time, happy coding!

References:

  • https://github.com/ipython/ipython/wiki/Cookbook:-Connecting-to-a-remote-kernel-via-ssh