Recently, I ran into some computational limits on my laptop while doing training some CNNs. I have a desktop, but given a tough enough problem, even it won’t suffice. Thus, the cloud is eventually going to be the only available option. Unfortunately, the cloud is so far away that I still can’t find it despite everyone talking about it, so how would we ever find the notebook?… Seriously though, most cloud operating systems don’t come installed with a desktop environment (think Ubuntu server). But let’s say you go ahead and install a desktop environment or start off with Ubuntu desktop. You’ll have to use X11 to send all your browser information through ssh, which can be slow; additionally, you’ll need to configure Xorg on the server. I’ll show you an easier solution that uses ssh tunneling and only requires ssh and jupyter notebook to be installed on your cloud instance. This post uses an EC2 instance from Amazon running Ubuntu server, but the method should work for any remote linux machine with ssh installed.
0. Launch a cloud instance and install the appropriate software
Launch an AWS EC2 instance or find some remote linux machine if you don’t already have one. I already launched an AWS EC2 instance and generated my key named “EC2Test.pem” which I left at ~/Desktop. So anytime you see ~/Desktop/EC2Test.pem just replaces it with /.pem. Additionally, I installed anaconda which comes with jupyter notebook, so make sure you’ve got the notebook in the cloud. Throughout this post make sure to use whatever ssh convention is necessary to properly ssh into your machine. I was running Ubuntu at ec2-35-161-33-142.us-west-2.compute.amazonaws.com , so if you’re also running Ubuntu in AWS just replace the address . Finally, make sure to add your IP address to your security group to save you a headache…
1. Open jupyter properly
ssh into your machine by opening a new terminal and entering
ssh -i ~/Desktop/EC2Test.pem ubuntu@ec2-35-161-33-142.us-west-2.compute.amazonaws.com
In that terminal open Jupyter Notebook with
jupyter notebook --no-browser --port 8886
Things to note
- –no-browser command tells jupyter not to open a browser; this isn’t neccessary, but jupyter prints some junk in the terminal without it
- –port 8886 tells jupyter to run on port 8886; feel free to swap that port for any available port on your cloud instance
2. Dig the tunnel
On your local machine, open a new terminal and enter
ssh -N -f -L localhost:8888:localhost:8886 -i ~/Desktop/EC2Test.pem ubuntu@ec2-35-161-33-142.us-west-2.compute.amazonaws.com
Things to note
- -N tells ssh that a remote command won’t be sent; it doesn’t attempt to log you into the server
- -f allows the tunnel to be forked into the background so you can close the terminal. If you use this option, you’ll have to kill the tunnel manually by finding the process. I usually leave out this option
- -L tells ssh that you are going to pass in the local and remote addresses
- localhost:8888 is the address on your local machine that you are forwarding to
- localhost:8886 is the address on your cloud instance that jupyter is forwarding to which is the same address that you specified when opening jupyter
3. Use your notebook
On your local machine, open a browser and go to localhost:8888. Viola! You should see your Jupyter Notebook.
Hack Till Dawn 
This is really great, helped me alot 🙂 Thanks!
LikeLike
Any reason why calls to %qtconsole don’t work when I use this method? Thanks
LikeLike