The perfect Google Colab setup for DL research
People are not usually aware of truly how much you can do with this VM and are stuck with only using python notebooks in colab. It's important to have a swift workflow in experimentation so that one can focus more on the research part.
Note: You can skip directly to this colab notebook I made instead of reading this article if you are already familiar with bash, ssh and extensions in VSCode. Else you can read the rest.
- Check GPU, CPU and memory details. Make sure you enable GPU in notebook settings.
2. Make your VM feel like your local machine. Add git global config, add your bash config preferences and install all the dependencies you normally use a lot. (like zsh and wormhole for example)
3. I highly recommend installing oh-my-zsh as it makes your terminal navigating life much easier.
4. https://ngrok.com/ is a free service that enables you to serve local services on the VM (such as ssh) and URLs on a public domain. This makes it possible to SSH remotely into the VM through the ngrok tunnel i.e localmachine→ngrokhost →VM.
In order to have uninterrupted ngrok service, sign up for free on their website using the same google account you are using. Login to https://dashboard.ngrok.com/get-started/setup and copy your ngrok key to the clipboard as seen above.
5. Paste your key and run the cell. In a few seconds, you will get the details of the SSH configuration i.e. the URL and port.
6. Copy this configuration onto your clipboard. You can also directly SSH into the VM as root using your local terminal. But it is best to use VSCode’s SSH extension to code. FIrst download the following extension:
7. Hit“Ctrl+Shift+P” on Windows/Linux or “Cmd+Shift+P” on Mac and type “Open SSH configuration file”. Paste in the config you just got into the file. You will now be able to SSH and code using VSCode.
8. Hit “Ctrl+Shift+P” on Windows/Linux or “Cmd+Shift+P” on Mac and type “Connect to host”. Choose gcp_colab_ssh and enter the password i.e. Password@123 in this case. You can now open a terminal in the VSCode window and browse through the file system using the terminal.
9. Run the last cell to ensure that the VM does not time out due to inactivity!
10. Additional tips:
- I recommend using python-magic-wormhole to send and receive files to and from your local machine. Or you can mount your Gdrive to the VM.
- I also recommend updating to Colab Pro. You get better GPUs and 24 hours of continuous runtime as well as a lesser chance of random timeouts.
- Always push your changes to GitHub to avoid the frustration of losing work due to timeouts. Save your model checkpoints to local machine or Gdrive as well.
Hope this helps! :)
