Getting started - til-ai/til-25 GitHub Wiki

Welcome, Guardian! This is your first-time setup guide to get you up and running.

Contents

• Prerequisites
• Get to know your Vertex AI Workbench
  • Open your Workbench instance
  • Your JupyterLab interface
  • Creating your first files
  • Shutting down your instance
• Clone the template repo
  • Generating a GitHub personal access token
  • Cloning your repo
• Set up your development environment
  • About Python virtual environments
• For power users
  • Using gh to persist Git credentials
  • Developing locally
  • Downloading the training data locally
• Further reading

You should already have access to your team's Vertex AI Workbench and your team's private channel in the TIL-AI Discord server.

You should know and memorize your Team ID. Your Team ID is a unique string that's used for identifying your images, buckets, workbench, results, and other resources. Your Team ID is the same as the name of your team's private Discord channel. This is different to your Team Name, which is the name your team used to sign up.

We strongly recommend that you have a GitHub account, which will allow you to create your own copy of this repository.

Vertex AI is an AI development platform provided by Google Cloud. One of its features is the Vertex AI Workbench, which gives you access to a powerful cloud computer — complete with a file system, terminal, and useful tools like Python, Jupyter Notebooks, and Git — where you can run code and train machine learning models.

Vertex AI is the primary platform on which you'll develop, train, and test your models. We'll run you through how to access and use your Workbench in this section, and then show you how to start developing and training.

If you're already familiar with Vertex AI, you can skip ahead.

Make sure you're logged in to the correct Google account. This is the account using which you completed the Onboarding Form.

Open the Vertex AI Workbench for TIL-AI 2025. You'll see a list of instances, each labelled instance-TEAM_ID. To find your instance, filter by the instance name. You'll need to know your Team ID.

Next, you need to start your instance. Your instance is a computer that lives in the cloud; just like a normal computer, you need to boot it up. Select the checkbox for your instance, then click Start. This might take a few minutes.

The icon to the left of your instance name is its status. A gray circle with a square inside means your instance is stopped (shut down). A loading indicator means it's currently starting up. A green checkmark means it's started and running.

Once your instance has started, click Open JupyterLab to access your team's Vertex AI Workbench.

You'll now see something like this. This interface is called JupyterLab.

Let's run through the main components.

1. This is the toolbar. You can select a tab in the toolbar to open them. The tools are, in order: File Browser, BigQuery, Running Terminals and Kernels, Git, Table of Contents, and Notebook Executor.
   1. These icons belong to the File Browser. They provide tools to manipulate the file system. You'll mostly use the first four: Create Launcher, New Folder, Upload Files, and Refresh.
   2. This is the file list. Use it to open files and folders, and navigate your directory.
2. This is the Main Work Area. When you open new files and terminals, they appear here.
   • It currently shows the Launcher. You'll use this to create new notebooks, terminals, Python files, and other files. You can always access the Launcher by clicking the Create Launcher button in 1ii.

Besides the File Browser, there are other useful tools in the Left Sidebar. These are the most useful:

• Running Terminals and Kernels shows the terminals and Jupyter notebooks that are currently running.
• Git is a version control tool, which you can use to track your code's history. JupyterLab provides a GUI for Git.

Take a look at the JupyterLab interface docs for a deeper dive. Explore all the menus and dropdowns to get yourself familiar with it.

If the Launcher isn't already open, click the File Browser in the Left Sidebar, then the Create Launcher button (the blue + sign).

Let's start with something familiar. To create a Jupyter Notebook, click the Python 3 (ipykernel) button under Notebook.

If you're familiar with Jupyter Notebooks, you'll be right at home. You can write Python and documentation, and run your code right from your notebook. For the uninitiated, Jupyter Notebooks let you run Python code in small chunks (called "cells"), and handle some of the tedious work of manually running pure .py files, which makes it convenient to test with. Learn more about Jupyter Notebooks.

Next, let's create our first terminal. This is similar to Command Prompt on Windows and Terminal on macOS; it lets you run commands through a command line. In the Launcher, click the Terminal button, under Other.

The terminal uses Bash, the standard shell scripting language. You don't need in-depth knowledge of it, but you'll need how to run simple commands using the terminal. In this Wiki (and many other docs), if you see an instruction that says "Run this:", and then some code that looks like app command --options, it means to run it in the Terminal. Learn more about shell scripting.

You can also run Bash commands inside Jupyter Notebooks. Just create a new cell, and type ! before your Bash command. That line will be run as a shell script, not Python.

Finally, let's take a look at how to create and run Python .py files. Head back into the launcher, and create a Python File and a Terminal (both of which are under Other). Then, save your Python file, and run it from the terminal using the command python your_filename.py

After you open many terminals, consoles, and files, they can get a bit tricky to manage. If a terminal is unresponsive, you'll want to be able to force it to close. To do this, open the Left Sidebar, and click the Running Terminals and Kernels tab. You can then see a list of open tabs, kernels, and terminals, which you can shut down individual or all at once. Think of it like Task Manager on Windows or Force Quit on macOS.

There are many other files you can create through launcher. You can create any text-based file (such as .json, Dockerfile, etc.) by creating a standard .txt file through the Launcher, and renaming the file with the correct extension. You can also upload any file you like from your computer.

To reduce compute costs and minimize the risk of losing data in your instance, you should shut it down when no one in your team is using it. This is the equivalent of turning off your laptop.

First, make sure no one else in your team is using your team's instance, and that no one has left it running to train models. Because your entire team shares an instance, when you shut it down, it's shut down for everyone in your team.

Then, head back to the Vertex AI Workbench instance list, find your instance, select the checkbox next to it, and press Stop.

Overview: You'll need to copy the files in the til-ai/til-25 repository to your Vertex AI Workbench so you can work on your project. You'll use our template to create your own private copy, and clone it into your environment.

Only one member of your team needs to do this.

Start by creating your own copy of this repo from the template. Head to the homepage of this repo and click: "Use this template" → "Create a new repository". Follow the instructions to create a repo in your own GitHub account.

To clone your private repo into your workbench, you'll need to authenticate to the Git client in your Vertex AI Workbench using a GitHub Personal Access Token (PAT).

These steps need to be done by the person who owns your repo.

1. Go to Personal access token settings for your GitHub account.
   • You can also access this by heading to github.com, logging in, clicking your profile picture in the top right corner, then Settings → Developer settings (in the left sidebar) → Personal access tokens → Fine-grained tokens.
2. Click Generate new token. You might need to sign in again. Follow the instructions to create a new token.
   • You can fill in any value for Token name.
   • Make sure your personal GitHub account is selected under Resource owner.
   • Choose an Expiration. After this date, your token will become invalid, so make sure it'll work for long enough to last you through the competition.
   • Under Repository access, select Only select repositories. Then, in the dropdown that appears, choose your copy of the til-25 repo.
   • Under Permissions, expand the Repository permissions section. Find Contents, then select Access: Read and write. You don't need to change anything else.
     • With these settings, your token can only read and write the source code in your repo. You can change these permissions in the future, in case you'd like it to have other permissions.
   • Click Generate token, and confirm your selections.
3. On the page that follows, copy the newly generated token and save it somewhere securely. It looks like github_pat_ followed by a long string of characters.
   • You'll only be shown your token once (immediately after you generate it). After this, GitHub will never show it to you again, so make sure you save it somewhere safe and secure.

Once you have your token, you can use it to authenticate the Git client in your Vertex AI Workbench. Every time you run a command that involves authenticating to your remote on GitHub, such as cloning, pushing, and pulling, you'll be asked to provide your username and password. Your username is your GitHub username, and your password is the PAT you generated in Step 3. You should not authenticate using your account's actual password, as this will give it access to everything in your account.

Read more about GitHub Personal Access Tokens.

Now that you're able to authenticate your Git client, we can clone your copy of the template repo and set it up. Open a terminal in your Vertex AI Workbench, and run this.

cd /home/jupyter/
git clone https://github.com/your-github-username/your-repo-name.git

You should now see the files in your repo appearing in a folder called your-repo-name/. Next, we need to initialize the submodules in your repo, which are links to other repos by til-ai: til-ai/til-25-finals and til-ai/til-25-environment.

git submodule update --init

Git is a useful tool for tracking the history of your code and protecting against accidents. You can read some of our suggestions for using Git in Pro tips.

Overview: Now that you've cloned your repo, you can finish setting up the dependencies your code requires.

conda create --name env python==3.12

conda activate env

In your Vertex AI Workbench, create a new terminal, and run this to create a new virtual environment, activate it, and install all the development dependencies in your project.

# Creates a virtual environment called "env"
conda create --name env python=3.12

# Activates the "env" environment. You might need to do this for every new terminal.
conda activate env

# Installs all the dependencies in requirements-dev.txt.
pip install -r requirements-dev.txt

One of the dependencies that's installed by pip from requirements-dev.text is the til_environment Python package, which you'll need to train and test your RL model. You can import this package in any Python code running in your virtual environment (import til_environment). We go over this package in RL with til_environment.

And now you're all set up, so get coding! Take a peek at the sidebar (on the right) to see what other basics you need to know.

This section isn't necessary to get your environment set up. But, we know there are lots of power users amongst our participants who'd like to use their own workflows for development. Since this section is for them, we'll assume a more experienced audience here.

If you're annoyed by having to repeatedly authenticate Git commands, you can install the GitHub CLI, gh using apt-get. Then, you can gh auth login once with your PAT, and your credentials will be persisted. If you do this, remember that your Vertex AI Workbench is shared between all your teammates, so when you log in, they'll be logged in with your PAT too. This also means only one member of your team can be logged in at a time. You can log out using gh auth logout.

If you'd like to stick to your favorite local dev tools like Visual Studio Code, we suggest that you clone your Git repo to your local machine, then push your code to your remote and pull it back into your Vertex AI Workbench. This is by far the simplest and least technical method.

Even if you prefer to develop locally, we recommend against submitting your models for evaluation from your local machine. Run the command from either the Vertex AI web interface or via SSH. Your Vertex AI Workbench is already set up with the tools and permissions to run submission seamlessly. If you really feel the need to do so, we've written a brief guide.

We'd honestly rather you not run up our data egress bill on GCP, but if you really insist, one of your fellow participants has figured out how to do it.

1. From your Vertex AI Workbench Instance, run gcloud auth print-access-token. See the GCP docs for details.
2. Write this access token to a file on your local machine.
3. Use this access token to authenticate your subsequent requests e.g. gcloud storage cp -r gs://til-ai-25-novice --access-token-file your/access/token/file .
4. Note that the access token expires after 30 mins to an hour, and the download takes longer than that, so you may have to refresh the token a couple times and repeat the process.

• Introduction to Jupyter Notebooks: https://realpython.com/jupyter-notebook-introduction/
• JupyterLab interface docs: https://jupyterlab.readthedocs.io/en/latest/user/interface.html
• Writing Bash scripts: https://www.freecodecamp.org/news/shell-scripting-crash-course-how-to-write-bash-scripts-in-linux/
• About Git: https://docs.github.com/en/get-started/using-git/about-git
• About GitHub repositories: https://docs.github.com/en/repositories/creating-and-managing-repositories/about-repositories
• About GitHub personal access tokens: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens
• About Git submodules: https://www.atlassian.com/git/tutorials/git-submodule
• About Python virtual environments: https://realpython.com/python-virtual-environments-a-primer/