15 Feb 2024

• 10:00am - 10:30am Web Graphing Today
• 10:30am - 11:00am Git Large File Support
• 11:00am - 11:30am Catchup on Your Front-End Homeworks, interviews next week
• 11:30am - 12:00noon Commit and save any dev work on AWS servers for this afternoon

Continuing on our co-creation work and the Edward Tufte reading, we'll be studying today how to provide data from our API to produce visual graphs.

We'll be using the Plot Javascript library from Observable, which is based on d3.js that you can also use directly.

https://observablehq.com/plot/marks/bar

Go to the class monorepo and hit . You may see a "space usage" warning. Don't worry, we'll cover that after our graphing introduction.

Change to your team's final project directory. If you don't have one yet, work with a member of your team who is taking the Web Infra track to create one, and copy over their latest Express API server here from assignments/<GITHUB_USERNAME>/infra.

Run the tree -I node_modules command to orient yourself to the file layout.

Add the lines below. Feel free to experiment with some of the numbers and options when you've finish the final step below.

That serves a list of data to a static page in your final project directory.

Create a new page for this, for example, called graph.html, in your static assets directory (called pages in our homework, check your server.js for the keyword static). It doesn't have to be connected to any other page, your landing page, or index.html.

To see your graph, run node server.js. You won't need an SSH tunnel or a database for this exercise today.

When you see a popup asking you about a public browser port 5000, go ahead and click "Open Browser" to open a new tab to view your server.

You should see a graph similar to the below

If you don't, don't worry. Open a web inspector and see if there are any errors.

You can also view the API call directly to see the JSON data.

You can do much more with Plot and d3 and next week our assignment will be to watch videos and apply our learning from the FrontendMasters d3 course.

The monorepo is a shared resource. At many organizations and companies, you are contributing to a codebase that is stored on every computer, including those of your peers, colleagues, classmates.

While we are still deciding as a community the licensing of code contributed to this repo, only use what is necessary for our purpose of learning together.

We may need to commit large files into Git for our final project, and we are already nearing our limit of a few hundred MBs.

Install ncdu on your AWS servers or laptops. (It's already in GitHub codespaces).

sudo apt install -y ncdu
cd <repo_dir>
ncdu .

Poke around and see

• how much space the mono repo is taking up in your local working directory
• which directories are taking up the most space. You can drill down to individual files

Try to identify a filename extension that is common to many of them, such as *.txt.

These are good candidates to move into LFS.

We'll install Git LFS on our laptops to offload these big files to a separate Git store.

https://git-lfs.com

git lfs install

Change to the root of the repo. You can track individual filename extensions with a command like this

git lfs track *.xxx

This creates a .gitattributes file in the current directory. Try finding this file and inspecting its contents with cat, less, nano or another method of your choice.

This will set up future files of type .xxx of being stored in Git LFS. However, existing files need to be migrated.

Run the following command to see which files would be migrated.

git lfs migrate info

Copy and paste the output into the Discord channel #sos-w24-frontend to get feedback from your classmates on which file extensions to migrate.

When the class has reached consensus, one of the TAs will perform the migration to LFS.

git lfs migrate import

Then they will force-push to GitHub.

git push -f

Inspect again with ncdu to see the change in the size of directories in the monorepo.

You may also like to use the find command

find . -name "*.xxx"

This is the easiest way to develop on any computer. On the monorepo GitHub website (not the wiki), hit . and follow our setup.

Windows Subsystem for Linux WSL is recommended. It is a virtualized Linux kernel implemented on hardware.

In a powershell run as administrator

wsl --install

Wait as it downloads and installs WSL and an Ubuntu image. Afterwards, you can run WSL like a special terminal from your Start Menu.

In VSCode, you can use this official extension to edit code inside WSL.

CS Faculty Candidate - Jessica Avery - Teaching Demo postponed until 29th

• 1:00pm - 1:30pm Infra Scenario: Ideate, Plan, Discuss
• 1:30pm - 2:00pm Back up AWS server work and keypairs
• 2:00pm - 2:30pm Turn off servers, register new subdomains, add back SSH access, shared tmux session
• 2:30pm - 3:00pm New nginx server blocks and TLS certs for second subdomain

Imagine that you are managing the infrastructure of a company with about 26 servers, divided between two secret data centers somewhere in the Eastern WA desert. You can drive there to physically wrangle machines, but it is a 3 hour drive and as an effective (and lazy) engineer, you want to do as much as possible remotely.

Each server runs a separate website, a subdomain of the company website (https://arcology.builders) Let's say each subdomain is a letter of the English alphabet.

Data Center H runs these subdomains:

https://a.arcology.builders
https://c.arcology.builders
https://e.arcology.builders
...
https://y.arcology.builders

Data Center T runs these subdomains:

https://b.arcology.builders
https://d.arcology.builders
https://f.arcology.builders
...
https://z.arcology.builders

Your goal in the following activity with a partner, is to make sure both of your AWS servers are equally replaceable (able to be turned off, and have your website served by another AWS server).

scp -i <PATH/TO/PEM> ubuntu@<YOUR_SUBDOMAIN>.arcology.builders:~/.ssh/id* ~/.ssh/

For each final project team, one partner chooses "heads" and the other chooses "tails".

One of the TAs flips a coin, and it lands heads (H) of tails (T).

Overnight, that Data Center stops working for mysterious reasons. You cannot log into them; they are dead to the outside world.

Your team's only consolation is:

• you have a backup of the 26 SSH keypairs (located in the files named ~/.ssh/id_* for the single ubuntu user)
  • which you share with each of the 26 customers, in case the customer loses theirs
• you've warned your customers not to develop remotely on their servers, so each machine individually and its data is totally disposable in case of a scenario like this

Debugging it will take time. It could be a cyber attack, hardware failure, corporate espionage, data center infestation, etc.

In the meantime, you as the infrastructure team need to make sure all 26 websites are still running, using only your AWS console and trusty terminal.

You've decided to have the b website share the server for a, the c website share with d, and so on, so that you can now run all 26 websites now with 13 servers.

(That is, for each final project team, one person's AWS server should be shutdown but not terminated in AWS, and the other person's AWS server will now serve

By "running", we mean

• SSH and HTTPS connections to a.arcology.builders and b.arcology.builders now go to the same IP address, etc.
• The API server for a can be listening on localhost:5000,
  • and the API server for b on the same host can be listening on localhost:5001
• each customer can SSH into the same server that they could previously, using their one of 26 keypairs listed above

In fact, your boss bets you that you could come up with a plan and implement it fast enough, that the customer won't even notice the difference as far as their website is concerned.

What's the fastest way to accomplish this with the remaining servers, that will cause you a minimum of "fixing up" afterwards? Discuss with your final project teammates or a pair partner for the day.

For your team, you will run one AWS server. If you are on a solo project, you are invited to find another solo project and share a single AWS server. You will pick one of your AWS servers, and shutdown the other one.

You may wish to keep the shutdown instance around for a week or two in case you left unsaved work on it.

Here are a suggested few steps needed to share an AWS server

Whichever partner turned off their AWS server, copy the contents of your id_ecdsa.pub and send it to your teammate (via Signal messenger, Discord channel #sos-w24-backend, but delete afterwards)

On your laptop

cat ~/.ssh/id_ecdsa.pub

Drag and select it, copy it, and paste it and send where your teammate can read.

The receiving partner should log into your AWS server, and append it to their

~/.ssh/authorized_keys

file, using whatever text editor (nano, vim, etc.) is available and convenient. If the file doesn't exist, you'll create it by opening it with your text editor.

Paste your partner's public key, and also while you're at it, paste your own public key into it, save it, and exit.

When you list your .ssh directory, check that authorized_keys has owner read+write permissions only.

$ ls -lh .ssh/
total 20K
-rw------- 1 ubuntu ubuntu 287 Jan 29 05:26 authorized_keys
-rw------- 1 ubuntu ubuntu 513 Jan 25 01:01 id_ecdsa
-rw-r--r-- 1 ubuntu ubuntu 185 Jan 25 01:01 id_ecdsa.pub
-rw------- 1 ubuntu ubuntu 978 Jan 25 01:42 known_hosts
-rw-r--r-- 1 ubuntu ubuntu 142 Jan 25 01:42 known_hosts.old

If your authorized_keys has different permissions, you can change it with

chmod 0600 ~/.ssh/authorized_keys

Then have both partners try logging in with

ssh ubuntu@<your_subdomain>.arcology.builders

You should both be able to login, and see each other on the same server.

When one user logs into a new AWS server with their SSH key, they may see a warning.

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:3kFfVr6XzUcEAR5d9y1lRFDZKB8zuDL+WFix8ef/tpg.
Please contact your system administrator.
Add correct host key in /c/Users/Riley/.ssh/known_hosts to get rid of this message.
Offending RSA key in /c/Users/Riley/.ssh/known_hosts:18
Host key for rilesbe.arcology.builders has changed and you have requested strict checking.
Host key verification failed.

Have one user start up tmux, and the other user join the session with tmux a.

Then try echoing messages to each other in your shared terminal.

$ # Hello from partner 1

or install and use the UNIX talk command.

$ talk
Hellooooo

Instead of your cron job / system service that registers your subdomain on reboot, you may want to write a shell script that registers two subdomains on reboot, and cron / schedule that job.

NOTE: in both tutorials below, skip the ufw firewall part. It is easy to skip a step and accidentally cut off SSH access, which is what inspired this multi-tenant server activity in the first place.

Add a new nginx site and symlink for the second subdomain, then use certbot to get a new certificate.

https://github.com/TheEvergreenStateCollege/upper-division-cs/wiki/2024-02-08#tls-cert

Then add a new proxy_pass to the second server block, using localhost:5001 or another port for the internal API server, since the first server is using localhost:5000

https://www.digitalocean.com/community/tutorials/how-to-configure-nginx-as-a-reverse-proxy-on-ubuntu-22-04

the first