Dominic's Dev Diary infra - TheEvergreenStateCollege/upper-division-cs-23-24 GitHub Wiki

HW 03 infra

lets start by identifying what the goal were trying to achieve.

Goals are listed as:

• Connecting our Postgres database to our Express API server using Prisma Relational Modeling
• Design a data schema for our Postgres Database
  • man, it bugs me Postgres only has one 's' because then it would be post gress, gress being latin for move or step. So, after step or behind move.
• create service for two projects using systemd (don't remember that being there...)
  • python HTTP server (being one)
  • Express API server (being two).

Now, how does that happen?

Searching through the discord using the query before: 2024-02-01 in: sos-w24-backend express api we can see that pswish has linked this wiki page https://github.com/TheEvergreenStateCollege/upper-division-cs/wiki/Automating-the-starting-of-the-SQL-and-API-servers that talks about Automating startup of SQL (that's in docker) and API servers.

Though his tutorial uses Cron, which is not necessarily a problem as far as I know, but, Cron is designed for timed events (ie backup this drive once a month). Systemd is designed for startup and restarting applications. Not that I know how to use one better then the other so, we will see what happens.

We have poperigbys info on using systemd and the assignmnets page also links to this https://freshman.tech/snippets/linux/auto-restart-systemd-service/. which seems to be enough info to get systemd to restart something (as it was born to do)

So that might be easier, and, the "correct" way to do it. Again, as far as I can tell.

That's only the last bullet point. But an easy find.

I know that the only systemd thing that's in place on the AWS server is the dns server update so I know the above needs to be done. But what else...

Well, we know the postgres in the docker container exist and just needs to be told to turn on. But docker can do that, doesn't need systemd.

trying to get on the AWS machine I first transferred the .pem file from my laptop to my desktop (sense the laptop is on its last limb) Ran into the same copmplication of its permissions being jank. chmod 600 NameOfFile.pem to change.

First thing to fix, I can't belive I didnt push anything from the server to the repo! MonoRepos are scary. So lets first make sure I didn't do anything stupid in git... So I did plenty but it was never merged to the main branch. Quick fix. No conflicts. Would be scary if there were.

If we go tree in my working directory within the monorepo we get

.
├── node_modules
│   ├── @prisma
│   │   └── client -> ../.pnpm/@[email protected][email protected]/node_modules/@prisma/client
│   ├── dotenv -> .pnpm/[email protected]/node_modules/dotenv
│   ├── express -> .pnpm/[email protected]/node_modules/express
│   └── prisma -> .pnpm/[email protected]/node_modules/prisma
├── package.json
├── pages
│   ├── index.html
│   ├── search-hit-1.html
│   ├── search-hit-2.html
│   └── search-hit-3.html
├── pnpm-lock.yaml
├── prisma
│   ├── migrations
│   │   ├── 20240202200642_init
│   │   │   └── migration.sql
│   │   └── migration_lock.toml
│   └── schema.prisma
└── testapi.js

We can tell a few things pretty fast. Firstly, prisma exist. pnpm seems to be doing its job.

We can see we have a schema.prisma which is one part of this. So that's already done. and I can say that with confidence as the database has been migrated. That's awesome as that's apart of the 03. Nice.

We can even curl -H "Content-Type: application/json" -X POST http://localhost:5000/user -d '{ "username": "deffff", "password": "12345" }' to prove its running.. as it replies. So the database is live. I have been adding more 'f's every time I try to curl because if I don't it doesn't reply as that user already exists and they are unique. So that's proof more or less that it is going into the database. As it remembers.

If I am not mistaken, that is the first two things this assignment asks. Awesome. You can see why I was focusing hard on the recovery aspect of the assignment.

Lets start with postgres within docker. docker ps shows that:

8654dd72059e   postgres:14   "docker-entrypoint.s…"   5 weeks ago   Up 5 weeks   0.0.0.0:5432->5432/tcp, :::5432->5432/tcp   pg

exist and that would be our database. Neat. but, when it was created, was it told to restart? lets fine out Searching shows us... docker run -e POSTGRES_PASSWORD=<nope not going to show you that> --name=pg --rm -d -p 5432:5432 postgres:14 So we can tell that the flag --restart always wasnt used. docker does have a restart command and a shutdown command. But im going to start by seeing if theres a way to append the flag to an already running container first.

Naturally stack overflow has our back with: https://stackoverflow.com/questions/26852321/docker-add-a-restart-policy-to-a-container-that-was-already-created docker also accepts docker update so that should work.

Ah. Tried to update but, naturally --rm means to automatically remove it. thats not what we want. ok.. how do you remove that.. https://superuser.com/questions/1609468/how-do-i-clear-the-rm-flag-from-an-already-running-docker-container Seems like you can't. oookkk... well then I guess we make a new one.

Well I think its safe to assume that stopping the container dumps its data but that's OK it wasn't doing much. The above has some info on how one might copy it. but lets remake the container with the correct flags. as listed in pswishes document docker run -e POSTGRES_PASSWORD=<Still nope> --name=pg --restart always -d -p 5432:5432 postgres:14

Ok, trying to curl now gives us a connection refused. lets look at the discord messages to see if theres anything there... My assumption is there is some way that the server needs to be connected to the database and by nuking it I changed that. But theres also a comment from Paul talking about node becoming a zombie but I don't think that is what we have here.

ps aux | grep name is going to let us seach for processes. theirs nothing for np restarting the test api node testapi.js making it a background process. annd trying to curl it like before. and it explodes. great. I bet the schema needs to be migrated again. OK. I know what I did wrong. I tried to change the password for the docker container becuase I thought it was a bad password and that password is needed somewhere within prisma. but.. the password Paul had us use by default is soo generic its in several different places. Great... password isnt in the .env. that was my first guess. I think Pope rigby was compllaining about this. and its something in the gitignore

Going to just change it back as I am unsure.

Migrated the database... and BAM we are back online! Should figure out where the password is placed.

And it does infact nuke the old database as we have reset our 'f's. --rm makes a lot of sense doesn't it.

Cool. So that is the docker container ready to restart. Great.

Last thing that needs to be done then is systemd for python http server

Rebooting machine now to see if it reboots and starts the python process

Its still failing. the program is exiting but, thats a good start.