web_devel - RicoJia/notes GitHub Wiki

========================================================================

========================================================================

1. Cookie: small text with with any detail the server wants;

   • Generated by browser, sent to site next time it visits

     flowchart LR
         A(Server creates cookie, <br>with unique identifier)
         -->B(browser)
         -->C(browser send <br>cookie back on request<br>)
         -->D(Get UUID, <br>database)
         ;
     

     Loading

   • User can see the cookie content!

2. Session is multiple HTTP transactions, where you can share data across multiple transactions.

   • can reuse session for http requests. Can get session ID. Example

   • Per session, Permanent (before the browser is closed), or after the browser is closed

     • Flask.session is dict like object

   • Example

     from flask import session
     session.get("email")

   • If a page takes too long to load: then may flask returns EOF

3. HTTP request and reponse. Get VS Post

   • Post: Change part of the server
   • When you refresh your page, you actually submits the form
   • has query after ?: http://127.0.0.1:5000/?email=sadf%40sdafasdfd
   • Get: Just get the info
   • Patch: to update part of an existing request

4. Jquery example

5. Functional Programming

   • no side effects (no intermediate results)

     def square_non_fp(x):
         s = x^2
         return s
     def square(x):
         return x^2
     

   • This is in contrast to OOP
   • Reactive Programming: asynchronous data, event driven, of-course
     • stream: ongoing sequence of data, like ros subscription, it's reactive. In games, there's double buffering. You can async buffer another frame, then swap it

6. Model, Controller, View

   • Model: business logic
   • View: HTML code
   • Controller: intermediary between view and controller

7. UUID (universal unique ID): 128 bit number. No centralized entity monitors it. (time and macaddress, or random number are two methods to generate them).

8. Dependency Injection: importing an object from another module

   class ControlPanel{
       ControlPanel(MQTTConnection* mqtt_connection){
           // this is dependency injection. you don't have to create an engine yourself.
           this -> _mqtt_connection = mqtt_connection;
       }
   }

9. Error Codes, status code:

   • 2xx are successful. 4xx are client errors (401 unauthorized, 403 no access, 404 not found); 5xx is server error
   • Forbidden Exception (you simply don't have permission)
   • HTTP code 500 vs 400x
     • 400: the client's fault
       • Bad Request (401) (if part of your request is missing)
       • Not found (404): resource not available
     • 500: request is valid, but file system is causing issue; database cannot respond
   • Funny cat pics for status code
     • response will be a well structured json
   • throw an error and let the framework handle that, if any bad requests happen

========================================================================

========================================================================

• with flask, CSS, JS, html can be loaded with refresh.

• It's a protocol to follow. <F12> -> network

  • There's return codes: 301(redirected), 200 is ok
  • IP (192.xxx) vs https://www...(URL, uniform resource locator)
  • HTTPS listens to port 443 by default.
  • HTML browser is usually written in C++. Builds a tree with nodes in HTML tags
    • This tree is called DOM. Javascript can modify the DOM dynamically

• button:

  • type

    ```html
    <!-- use email and password as types, so 1password would know -->
    <input type="email" name="email/"> 
    ```
    

• list: Can use bootstrap

• Search Bar hiding & showing:

• Pain points:

  • datalist automatically hides options that do not contain your input string. E.g., if input = "stewed beef", then suggestion "stew beef" is skipped.

• spacing: there will be two lines in the if after rendering

  <div>
      {% if true %}
          yay
      {% endif %}
  </div>

  • but with -. you will see this: <div>yay</div>

    <div>
        {%- if true -%}
            yay
        {%- endif -%}
    </div>
    

  • Or, in flask: this will keep the indentation, but remove new lines

    app.jinja_env.lstrip_blocks = True
    app.jinja_env.trim_blocks = True
    

• HTMX: allows us to partially update the page instantly

  • Partially refresh the page, only

    <script src="https://unpkg.com/[email protected]"></script>
    <form> htmx ..., target = "target"
        <input field>
    <form>
    
    <display>
    
    - In python:
        def func():
            return func
    

  • We want:
    • If text only: display them. This I'm not sure if htmx will support: because in other cases, we want to reload the page. So, we have "return redirect_url()", which returns a response object.
    • If it were text only return, it will be easier
    • So, it's tricky to just update the page

1. Fonts

1. double underscore (e.g., form__button)is used in BEMs (Block, Element, Modifier) in html and css

2. API names use kebab case

========================================================================

========================================================================

1. app.get()

   # equivalent to app.route(, methods=['GET', 'POST'])
   @app.get('/login/')

2. render_template(str_withnewline) will render the string

   "dsafsd 
   dsfadsf
   "

   into something like

   <br>                dsafsd
                   
   <br>                dsfadsf

   So, one way is to make string with new line in to a list, then in html, add <br>

   • In python, item.steps = item.steps.split('\n')

   • in html:

     {% for para in item.steps %}
         {{para}}<br>
     {% endfor %}

3. How to create a form:

   • in html: all inputs under this node will be submitted. This automatically becomes site.com/something?q=something&another_thing=anotherthing

     <form action="/something">
         <input name="q" type="text"/>
         <input name="another_thing" type="text"/>
         <button type="submit">
     </form>

   • In Flask, do

     @app.route("/something/")
     def func():
         query = request.args.get("q")
         # This is because in the search bar, you submission is considered an arg.

• set variable in jinja
• | join filter in jinja
• macros for post processing (if certain items exist, append somethign to it. Good separation)
• Escape. Escape generally means "skip rendering something".

  # 1. for main.jinjia2
  # app
  return render_template("main.jinjia2, html_code="<div>hej</div>")
  # main.jinjia2 file, a pipe to escape filter
  {{ html_code | e }}
  
  # BUT if this is main.html, flask will AUTO-ESCAPE
  so in main.html, 
  {{ html_code }}
  If you want to actually evaluate it,
  {{ html_code | safe }}
  
  # In main.html, if you
  

========================================================================

========================================================================

1. AJAX: asynchronous Javascript and XML
   • Funcitonality:
     • On event, create XMLHttpRequest object and send to server -> Server processes it, create a response back -> Process response using Javascript, update page
     • Not a launguage, but a built-in XMLHTTPRequest object (to request data from server) + DOM for JavaScript and HTML (Document Object Model). You Can also use JSON / plain text, not just XML.
   • Side notes:

     • DOM represents a page in a tree structure, which is object oriented. Then, a webpage can be manipulated by Javascript. For example:

       // document is the html page itself in DOM, so js can access.
       const paragraphs = document.getDocumentElementById('paragraphs');

     • Technically, you can use DOM in python directly as well, though People always use JS for it

2. document.querySelector(selector_id)
   • a css selector is used to select an HTML element. Like id, element (like paragraph).

1. In kwargs for the template, always have the user name specified. Though it could be none

2. google oauth requires running on https.

   • To run flask, one can use

     pip install pyopenssl
     openssl req -x509 -newkey rsa:4096 -nodes -out cert.pem -keyout key.pem -days 365
     Go to https://support.google.com/cloud/answer/6158849?hl=en to add the keys
         - Add login callback URL there, too
     flask run --cert=cert.pem --key=key.pem 
     

========================================================================

========================================================================

1. how to initialize class member variabes?

   export class Foo{
       private readonly someNum: int = 0;
   };
   

2. what is 36e5? that's 36 * 10^5
3. How to implement a failure case in nest js successfully?
   • create a custom module following this guide
4. Can I add async to a function without await? - nope. However you can silence it if you ahve a good reason
5. How do I run a test? npx jest TEST_NAME
6. Is there only 1 describe in a jest test? YEs you can have multiple
7. In a jest test, how to post? quite a few steps: TODO
8. how to await sleep? in 1 line: await new Promise(resolve=> setTimeout(resolve, 4000))
9. How to run a test in nestjs?
   • npx jest PATH_TO_TEST run this in node's root folder
   • What is jest.fn(): it's a mock function. in getLastUpdateTime: () => jest.fn(), We can mock return value on this.:
   • In nest, how to return an optional object
   • sleep: await new Promise((resolve)=> setTimeout(resolve, 1000))

========================================================================

========================================================================

• monolith: code in a single code base. UI, server side app, database, tight-coupling
  • so if anything changes in code, needs re-deploy
  • monolith has a size limit. Usually done by horizontal scaling. Assuming they are all the same
  • serverless - monolith Diagram:

    cronjob -> monolith1, monolith2, monolith3 -> redis database (fast, stateless)
    SQS queue (simple queue service) 
    

    • if one dies, its data will be put on SQS. another pod will come alive
• micro-service: comm via HTTP/rest. Smaller artifacts
• Lambdas: like lambda functions. They are server-less. Needs glue code and orchestration to determine which calls which
  • how containers talk to each other
  • How to do without downtime
  • Like amazon Lambda, no size limits
  • But this could be a headache, stick to monolith for as long as possible
• Container:
  • New container can be created to replace the old one
  • each container has a DNS, end points. Kubernetes can configure them

1. A pod:
   • smallest deployable unit. Containers in the same pod share the same IP (so they comm over the same localhost)
2. Nodes: virtual machines. They can be the same physical servers as well
3. cluster: Nodes are grouped together there
4. Control plane: controls k8s clusters

We have three more layers 1. (Route 53, DNS resolver). Route 53 is not free. 2. NLB (network load balancer), can fail, 1 or 2 failures per year), - whitelisting happens here - Layer 4: TCP/UDP ports loadbalancing. Layer 5 manages sessions to connect to different devices - TLS 3. Ingress controller (Nginx docker container) - ingress is to direct external traffic to correponding services - controller is the boundary with ingress and egress. Good idea to return a promise, because HTTP is async

4. Kubernetes (pod): in front it, they may return HTML. So, do not believe the response is the one you intentionally want get. Have a header check.
5. On the robot -> remote access point -> AT&T hospital network -(Bi-directional VPN authorization)-> AWS

========================================================================

========================================================================

• Flask
• HTML
• CSS
• JavaScript

• Web Crawler
  • People's linked in updates
  • React front end, node backend
• Regular courses
  • Node backend? Swagger? Curl API in node?
  • React front end?
  • SQL? Deploy the service on server?
  • JS up & going, JavaScript Good Parts

• JERJ Robotics

========================================================================

========================================================================

1. Setup
   • nvmrc
   • Cheerio
   • node-fetch
   • typescript
   • ts-node
2. Test set (the dom)
3. Send search request to linkedin
4. Fetch website
   • fetch
   • async vs await
5. Get elements
6. Crawl the links (linked in search page)
7. Ignore seen link
8. Stay on the same page
   1. urlParser.parse
9. absolute vs relative links
10. download content

========================================================================

========================================================================

• migration files: should be immutable.
  1. Modify liftnet-config.entity.ts
  2. do "migrate:create".
  3. Append description to the end of the migration file

========================================================================

========================================================================

1. Env vars:
   1. env.example
   2. npm run setup generates .env, which copies the env example there
   3. put env vars there.

========================================================================

========================================================================

1. Use selenium to simulate button clicks?

   • However, if you need to perform automated tests without launching a visible browser window, you can use Selenium with a "headless" browser.

   • For an automated test, how does it launch mbot? (use the ECR docker)

   • Behavior-Driven Development (BDD) basics

     • .feature gherkin files, used in BDD framework like Behave. a feature file represent a scneario under testing
     • step: defines each feature
     • environment.py: set up and teardown, like data base initialization

   • QA bart? test env. Server: EC2. Mbot needs ui-sim

2. Steps:

   1. Create an instance of Mbot container - pull ui-sim dependencies
   2. Create a Behave Testing Container through docker-compose
   3. run the instance everyday in a QA-BART server, if gitlab doesn't play well?
   • ML/AI testing?? COREY IS AN ML LOVER!! Learn How ROC uses it.