web_devel - RicoJia/notes GitHub Wiki

========================================================================

Basics

========================================================================

  1. Cookie: small text with with any detail the server wants;

    • Generated by browser, sent to site next time it visits

      flowchart LR
    A(Server creates cookie, <br>with unique identifier)
    -->B(browser)
    -->C(browser send <br>cookie back on request<br>)
    -->D(Get UUID, <br>database)
    ;
      Loading

    • User can see the cookie content!

  2. Session is multiple HTTP transactions, where you can share data across multiple transactions.

    • can reuse session for http requests. Can get session ID. Example

    • Per session, Permanent (before the browser is closed), or after the browser is closed

      • Flask.session is dict like object

    • Example

      from flask import session
session.get("email")

    • If a page takes too long to load: then may flask returns EOF

  3. HTTP request and reponse. Get VS Post

    • Post: Change part of the server
    • When you refresh your page, you actually submits the form
    • has query after ?: http://127.0.0.1:5000/?email=sadf%40sdafasdfd
    • Get: Just get the info
    • Patch: to update part of an existing request

  4. Jquery example

  5. Functional Programming

    • no side effects (no intermediate results) 
      def square_non_fp(x):
    s = x^2
    return s
def square(x):
    return x^2
    • This is in contrast to OOP
    • Reactive Programming: asynchronous data, event driven, of-course
      • stream: ongoing sequence of data, like ros subscription, it's reactive. In games, there's double buffering. You can async buffer another frame, then swap it

  6. UUID (universal unique ID): 128 bit number. No centralized entity monitors it. (time and macaddress, or random number are two methods to generate them).

  7. Dependency Injection: importing an object from another module

    class ControlPanel{
    ControlPanel(MQTTConnection* mqtt_connection){
        // this is dependency injection. you don't have to create an engine yourself.
        this -> _mqtt_connection = mqtt_connection;
    }
}

  8. Error Codes, status code:

    • 2xx are successful. 4xx are client errors (401 unauthorized, 403 no access, 404 not found); 5xx is server error
    • Forbidden Exception (you simply don't have permission)
    • HTTP code 500 vs 400x
      • 400: the client's fault
        • Bad Request (401) (if part of your request is missing)
        • Not found (404): resource not available
      • 500: request is valid, but file system is causing issue; database cannot respond
    • Funny cat pics for status code
      • response will be a well structured json
    • throw an error and let the framework handle that, if any bad requests happen

  9. Javascript Terms

    • a static website hosts fixed html, css, javascript

      • An API server is to provide HTTP endpoints

    • Artifact: output of a compilation, source maps

      • Javascript compilation: bundles and transpilation (from modern JS (ES6+) to an older version)

      • bundled and minified javascript: dist/bundle.js. Example:

        // module1.js
export function greet(name) {
    return `Hello, ${name}!`;
}

// module2.js
import {greet} from "./module1.js"
console.log(greet('Rico'))

===> Webpack
// dist/bundle.js
(()=>{"use strict"; console.log("Hello Rico!")})();

    • Source maps: {reduced_map: original code, good for debugging}

    • Asset: images, fonts, media files. They might be optimized for deployment

    • Transactional vs streaming: TCP vs UDP; Batching vs streaming

    • SLA: things like: respond in 10s, monitor report is out every month.

    • ACID: Say we have a bank system

      • Atomicity: Each transaction needs to debit A and gives B credit. These should happen without interruption
      • Consistency: Each database operation puts the system in a valid state.
      • Isolation: One transaction should be isolated from the other.
      • Durability: Data won't be loss upon crashes / power loss

========================================================================

HTML: hyper text transfer protocol

========================================================================

  • with flask, CSS, JS, html can be loaded with refresh.

  • It's a protocol to follow. <F12> -> network

    • There's return codes: 301(redirected), 200 is ok
    • IP (192.xxx) vs https://www...(URL, uniform resource locator)
    • HTTPS listens to port 443 by default.
    • HTML browser is usually written in C++. Builds a tree with nodes in HTML tags
      • This tree is called DOM. Javascript can modify the DOM dynamically

  • button:

    • type html <!-- use email and password as types, so 1password would know --> <input type="email" name="email/">

  • list: Can use bootstrap

  • Search Bar hiding & showing:

  • Pain points:

    • datalist automatically hides options that do not contain your input string. E.g., if input = "stewed beef", then suggestion "stew beef" is skipped.

  • spacing: there will be two lines in the if after rendering

    <div>
    {% if true %}
        yay
    {% endif %}
</div>
    • but with -. you will see this: <div>yay</div> 
      <div>
    {%- if true -%}
        yay
    {%- endif -%}
</div>
    • Or, in flask: this will keep the indentation, but remove new lines 
      app.jinja_env.lstrip_blocks = True
app.jinja_env.trim_blocks = True

  • HTMX: allows us to partially update the page instantly

    • Partially refresh the page, only 
      <script src="https://unpkg.com/[email protected]"></script>
<form> htmx ..., target = "target"
    <input field>
<form>

<display>

- In python:
    def func():
        return func
    • We want:
      • If text only: display them. This I'm not sure if htmx will support: because in other cases, we want to reload the page. So, we have "return redirect_url()", which returns a response object.
      • If it were text only return, it will be easier
      • So, it's tricky to just update the page

  • iframe here refers to the <iframe> tag that embeds one webpage into another

css

  1. Fonts

Conventions

  1. double underscore (e.g., form__button)is used in BEMs (Block, Element, Modifier) in html and css

  2. API names use kebab case

========================================================================

Flask & Jinja

========================================================================

  1. app.get()

    # equivalent to app.route(, methods=['GET', 'POST'])
@app.get('/login/')

  2. render_template(str_withnewline) will render the string

    "dsafsd 
dsfadsf
"

    into something like

    <br>                dsafsd
                
<br>                dsfadsf

    So, one way is to make string with new line in to a list, then in html, add <br>

    • In python, item.steps = item.steps.split('\n')

    • in html:

      {% for para in item.steps %}
    {{para}}<br>
{% endfor %}

  3. How to create a form:

    • in html: all inputs under this node will be submitted. This automatically becomes site.com/something?q=something&another_thing=anotherthing 
      <form action="/something">
    <input name="q" type="text"/>
    <input name="another_thing" type="text"/>
    <button type="submit">
</form>
    • In Flask, do 
      @app.route("/something/")
def func():
    query = request.args.get("q")
    # This is because in the search bar, you submission is considered an arg.

Jinja

  • set variable in jinja
  • | join filter in jinja
  • macros for post processing (if certain items exist, append somethign to it. Good separation)
  • Escape. Escape generally means "skip rendering something". 
    # 1. for main.jinjia2
# app
return render_template("main.jinjia2, html_code="<div>hej</div>")
# main.jinjia2 file, a pipe to escape filter
{{ html_code | e }}

# BUT if this is main.html, flask will AUTO-ESCAPE
so in main.html, 
{{ html_code }}
If you want to actually evaluate it,
{{ html_code | safe }}

# In main.html, if you

========================================================================

Javascript

========================================================================

  1. AJAX: asynchronous Javascript and XML
    • Funcitonality:
      • On event, create XMLHttpRequest object and send to server -> Server processes it, create a response back -> Process response using Javascript, update page
      • Not a launguage, but a built-in XMLHTTPRequest object (to request data from server) + DOM for JavaScript and HTML (Document Object Model). You Can also use JSON / plain text, not just XML.
    • Side notes:

      • DOM represents a page in a tree structure, which is object oriented. Then, a webpage can be manipulated by Javascript. For example:

        // document is the html page itself in DOM, so js can access.
const paragraphs = document.getDocumentElementById('paragraphs');

      • Technically, you can use DOM in python directly as well, though People always use JS for it

  2. document.querySelector(selector_id)
    • a css selector is used to select an HTML element. Like id, element (like paragraph).

TIL

  1. In kwargs for the template, always have the user name specified. Though it could be none

  2. google oauth requires running on https.

    • To run flask, one can use 
      pip install pyopenssl
openssl req -x509 -newkey rsa:4096 -nodes -out cert.pem -keyout key.pem -days 365
Go to https://support.google.com/cloud/answer/6158849?hl=en to add the keys
    - Add login callback URL there, too
flask run --cert=cert.pem --key=key.pem

========================================================================

Web Frameworks

========================================================================

Fast API

Introduction. It's pretty fast (like NodeJs), and can generate Swagger UI Documentation. - Also built on top of type hints. - And supports async programming with async/await

Min Example: python # app.py from fastapi import FastAPI app = FastAPI() @app.get("/items/{item_id}") def read_item(item_id: int, q: str=None): return {"item_id": item_id, "q": q} - To run: uvicorn app:app --reload - http://127.0.0.1:8000/docs will have the API documentation

Nest JS

  1. how to initialize class member variabes? 
    export class Foo{
    private readonly someNum: int = 0;
};
  2. what is 36e5? that's 36 * 10^5
  3. How to implement a failure case in nest js successfully?
  4. Can I add async to a function without await? - nope. However you can silence it if you ahve a good reason
  5. How do I run a test? npx jest TEST_NAME
  6. Is there only 1 describe in a jest test? YEs you can have multiple
  7. In a jest test, how to post? quite a few steps: TODO
  8. how to await sleep? in 1 line: await new Promise(resolve=> setTimeout(resolve, 4000))
  9. How to run a test in nestjs?
    • npx jest PATH_TO_TEST run this in node's root folder
    • What is jest.fn(): it's a mock function. in getLastUpdateTime: () => jest.fn(), We can mock return value on this.:
    • In nest, how to return an optional object
    • sleep: await new Promise((resolve)=> setTimeout(resolve, 1000))
  10. Model, Controller, View
    • Model: business logic
    • View: HTML code
    • Controller: intermediary between view and controller

========================================================================

Kubernetes

========================================================================

  • monolith: code in a single code base. UI, server side app, database, tight-coupling
    • so if anything changes in code, needs re-deploy
    • monolith has a size limit. Usually done by horizontal scaling. Assuming they are all the same
    • serverless - monolith Diagram: 
      cronjob -> monolith1, monolith2, monolith3 -> redis database (fast, stateless)
SQS queue (simple queue service)
      • if one dies, its data will be put on SQS. another pod will come alive
  • micro-service: comm via HTTP/rest. Smaller artifacts
  • Lambdas: like lambda functions. They are server-less. Needs glue code and orchestration to determine which calls which
    • how containers talk to each other
    • How to do without downtime
    • Like amazon Lambda, no size limits
    • But this could be a headache, stick to monolith for as long as possible
  • Container:
    • New container can be created to replace the old one
    • each container has a DNS, end points. Kubernetes can configure them
  1. A pod:
    • smallest deployable unit. Containers in the same pod share the same IP (so they comm over the same localhost)
  2. Nodes: virtual machines. They can be the same physical servers as well
  3. cluster: Nodes are grouped together there
  4. Control plane: controls k8s clusters

State

We have three more layers 1. (Route 53, DNS resolver). Route 53 is not free. 2. NLB (network load balancer), can fail, 1 or 2 failures per year), - whitelisting happens here - Layer 4: TCP/UDP ports loadbalancing. Layer 5 manages sessions to connect to different devices - TLS 3. Ingress controller (Nginx docker container) - ingress is to direct external traffic to correponding services - controller is the boundary with ingress and egress. Good idea to return a promise, because HTTP is async

4. Kubernetes (pod): in front it, they may return HTML. So, do not believe the response is the one you intentionally want get. Have a header check.
5. On the robot -> remote access point -> AT&T hospital network -(Bi-directional VPN authorization)-> AWS

========================================================================

Basics

========================================================================

  1. Motivation: component tree 2. make your own element (like paragraphs?)

    • each component has its own css and html
    • diffing: what's changed. Much more interactive

  2. File Structure index.js -> App.JS (App, App could be an extended module, or component) -> if App() is a component, provide render()

  3. Airbnb coding guide

    1. One component per file, called component.jsx
      • App.js also in components/
    2. states:
      • For stateless components, use functionsfunction foo{return HTML};
      • For stateful components, use classes class Component extends React.Components{}

  4. Env vars:

    1. env.example
    2. npm run setup generates .env, which copies the env example there
    3. put env vars there.

========================================================================

Video Widget

========================================================================

Theory

  • MJPEG: basically a series of jpeg images, easy to set up
  • RTMP
  • HLS: (HTTP Live Stream), developed by apple.
    • Break down a video into a few second segments
  • WebRTC
    • gRPC uses protobuf, which is a compact binary data format for structured data (with types). But gRPC is not optimized for videos

Implmentation

  1. MJPEG videos are easier to display in React

========================================================================

Postgress

========================================================================

  • migration files: should be immutable.
    1. Modify liftnet-config.entity.ts
    2. do "migrate:create".
    3. Append description to the end of the migration file

========================================================================

Misc Tools

========================================================================

  1. iperf to measure bandwidth between two nodes:

    • If I have a Python Flask "MyServer", I would launch "iperf" separately, on a differnet port
    • Then, run an iperf client, to the port. The client will consume all TCP packet bandwidth

  2. Use selenium to simulate button clicks?

    • However, if you need to perform automated tests without launching a visible browser window, you can use Selenium with a "headless" browser.

    • For an automated test, how does it launch mbot? (use the ECR docker)

    • Behavior-Driven Development (BDD) basics

      • .feature gherkin files, used in BDD framework like Behave. a feature file represent a scneario under testing
      • step: defines each feature
      • environment.py: set up and teardown, like data base initialization

    • QA bart? test env. Server: EC2. Mbot needs ui-sim

  3. Front end Testing - Steps:

    1. Create an instance of Mbot container - pull ui-sim dependencies
    2. Create a Behave Testing Container through docker-compose
    3. run the instance everyday in a QA-BART server, if gitlab doesn't play well?
    • ML/AI testing?? COREY IS AN ML LOVER!! Learn How ROC uses it.

Rico's wiki

Home | Copyright © 2021 ricojia

⚠️ **GitHub.com Fallback** ⚠️