Meeting Notes - lmucs/grapevine GitHub Wiki

#Tuesday, September 1, 2015 ##Class

Went over 401 class syllabus
Glanced over K'tah wiki
xkcd of the Day: Understanding Flow Charts
Listed possible project ideas:
- Service organization app
- Autonomous zero-knowledge reasonably secure point-to-point easy-to-use encryption in transit (maybe)
- Calendar produced by monitoring social networks
- Friendbuy/Criteo projects
- App sports team analysis with notifications
- Location-based get-together with friends
- Modified billiards game
- Game to teach fighting
- Something with Oculus Rift (physical game)
- Social chore/activity manager
Homework:
- Begin reading Uncle Bob’s Clean Code (ongoing)
- Read K'tah’s meeting class notes
- Read essay on frequently forgotten facts about software development
- Self-organize
- Come up with a catchy name

##Meeting

Group: Nicole Anguiano, Cameron Billingham, Juan Carrillo, Jeff Fennell, Matt Flickner, Joaquin Loustau, and Rachel Rivera
Absentees: Nicole Anguiano, Joaquin Loustau
App Idea: Calendar produced by monitoring social networks
App Objectives:
- login
- ability to add and remove Twitter/Facebook(/Instagram?) accounts for calendar
- natural language processing that will parse posts, deduce which posts are events, add tags to events (food, sports, etc.), and add those events to calendars
- ability to delete events
Low Priority Goals:
- sync with personal calendar
- add custom events
Technologies/Languages:
- server: Node.js/CoffeeScript
- client: TODO (possibly web, android, and iOS clients)
- DB: TODO
Research to do for next meeting:
- Twitter API Exploration: Juan
- Facebook API Exploration: Jeff and Matt
- Google Calendar API Exploration: Rachel
- Natural Language Processing API/Framework Exploration: Cameron and Joaquin

Nicole: choose at least one to research before next meeting

#Tuesday, September 3, 2015 ##Pre-class Sync-up/Class

Initial Architecture Design
Research Results
- Possible Text Analytics API: Meaning Cloud
- Twitter/Facebook API: will do updates through HTTP GETs instead of setting up feeds for now
- Google Calendar API: will explore more later; (other objectives have priority)
Notes
- Social Media API = priority
- Possible task tracking mechanisms:
- Project Manager: Jeff?
- GitHub Workflow:
  - Pull Requests (naming: {your initials}/{feature name})
  - Two people must review each PR before merging into master
TODO
- Meaning Cloud experimentation
- REST research; everyone come with a question
- Node.js/CoffeeScript tutorials
- look into API Blueprint
- Google web fundamentals course
- sketch out 2-3 milestones
- Nicole: get Twitter API token

#Sunday, September 6, 2015 ##Meeting - 11:00am

Present: everyone, Nicole and Juan through Hangouts
Decided on GrapeVine as name for our app
Review of Node and building REST API
- Watched Node.js intro video CodeSchool NodeJS
- Talked through Node.js npm and middleware
- Added morgan middleware to server for easy debug and logging
- Walked through server.coffee and evaluated what instructions did
Initial social media api sketch
- Outlined necessary Facebook Graph API calls
  - getting ID from page_name
  - calling page posts and events API ( e.g. -> ../{facebook_ID}/posts/ )
  - went through necessary parameters
    - &since={unix timecode}
    - &limit={int} (max: 100, default: 25)
- added this functionality to API together
- Repeated steps above for Twitter API
  - differences in access - OAuth object
  - similar parameters
- TO DO:
  - Juan: add parameter functionality to API twitter calls
  - Juaquin/Cameron: clean up Facebook server module
  - Everybody: research open-source NLP's and external API NLP's

#Tuesday, September 8, 2015 ##Class

Alex Schneider gave a presentation on security.
- Threat Modeling - Identify potential vulnerabilities and define countermeasures
- OWASP Top 10 Vulnerabilities
- Don't forget to update your list of vulnerabilities as application grows!!!
- Resources
  - OWASP
  - Microsoft SDL
Talked about Bobby Tables

##Meeting

Discussed ideas for event objects
Discussed NLP Apis
Discuss what we want to see in the calendar
- Hierarchy of views: Month > Week > Day > Events
TODO
- NLP Group should compile a list of tweets/posts and compare different NLP frameworks
  - Discriminate between events and nonevents
  - Identifies all relevant information (time, location, etc)
- Nicole will post calendar for us to see
- Juan will show some concept art

#Thursday September 10 ##Meeting @ 9:00am

Nicole shared calendar designs and theme song she created
Nicole shared some mockup ideas she had for web app
Rachel talked about possibility of google maps integration
Talked about green/purple color scheme
Jeff shared state diagram for user views he created
Jeff suggested Backbone.js to Nicole for frontend
Rachel discussed code jam for Sunday afternoon (3:00 or 4:00)
Discussed possibility of jamming in the Loft sometime :thumbsup:
To work on:
- Jeff and Matt: Investigate calendar packages for IOS and Android, start working on login screen view
- Nicole: design Some mockups for login screen

##Class

Agile / scrum lecture by Augie Alzuabi from Symantec

#Sunday September 13 ##Meeting

Rachel reviewed work on API
Jeff gave introduction on Android
Talked about ideas for database:
- We need to provide users id and authorization tokens
- Way to find events associated with a user
- List of feeds the user is following
- Add/Remove feeds to follow

#Tuesday September 15

Meeting

Decided to use Unix timestamp
- universal, would work on all platforms iOS, Android, Web
Cameron made a list of about 100 tweets and Facebook page posts for a sample dataset
NLP update
- finding it hard for Meaningcloud to pickup dates
- NLTK - Cameron's gonna test it out with the its date package this week
- Prioritize generating events from post (doesn't necessarily have to be exactly correct just make sure we can throw out some of the garbage)
- Joaquin - Stanford NLP might not work, it doesn't work for holidays (ex: BBQ on Memorial Day)
API almost good to go for front end to start working with
Dr. Toal is giving a Git talk today

#Thursday September 17

Class

Jerry from CrowdStrike
- talk about cybersecurity
- don't store passwords in cache (that means you Windows)
- authentication, logs, access tokens, all good things to have
- China Chopper
  - one line of code hidden away somewhere
- Outlook
  - put a rar file into the outlook picture folder
  - would retrieve with a simple GET
- Chinese hackers don't attack on Chinese New Year
Narrowed down to 2 NLPs
HTTP set up going to try to start hitting it from client side
gonna get Rally going in the next few days

#Tuesday September 22

Decided to go with Pivotal Tracker over Rally
Setup Pivotal
- Added everyone to our pivotal page
- Added user stories
Intro to Swift from Matt

#Thursday September 24

Mark from Google: *beware of compound words (e.g. "white house") *how many tweets would we need for effective clustering?
*Recommendations: Weka, LDA

#Sunday September 27

blocked by HTTPS: Android prevents from using self-signed certificates: There is a way around this but it makes things a lot more complicated
- in order to continue progressing, putting HTTPS in the backlog

#Tuesday September 29

Toal lecture on networking:
- Intro *networks, internet, the Internet, heterogeneity, security, ARP
- Seven layers (or alternative four layers): 7. Application 6. Presentation 5. Session 4. Transport 3. Network 2. Data Link
  1. Physical
- Data and Packet Transmission

#Thursday October 8

Lecture from Marcello Tallis
Machine Learning
What is Machine Learning?
- Difficult problems such as recognizing handwritten digits, checking credit card fraud don't know how we solve these in our brain program would be extremely complicated
- Instead of writing program by hand, we collect examples and specific the correct output
- recognizing patterns:
  - objects in real scenes
  - facial identities
  - spoken words
- Recognizing anomalies
  - unusual sequences for credit card transactions
  - unusual patterns of sensor readings nuclear power plant
- predictions
  - futures stock prices or exchange rates
Def: Constructing and studying methods that learn from and make predictions on data
Terminology:
- Observations
- Features
- Labels
- Training vs Test Data
Two common ways:
- supervised learning - we know training data labels
- unsupervised learning - unlabeled observations
- supervised learning: classification, regression - labels are continuous
- unsupervised learning - clustering, dimensionality eduction
supervised learning pipeline -
1. obtain raw data
2. feature extraction - initial observations, incorporate domain knowledge
3. supervised learning - run labeled data through learning algorithm
4. evaluation - how do me determine the quality of the model, we evaluate, test with test data, don't like then we iterate
5. predict - once we have our model we can predict other data

Sunday October 11

Meeting with Jeff and Rachel:
- discussion of exactly what kind of data the clients need from the API
- Slight altering of the endpoints
- deploying to Heroku
- code jam

Tuesday October 13

Code jam day in class:
- Rachel: further work on the API; deployment of both APIs to Heroku; EER of Postgres Schema; Heroku Postgres add-on; replace spoof DB calls with real ones
- Jeff: code jam with Rachel; help to finalize endpoints; Android event feed
- Juan: ?
- Joaquin: ?
- Cameron: ?
- Nicole: ?
- Matt: ?
TODO: everyone write 5 pivotal tasks

Thursday October 15

Lecture with Myron Flickner: an overview of the TrueNorth neuromorphic chip
- brain inspired architectures: Neurons, Axons, Synapses, Spikes
- chip used to run deep learning algorithms
- image classification at a very, very low power consumption
- the kind of people IBM is looking for to work on these kind of projects:
  - background in machine learning is a plus
  - good background in statistics
  - developers who have a lot of experience with highly parallel systems
  - MOST IMPORTANT: people who have learned how to teach themselves
TODO: release within the next week

Tuseday October 20

Stand-Up

Rachel
- Worked with Jeff and got databases deployed on Heroku
- Get well soon!
Juan
- fixed chronos asynch tests
Jeff
- fixed up Heroku API endpoints on Android
- login and error messages are up and running off of Heroku API
Matt
- calendar module up and running on iOS - views looking good!
- working on fixing API endpoints
Nicole
- fixing up new Heroku API endpoints for web app
Cameron
- started branch for classification trainer
- basic NLTK import and document function up and running
- TODO: organize test data in excel, set up Naive-Bayes, and some others (SVM)

REST lecture from Dr. Toal

http://cs.lmu.edu/~ray/notes/rest/
Representaional State Transfer

introduced by Roy Fielding

Important Concepts

Every resource has a (unique) identifier (e.g., URI)
Resources are generally nouns. Name your resources accordingly:
- Example: http://www.yourcompany.com/products/2443
- Non-example: http://www.yourcompany.com/getProduct?id=2443 (THIS IS BAD)
HATEOAS stands for Hypertext As The Engine Of Application State. It means that hypertext should be used to find your way through the API.
Small, fixed number of verbs
- Example: In HTTP, GET, PUT, POST, DELETE, HEAD, OPTIONS, TRACE only!
- NO VERBS IN NAMES/RESOURCES!

Thursday October 22

Jerry came to critique security measures
Notes from Jerry for security
- TLS for encryption(SSL is no longer safe)
- encrpyt postgres database
- transfer risk over to server host - SLA
- hash and salt passwords (pepper it as well if possible)

Tuesday October 27

NVIDIA
- HW
- SW
- Business
- Culture
Have to look at performance/power
Moore's Law -> approaching the end
- go back 8 to 10 years -> clock cycle was 3 gHz
- present day -> also 3 gHz
Instruction level parallelism that you can't get anything more out of
- need results from other pipelines
electrically -> voltage leakage
- transistor smaller and smaller
  - comparison: losing electricity with longer extension card
  - same concept happens at nm level with transistor
- as you send more and more power through the transistor, less is used for switching
CPU vs GPU
- CPU -> 12 to 18 cores
- GPU ->
  - first NVIDA GPU had about 256 cores
    - openGL and DirectX
  - today - 3000 cores in them
CUDA -> program a GPU - highest level of efficiency
- equivalent program in systems language
Amdels Law
- any sort of serial code
- it will dominate the execution time
CPU memory - 25-50 GB/sec for DDR3-DDR4
PCI latest - 16 GB/sec peak, 12 GB/sec sustained
GPU memory - 288 GB/sec GDDR5
3D ram is new future of memory (8 stacks) -> 1000 GB/sec
- through silicon not copper wire
- super low power
GPU's can get more out of Moore's Law -> because of parallelism
languages
- many different levels of libraries for different subjects for programming
- CUDA, CuDNN, CuFFT, DIGITS

Thursday October 29

Selected date for presentations
Monday, Dec. 14 - 5:00pm

Database Crash Course - Toal

Separate persistence concerns
ACID
- Atomicity - all or nothing, transacions are atomic
- Consistency - all actions preserve constraints
- Isolation - can't see intermediate results
- Durability - committed transactions withstand crashes
BASE
- Basically Available - CAP theory (Consistency, availability, partition tolerance)
  - Partition tolerant - redundancy, keeps database available even with partition
  - You get 2/3 really - trilemma (can't have consistency with partition tolerance and availability)
- Soft state
- Eventual consistency
Data Model vs Data System
- Model - logical
  - Big 4
    - Key-Value
    - Document
    - Graph
    - Relational (most common)
  - Some others
    - Column-family
    - triplestore - semantic processing
    - Object-Oriented
  - Historical
    - Heirarchical
    - Netwoek
Need to index in order to be fast
- hash-maps - constant
- search trees - logarithmic
  - log base 2 of a trillion -> 40
  - super fast
Scaling
- vertical scaling - buy a bigger machine
- horizontal scaling - spread over lots of machines
Key-Value
- massive amount of data
- see everying by key
- REDIS
  - very common
  - http://try.redis.io
Document stores
- stores all data of an entity in one place
- stores by nesting documents in other documents
- really nice for horizontal scaling -> super easy
- MongoDB
  - Why you should never use MongoDB
Graph
- Nodes and connections - a network
- Neo4J
  - Neo4J browswer - visualize database
- Both nodes and relationship have data
Relational Store
- tables, primary keys, foreign keys
- PostgreSQL
- MySQL
- don't scale horizontally very well
What is NoSQL?
- It is a movement
- can stand for Not Only SQL or No SQL
- kind of a reaction to relational databases dominating
  - relational databases aren't bad
  - there are just other things out there
- can be scaled out horizontally well
- big data
- flexible schemas - can add new datatypes and attributes without changing underlying structure

Tuesday November 3

Code Jam
Check in with Toal; TODO from Toal:
- broader description of the project; paragraph level— small to medium
- branching: keeping more up-to-date
- need a release
- update Pivotal

Thursday November 5

Classification brainstorming: https://docs.google.com/spreadsheets/d/1CUZ5XSmsX2hZNdcOEVVaOEd2Op6KRa4jq-7787CyFoQ/edit#gid=0

Tuesday November 10

Uncle Bob on Naming
- should reveal intent
- true cost of software in maintenance
- names that describe implementation details do NOT describe the intent
  - names are not for your convenience, they are for communicating your intent
- avoid disinformation
  - one of the worst sins a programmer can commit
  - example: concrete name to abstract class;
- pronounceable names
- drop silly prefixes; let IDE, compiler, and unit tests do the rest
- parts of speech
  - classes/variables are nouns, methods are verbs
  - avoid "noise words" (e.g. info, data, etc.)
  - booleans should be predicates
- Relationship between Scope Length and Name Length
  - shorter the variable scope, shorter the name; longer the variable scope, longer the name
  - highly visible function/class, shorter the name; not very visible function/scope, longer the name
    - exception: derived classes (start adding adjectives)
TODO: clean up bad names in code

Tuesday November 24

Brian- indie gamer
- worked at Disney: prototyping engineer
- now at USC games masters program
Resources:
- Game Developer Conference
- IndieCade— useful for reaching out to publishers directly
- Gamedevmap
- International Game Developers Association Los Angeles
- Women in Games International— monthly events
- Glitch City LA
Advice:
- communication is important
- code clean
- regularly reassess your time
- imposture syndrome is something everyone
Helpful day to day skills:
- version control
- naming conventions
- comments
- data structures
- algorithms
- graphics
- UX
- senior project skills (working with others, motivation, communicating)

Thursday December 3

Kine Lane: API Evangelist
Technology, politics and business of API
Politics
- APIs are copyrightable: Oracle vs Google
- APIs should be more open and interoperable
18f— agile tech for federal use. Geek squad for US gov
Code for America
Grant for data cleaning: endless opportunities for spreadsheet to JSON
Check out NASA APIs for funsies
HTTP literacy needed
Collision of hardware vs software notion of an API; SDK vs everything should live in the cloud
Shame vs Superpower:
- API Design
- Monitoring
- Performance
- Security: Snapchat shaming. A world of unintentional APIs. Security through obscurity is NOT a strategy,
- Privacy
Charles Proxy
Content type headers are a commonly misunderstood
POSTMAN and Apiary becoming new API IDEs?
Preferred tool for generating SDK: APIMATIC
API Transformer, Runscope

Tuesday December 8

Ed Bramanti A day in the life
- warm up questions *[4] * [4]
  - type of NaN
  - "Why am I a “ + type of + “”;
- JavaScript:
  - Makes you marketable
  - Node.js lets you start doing server-side work
- Go
  - Fast compile!
  - Strongly typed
  - concurrency
- KoaJS rocks
- Philosophie
  - agile: good to be introduced to it in college
  - "Make it better”
- How to get a job?
  - codewars/cracking the coding interview; work in little chunks
  - read the source code
  - Introduction to Algorithms
  - That “whole person” education: communicator, interaction with clients, how would you deal with challenges (be specific!)
  - email companies if they have no job listings fitting your qualifications