Developer Log: Alexi - alyoshenka/neo GitHub Wiki

This will be used to keep track of my progress throughout the completion of this project

January: 12 hrs

start 1/2

Following the tutorial here

  • Issue: can't connect using different topic name
  • Solution: change device policy (it was only allowing pub/sub with "sdk/test/python"

success! I can turn an LED on and off using the MQTT test client

time: 2 hrs

Proposal redo: 2 hrs

playing around with mqtt: 1 hr

  • Having issues SSHing into Pi in VSCode
    • updated and restarted

logged 1/14: 5 hrs

  • Working on issue #10
    • time: 2 hrs
  • Other stuff
    • gathering example vids: 1 hr
    • meeting with potential teammate: 1 hr
    • discussing with teammate; reviewing stock ticker: 1 hr
  • Revamping drawing library: repo
    • nested git repos? stock ticker should build on drawing library. probably too much stuff to make this a gist. I think I'm just gonna start it over.
    • add python project and set up linting + tests with Actions: 1 hr
    • you can draw a board!: 1 hr image

logged 1/27: 7 hrs

February: 14 hrs

Neopolitan

  • lots of pylint and import errors: 1 hr
  • redoing data transformation for writing symbols: 1 hr

Neo

  • make code easier to understand: 1 hr
  • install and configure pylint: 1 hr

Neopolitan

  • add CL arguments for message display: 1 hr
  • working on GH Actions error that is causing tests to fail: 1 hr
    • local pylint + pytest solved by removing __init__.py from src/
    • only run CI pytests in tests/ directory
    • add __init__.py with import sys; sys.path.append('src/') to tests/
  • CI tests pass: 1 hr
    • couldn't merge branch because I was updating branch protection rules for the wrong repo...whoops

logged 2/3: 7 hrs

Neopolitan:

  • did a bit more cleaning up: enabled scrolling speeds and padding if it scrolls itself out of data: 1 hr

Neo:

  • reorganize mqtt connection code a bit and work on linter: 1 hr
    • probably doesn't work because aws_iot is not included in repo
  • research git submodules to include aws_iot in the project without manually adding all the code to my repo
  • figured out linting error: was downloading pylint twice (once directly, once in requirements): 1 hr
  • get all changes merged in and resolve merge conflicts and clean up branches: 1 hr
  • get linter to work: 1 hr
    • solution
    • also added a .bashrc alias to run my linting command on lint
  • added pytest locally: 1 hr
    • need to configure secrets to get workflow to pass
  • set up local secrets: 1 hr
    • linting is good. tests fail, but are set up (almost) correctly
    • env vars need to match case. duh.

logged 2/10: 7 hrs

Neo

  • added an AWS MQTT test: 1 hr

Dibiasky

  • start a (simple) react app and start looking into AWS authentication: 1 hr
  • AWS Amplify tutorial: 1 hr
  • Different AWS Amplify tutorial: 1 hr

March: 34 hrs

logged 3/5: 4 hrs

Dibiasky

  • Got stuck on that tutorial and started looking for other ones: 1 hr
  • Working on AWS Amplify tutorial - trying to attach policy: 2 hrs
    • Success!
    • Solution: needed to add wss://* and */mqtt to aws_pubsub_endpoint

aws-to-pi-pubsub

  • Migrating example code to repo: 1 hr
  • Diagnosing ConnectionDisrupted errors: 1 hr
    • Solution: aws configure != sudo aws configure (run the latter)
    • IAM > Users > Allow user to IoTAttachPolicy
    • sudo aws iot attach-policy --policy-name 'policy' --target 'region:cognito' --output json
  • Set up repo secrets and push to GH: 1 hr
  • Read about designing MQTT topics: 1 hr
  • Design IoT topics and abstract MQTT connection code: 1 hr

Neo

  • Design IoT topics (same as above, just in Python): 1 hr

logged 3/10: 9 hrs

Neo

  • Design IoT topics on Pi; debug errors to push: 1 hr

Dibiasky

  • Working on designing IoT topics to get a response after publishing a command: 2 hrs
  • Setting up CI; adding some tests; setting up branch protection: 1 hr
  • Writing tests and dealing with Cognito/auth issues in Jest: 1 hr
  • Set up linting workflow: 1 hr
  • Fixing merge conflicts, messing around with React: 1 hr
  • Messing around with React, debugging useState issues: 1 hr
  • trying to figure out why it won't connect to Amplify: 3 hrs
    • Pi can still pubsub
    • still no idea
    • updated packages
    • checked that current PubSub version has no errors
      • potential issue with v5
    • solution: delete all aws-amplify files in node_modules and reinstall

logged 3/24: 11 hrs

Dibiasky

  • working on Cognito test: 1 hr
    • Can still get connection status and cognito id without withAuthenticator
    • web app running or not doesn't affect it
    • still struggling, tabling for now

Neo

  • Make it so webpage can run board operations: 1 hr

    • Allow all users to run neopixeltest with sudo without password
    • Add alias to neopixeltest in .bashrc
    • it kind of works, but not great (yet)
    • still need to input sudo password on first go

  • Working with sudo permissions to run neopixeltest: 1 hr

    • solution: allow command to flash LEDs (using sudo) to individual user, not all users
    • I can trigger an LED animation from a webpage, that's pretty darn cool!

  • Design a data topic: 1 hr

  • req/res to get available Pi operations: 1 hr

Dibiasky

  • webpage design, more repo organization: 1 hr
  • make webpage wireframe match design: 1 hr
  • working with flexbox: 1 hr

Neo

  • broke power supply somehow. Now just powering off the Pi: 1 hr

Dibiasky

  • refreshing on Redux: 1 hr

logged 3/31: 10 hrs

Official Start

Apr: 57 hrs

Dibiasky

  • same as above, more Redux: 1 hr
  • displaying log and current subscriptions: 2 hr
  • display Pi connection status: 1 hr
    • aws iot list-things --output json
      • "An error occurred (AccessDeniedException) when calling the ListThings operation: User: arn:aws:iam::{} is not authorized to perform: iot:ListThings on resource: * because no identity-based policy allows the iot:ListThings action"
    • Potential fixes:
    • This is the solution

Neopolitan

  • look into multiprocessing and events to allow listening for messages from Neo: 1 hr
  • same area, research threading and events in Python: 1 hr
    • here's a look at what I have so far
  • working with event queue to change displayed message: 1 hr
    • (not the best quality video) interrupt_with_event
  • code organization: 1 hr
  • added pylint, updated readme: 1 hr

Dibiasky

  • researching Amplify with multiple devs article and amplify auth: 1 hr

Neo

  • setting up laptop as a Thing: 1 hr
  • adding env var for client id, needed to get tests to pass: 1 hr
    • running neopixeltest twice also breaks on laptop, even after bash says 'command not found' on first run
  • trying to figure out why cl command stops execution on second run: 1 hr
    • works just fine using echo after taking off bashrc setup stuff
    • but neoptest then echo also breaks
    • as does echo neoptest
    • happens without sub.communicate as well
    • happens when command not found
    • shell=False same effect
    • issue was the '-i' arg

Dibiasky

  • add connection state icon (instead of just text): 1 hr

Overall

  • new teammate onboarding document: 2 hrs

logged 4/7: 16 hrs

Overall

  • cleaned up project board and added a bunch of milestones: 1 hr
  • added ideal roadmap to new developer onboarding document: 1 hr
  • teammate onboarding discussion: 1 hr

Neo

Dibiasky

  • added device connection status icon: 1 hr

Overall

  • team meeting: 1 hr
  • reading into Jest testing, helping teammate: 1 hr
  • team meeting: 1 hr

logged 4/14: 9 hrs

Neopolitan

  • document event-passing solution: 1 hr
  • having issues SSHing into Pi: 1 hr
    • update and restart didn't solve
    • updating and restarting computer didn't work
    • downgraded ssh extension to 0.96 fixed it
  • make the code work on the LED board: 1 hr
  • swap between hardware and graphical board: 3 hrs
    • still having ssh issues in code. removed .vscode-server/ from Pi

https://user-images.githubusercontent.com/38815390/233147289-9125a1d9-fb15-48fe-8f5b-a17ad2885765.mp4

Overall

  • team meeting, figuring out where to suggest we go next: 1 hr

Neopolitan:

  • working with events code and writing tests, make event handling happen in main: 1 hr
    • git remote update origin --prune to get rid of deleted remote branches locally

Neo:

  • change env vars to not be prepended with "Hubble": 1 hr
    • needed to add CLIENT_ID secret to workflow file

Neopolitan:

  • work on making abstract classes to swap between graphical and hardware displays: 1 hr

Overall:

  • helping teammate, reading about git: 1 hr

Neopolitan:

  • redesigning available operations workflow: 2 hr
    • git config --global push.default current to autosetup remote
  • making available operations buttons that can actually trigger the operations: 2 hr
    • + some project board organization

logged 4/21: 15 hrs

100 Hours

Overall:

  • team meeting, discuss where we should go next, canvas updates: 1 hr

Dibiasky:

  • working on getting a dynamic status of connected devices: 2 hr
    • lots of errors
    • was updating array state in useState wrong. solution
    • now issue of multiple additions to connected devices state

Neo:

  • adding neopolitan as a submodule: 3 hrs

Neopolitan:

  • learning about Python modules and packages: 1 hr

Dibiasky:

  • dynamic status of connected devices, lots of state update issues: 2 hrs
    • current state within a callback
      • useRef() seems to have fixed it
    • still having errors with saving a file not reloading page fully, probably due to my own misunderstanding

Neopolitan:

Dibiasky

  • working on making an available operations dropdown: 1 hr
    • using this component
  • populate the dropdown with available operations: 1 hr

Neopolitan

logged 4/28: 17 hrs

May: 88 hrs

Neopolitan:

https://user-images.githubusercontent.com/38815390/235481355-d68a4482-5063-4da7-bac1-ed0c60a8eb90.mp4

Neo:

  • integrate neopolitan package: 2 hrs
    • AWS IoT MQTT Test Client not loading
      • still doesn't work after clearing history and restarting computer. opening in private window works. no idea why
    • can run neopolitan from dibiasky (still WIP though)

https://user-images.githubusercontent.com/38815390/235502913-71c8bfc6-5d49-4cbf-a96e-da5df188d214.mp4

Overall:

  • work on integrating the three repos together: 1 hr
  • added logging to Neopolitan and Neo: 1 hr
  • build routes between Dibiasky and Neo: 1 hr
  • team meeting: 1 hr
  • Continue integrating - work on sending data to update display: 4 hrs
    • almost ready to go
    • it works!

https://user-images.githubusercontent.com/38815390/235954928-63d8fdaa-0867-4c91-b078-bc1365b896dd.mp4

neopolitan

  • get package to work on Pi: 4 hrs
    • Python version issue
    • VSCode hangs when installing project dependencies (neopixel -> scipy,numpy)
    • try installing with default python version (3.7)
      • can't because it needs >=3.8
      • what happens if I install it with 3.7 and run it with 3.8?
        • it installs, so does scipy. but will it run??
        • it will not run.
    • try installing troublesome packages on their own

neo

  • update logging and workflow: 1 hr
    • shouldn't have messed with CI. If it ain't broke...

Dibiasky

logged 5/5: 23 hrs

Dibiasky

  • working on deployment, setting custom domain: 1 hr Neo
  • Working with sub package logs: 1 hr

Neo

  • Revisit project objectives, research AWS EventBridge for #3: 1 hr
  • practice triggering lambdas from MQTT and MQTT from lambdas: 1 hr
  • research setting up EventBridge Schedule from a Lambda: 1 hr

Overall

  • team meeting: 2 hrs
  • look into using Step Functions instead: 3 hrs

logged 5/12: 10 hrs

Neopolitan

  • trying to get package to work: 4 hrs
    • dealing with Python versions
    • sudo apt-get install python3-pyaudio
    • sudo apt install libatlas-base-dev
    • neopixel dependencies
      • sudo pip3 install rpi_ws281x adafruit-circuitpython-neopixel
      • sudo python3 -m pip install --force-reinstall adafruit-blinka
      • pip3 install adafruit-circuitpython-neopixel
      • sudo python3 -m pip uninstall neopixel* issue was other neopixel library installed. it does something completely different
    • works, but not in venv
      • sudo apt-get install python3-dev python3-rpi.gpio didn't fix
      • pip install RPi.GPIO worked

Neo:

  • trigger Step Function that invokes a Lambda from an IoT Topic: 1 hr

Dibiasky:

  • add test button to schedule with an execution time: 1 hr

Neo:

  • work on publish -> lambda -> step function -> lambda pipeline: 1 hr
  • redesign topics and payloads: 2 hrs

Dibiasky:

  • update to work with new payload design: 1 hr

Neo:

  • brainstorm how to interface between React and DynamoDB: 1 hr

Dibiasky:

  • add schedule button: 1 hr
  • hook in schedule button to publish to MQTT: 1 hr

Neo:

  • lambda to validate operation and add to database: 1 hr
  • lambda to get scheduled operations database: 1 hr
  • work on publishing to actual operation lambda: 1 hr
  • document Scheduling Functionality: 1 hr

Overall:

  • make sure everything is working to prepare for presentation videos: 1 hr
  • make sure AWS code matches my documentation: 2 hrs
  • test and debug entire scheduling pipeline: 2 hrs
    • It works! I can schedule board events and have them run in the future
  • midterm presentation slideshow outline: 1 hr

logged 5/19: 23 hrs

Dibiasky:

  • add timepicker and make datetime inputs update commandTime value: 1 hr

Neopolitan:

  • update readme to document code usage: 1 hr
  • add demo code and videos: 2 hrs
  • update readme usage documentation: 1 hr

Dibiasky:

  • allow deletion of scheduled items: 2 hrs

Neo:

  • delete past items from database when its contents are queried: 1 hr

Dibiasky:

  • try to get Amplify to work with DynamoDB: 2 hrs
  • use AWS SDK instead: 2 hrs
  • update datetime picker for better UX: 1 hr

Overall:

  • working on midterm presentation: 2 hrs

Dibiasky:

  • work on setting both update options and scheduled time: 1 hr

Overall:

  • set up demo account for presentation: 1 hr
    • looks like it will take too much time to get it working for presentations tonight
  • small fixes for presentation: 1 hr
  • making diagrams: 1 hr
  • team meeting: 1 hr
  • add demo videos: 1 hr

Dibiasky:

  • try to make it so that you can update a scheduled operation: 1 hr

Overall:

  • team meeting, presentation prep: 1 hr
  • midterm presentations: 2 hrs

Dibiasky:

  • working on updating scheduled operation, little tweaks: 1 hr
  • start integrating Victor's changes: 1 hr

Overall:

  • re-prioritize important issues and make a game plan for tech showcase: 1 hr

Dibiasky:

  • restructure folders: 1 hr

Overall:

  • team meeting: 3 hrs

logged 5/26: 32 hrs

200 Hours

All Milestones Achieved

Overall:

  • playing around with getting stock ticker to work: 3 hrs

Dibiasky:

  • debugging issue where options disappear: 1 hr

Overall:

  • some general brainstorming and playing around: 1 hr
  • adding some more demos: 2 hrs
  • made simple stock ticker display: 1 hr

Dibiasky:

  • work on deleting step function along with scheduled operation: 3 hrs

Overall:

  • team meeting: 1 hr
  • messing around, fixing tiny things: 1 hr

Dibiasky:

  • change connectionstatus to Redux: 1 hr
  • trying to add demo user: 1 hr
  • broke Auth - swapped auth and unauth roles because I didn't read carefully. fixed: 1 hr
    1. Make new user
    2. Verify new user
    3. Get user credentials
    4. Attach policy to identity ID
  • connected devices when opening web app second; heartbeats kinda: 1 hr

Overall:

  • project organization, presentation prep: 1 hr
  • work on Topics and Payloads documentation: 1 hr
  • messing around with stock ticker: 1 hr
  • team meeting about tech showcase: 1 hr

Dibiasky:

  • fix connection status issue: 1 hr

Overall:

  • buy materials to make LED board case: 1 hr
  • work on LED board case: 2 hrs

June: ? hrs

logged 6/2: 25 hrs

Overall:

  • work on case: 1 hr
  • work on wiring: 1 hr
  • working on case: 2 hrs
  • architecture diagrams: 1 hr
  • tech showcase presentation writing: 1 hr

Neo:

  • clean exit of stockticker: 1 hr

Overall:

  • team meeting: 1 hr

Neo:

  • small bug fixes: 1 hr

Overall:

  • rebuilding display case: 3 hrs
  • trying to fix wiring mistake: 1 hr
  • documentation, working on final presentation: 1 hr
  • rewiring, trying to fix board: 1 hr
  • soldering and assembling board: 1 hr
  • working on tech showcase presentation: 2 hrs
    • metrics

Neo:

  • board clears on ctrl + c: 1 hr

Overall:

  • test presentation: 1 hr

Dibiasky:

  • team meeting and small fixes: 1 hr
  • fixing routing issue in deployment: 1 hr

Overall:

  • practice presentation: 1 hr

Dibiasky:

  • allowing multiple pages in deployment: 1 hr
  • working on About page content: 1 hr

logged 6/9: 25 hrs

Dibiasky:

  • filling out the About page: 1 hr

Overall:

  • small tweaks: 1 hr
  • setting up board: 1 hr
  • team meeting, small tweaks: 1 hr