career - RicoJia/notes GitHub Wiki
Necessary Skills (from BrainCorp Job Search)
-
Machine Learning Expertise
- Techniques: Proficient in supervised, unsupervised, reinforcement learning, and learning from demonstrations.
- Frameworks: Experience with deep neural network packages such as TensorFlow and PyTorch.
- Optimization: Strong understanding of optimization algorithms.
-
Robotics and Perception
- SLAM and Motion Planning: In-depth experience with SLAM (Simultaneous Localization and Mapping), perception, and motion planning algorithms.
- Robotic Middleware: Skilled in designing and developing robotic systems using middleware like ROS (Robot Operating System).
-
Programming
- Python and C++.
- CI/CD tools and release management. Experiences
- Successful implementation of research findings.
- Demonstrated ability to bring machine learning-driven products to market. Work
-
Developing adaptive algorithms for robotic learning from human demonstrations.
-
Creating algorithms for supervised, unsupervised, and reinforcement learning for data analysis.
-
Enhancing robotic perception, mapping, and motion planning across various hardware platforms.
- Have an honest assessment of yourself w.r.t your peers
- Your nav and ML are relatively junior.
- Companies could hire junior to cut costs
- The job markets have fears about hyperinflation, but if these concerns are unfounded, tech companies will do better than expected.
- Then jobs would come back. 70% people could land a job within 3 months after lay-off. So the "high turn over rate" is "kind of" a hoax
- AI/ML is booming. AI Data Scientist, Software Engineer with an AI skillset, Machine learning engineer are all in demand.
- The working from home trend in general is shrinking. Don't expect too much from it
- Keep Learning, Keep Asking. Some big companies don’t like spending time sharing knowledge. Ask to transfer to a different team.
- Company expects loyalty but gives none in return. Get ready to be laid off.
- The learning curve is logarithmic, the first 5 yrs is good, then you plateau
- Learning what jobs you don’t want is also cool
- Perception of your work gives true value to your work!
- Company exec: so much money, your job is to be in meetings.
-
Long Term Items
- Write a letter to your future self, what do you want to be?
- Stand on a strategic level sometimes, don't drown yourself in day-to-day operations
- Use weedends to think about what you can do differently to get closer to your goal
- Touch base with your cohorts, and profs
- Shut off some old thoughts
-
Incrementally working on things you're not familiar with
- Personal Finances
- HR
- On a company level, see the storms of the business, not drawn in complacency.
-
What a good documentation should have
- minimal example. (short and sweet)
- illustrative description that documents its purpose
- 最后是职业发展。我们对导航感兴趣,并打算往那个方向深耕。
-
过去:
- AGV. AGV uses electro-magnetic tracks to navigate. (1953)
-
现在
- Multi-view geometry needs clear theoretical background. Deeplearning Blackboxes are not so successful yet. Deep learning does not require a very high bar - no need for computer vision basic knowledge. But SLAM does: needs: Bundle Adjustment, Camera Model, Epipolar Geometry, and Feature Extraction.
- 在国内,有indemind 做视觉导航
-
我不知道做导航的未来怎么样?
- 机器人必将更加广泛化。大多数机器人都需要导航。自动拖拉机等非传统机器人也将会有应用
- 未来可能前景:- FSD 端到端; language LLM + localization + planning。Multimodality (image, text, face)
- Method 1:
- Use lots of synthetic data
- Method 2: VinT from UCB, visual-nav-transformer. (Goal observation --network-->embedding, observations --network-->embedding) --> attention
- Method 1:
Goal: Phase 1: graph optimization. G2O, GTSAM; Front end visual slam, NDT, ICP; Phase 2: 在工作中寻找机会应用深度神经网络技术,逐步积累经验。之后,一定会有更多行人识别等深度学习的项目
-
Big Principles
- Don't get into a vicious cycle, you need to build a virtuous cycle. In a vicious cycle, you get attacked by other workers, and finally get expelled
- New ppl are unknown, so they have less credibility as ppl from within. Win their credibility!
-
Immediate Things to do
-
Get familiar with the organization structure, informal way of communication, Corporate Culture
- Observe how ppl respond, who points fingers, who has a broader view of business, who answers directly, Who are rly competent and who are new as well?
- Be alert about: the channels info will flow to ur boss
-
Talk to them about what you're trying to accomplish, not proving what you've achieved
- Think about solutions, and what you need.
- Be early about bad news.
-
Signal that things are changing, small victories, your early actions will determine how you're perceived.
-
Find out what's required for your new role, What you did previously may not work here!
- Expectations?
- Learn fast!
- Make a learning plan, prioritize
- Focus on a couple areas first.
-
Have a mental transition for the new role
- Don't be afraid, you can nail these new skills!
-
Soft Items
- Ask salespeople: what are customers getting from competitors, not us?
- Style: work early? Form of communication
-
- "Zero defects policy": once you shipped the code, it's much harder to track down the bug.
- Have a schedule
- Bug database
- reproduce the bug
- Test
- all test strings must have a comment
assert something // should always ...
- Unit test should be most common
- catch ros: https://github.com/AIS-Bonn/catch_ros
- Must create automated test
- Xray ticket
- all test strings must have a comment
- Your code is not personal, don’t be afraid of the criticism
- Build a Bug list
- Allow extra days not just development, but also for testing. Factor in everything.
- advice for integrating someone else's code: see the data structure (what's included) before you get started
-
robot autonomy team; learn from manipulation, perception
-
base salary 125000, 3000 shares, option stocks. Full time: health care, dental, vision: blue-cross blue shield. add dependence, Unlimited PTO, 401k.
- formal letter offer.
- 05/23/2022
-
Notes:
- They're not perfect, params may not be perfect, datasets may not be perfect. Learn to be critical
-
BEGINNERS, don't be too deep and too fine for things that you DO NOT need. When you need it, learn it deep, that's way more efficient than 为了学而学. Otherwise, DO NOT BLAME yourself for not being able to write the same code and wasting your time going too deep into it.
- Try run it yourself
-
For starting a new job:
- Learn how to deploy code on store, fast.
- might need to set up github on the robot.
- Learn how to debug code (python), etc.
- Learn how to deploy code on store, fast.
-
TODO items:
- In daily updates, tell the team who can be unblocked
- Be that "spark", who opens up an 30% effort and makes a difference.
- Think about higher-level mgmt, calibration tool, when to do what.
- Beware of solution bias.
- Absorb the changes along the way
-
Design Review:
- Problem background: what is ride mgmt?
- Existing Solutions
- High level: what pieces are required? which pieces are simple (boiler-plate), and which ones are not:
- Work out a structure,
- Implementation PLan: production-ready -> feature-ready
========================================================================
========================================================================
-
Books
-
What is right for you?
- What energizes you? This is important because you need to be engaged.
- I like robots actually doing good things for you. Not particularly fixated on a technology (e.g., detecting X through webcam, or localizing the robot using X and Y sensors.)
- Tech Lead is a good idea. Tight sense of community. Sometimes your work is not a top company priority.
- What energizes you? This is important because you need to be engaged.
-
What do Staff Engineers do:
- write code, coordinate projects, build relationships. But also more about API design, setting directions, not by personal interest, but for business choices.
- Mentor others
- Engineering decisions. Unexpected meetings + regular.
- Explore. We are hill climibing: go to the highest point in your area, keep doing that, hoping that it will work out. Staff engineers are always the one doing that.
- Some days, you feel like accomplishing nothing ... This is because early in your career, your create -> test -> ship cycle comes fast. But later, your cycle becomes weeks ...
- You don't have to prove your value so much. But your interest will be different
- Types
- tech lead, like jimmy, 1 for 8 engineers, coordinating teams, unblocking them, close to product manager; delegate task, specific technical goals, and might step in to solve problems.
- architect: domain expert, large company
- right hand: engineering vp, something like that
-
What life is like feedback loop cycle is longer. Expectations around your impact is growing, time available for work shrinks. Pace your work to your life
- avoid snacking: just focus on low impact, easy work
- Compensation for this role probably not so different than senior engineers
-
fast growing startups
- Problem: disk space runs out
- Should have staff engineers doing the invisible "glue" work.
-
As you continue in your career, expectation on your impact will keep growing. You tried to deprive your other time, but work maintains an aloof indifference to your sacrifice. Rather than rewarding it. Good read on how to pace yourself
- Pace yourself. For seniors, you need to achieve more in less time
- find meaningful work (yes, robotics)
- pace yourself, Let burnout happen later. Overall quality of your work reflects your mental health. Also, you need to stay in the same environment for a long time to build depth.
- So you need long term energy management:
- How anxious do you feel about work? If work anxiety is constant, maybe it's a good time for vacation
- Sustained impact, not spikes
- Pace yourself. For seniors, you need to achieve more in less time
-
Senior engineer what to avoid
- Avoid snacking. Keep most of your focus on high-impact items, even though they're large.
- Avoid preening (bird feather to look nice)
- What company values and your personal growth. Folks who focus on their energy on appearance rather than your progress? If you can't meet this, you will slip backwards.
- Avoid chasing the ghost: low-impact, high effort items (you're so familiar with it, but this is not what brings you the most benefit)
- know the status quo first.
- Know that success may not get attributed to you.
- Existential issues: financial
-
People:
- Serendipity is where you find most interesting opportunities. So be out there, and do it.
- Join a large company, meet people, and they will spread across the industry.
-
Compounding values of maintaining a good relationship with coworkers is HUGE. Long after you stop working together...
- Advice on difficult situations
- Investors of a start-up.
- Learn to leverage experts
- mentoring: mentor people that you will work for oneday; Push them harder, treat them better; Interns I mentor;
-
prestige
- prestige, unfortunately, is important. FAANG, students from elite schools.
- Accumulate each bit of prestige
-
Management problem:
- short attention span: have to hear a problem, and make a decision soon. But you haven't
- At BART we talked about life, we had ppl from range of ages, we really have a "family vibe"
- Meanwhile, I didn't feel that way from Robot Software
- Eng Manager:
- People: diversity: young vs old
- Torc Robotics
- The Team/Torc Fit interview will focus on your engineering experience, fit to the organization and which team is best for you. The purpose of this interview is to get a more in-depth overview of your experience, work projects and the technologies/languages you are most comfortable with (example - C++, Linux, etc).
- This will be a 60 minute panel interview with 2-3 senior members of our team (Engineering Managers, Directors). scope, fit to the organization, ideal team, and role
- They will be looking for someone who can speak to the projects they have worked on and the technologies, tools used. Your ability to speak to the technology/tools, project/product (successes, failures, and learning experiences) will be important.
- Ask clarifying questions, make sure you understand the questions and scope of the problem you are trying to solve.
- Be collaborative. Use the other resources in the room (interviewers).
- Humble, Hungry & People Smart. We are not looking for perfection. Focus on what you can solve, be transparent on what you can’t.
========================================================================
======================================================================== Story of Polymath Robotics
-
Reinforcement learning:
- data driven when Boston dynamics engineers are not available. <100hz is slow. For drones, up to 500hz. Radio -> ROS -> GPU is very important. You simulate action candidates across time horizon at 10hz. That is low level Pytorch model & CUDA optimized.
- Robotics startups failed because they can't do tasks cheaply
- Model-based learning allows you to see which pieces are being evaluated?
- Facebook is using reinforcement learning to mitigate hate speech.
- Timnit Gebru: AI ethics
- Reinforcement learning: 4.6M to train GPT3, openAI. Too expensive. Being used in Google datacenters' cooling system. Because 1. we know how to control and how the environment reacts very well. 2. We have a back-up. But it's trained on Reddit corpus
- Amazon ring security robot: security camera flies through your house. Roomba is simple, but cheap. That's why it's successful. Dishwasher unloading robot is extremely challenging. You might train one robot for one dishwasher, but others?
- Google deepmind: modular robots. Why don't we have Robots at home? Too big of a scale.
- Ray, Rlib from berkeley: for param tuning.
- Sensitive to small differences. Mujoco 1.5 vs 2.0, different.
-
Work tips
- Add gif for evidence, to show that your work is valid
- How to manage a software project:
- Have a user on your team, or you become a user.
- Shrink and rethink about your goals, until you have something useful
- Have a backlog
- Client can remove backlog work anytime.
- Changing ongoing work is a bit of a stretch.
- Make the actual software
- Changes a lot! Features will change.
- So build your stuff incrementally. A sprint of work is good for a review
- YOU GOTTA MAKE useful stuff during that sprint
- Review and change plans
- Improve your process
- Have a user on your team, or you become a user.
- Merge dev more frequently to resolve problems
- If you have related tickets, design them together, at least before implementation
- Lesson Learned: in failure logging ticket, I didn't do a thorough enough design session. I should have thought more about what options I had. (adding params to flexbe states)
- When onboarding a new job:
-
Expectation, onboarding doc
-
Have a 30 min session to introduce yourself. Build Trust.
-
Schedule 1-1 with team members: ask them what they are working on
-
Everybody's growth path? critical feedback?
-
Onboarding peers: problems in the teams, because your boss may not tell you that
-
Try to build some relationships: First year, no exceed?
-
Ask tech leads to draw some flowcharts
-
Start from fixing bugs, and reviews,
-
Use the "I'm new card".
-
Sometimes ppl don't have good comm skills. Explain in one sentence without 3 sentences.
-
Ask others to explain certain things to you
-
Watch previous all-hands
-
Promise something, then deliver it.
-
There's no "good/bad", just "fit/not-fit" for your career goals
-
在一个行业,you gotta reach a very high level, then once you want to pick it up, it will be easy. If not, 一毛半毛的,很容易退化
-
现在投入的多,以后更轻松
-
- How to manage a software company:
- stack health. make a chart, use some rubrics on its style tests, sim.
- early stage: you need tech, and service. Find big companies
- Need vision, localization, control
- big client - data - algorithm - small clients
- At last, there'd be 3,4 big companies, with small companies for specific scenarios
- Are there constant new sales? That's how mabu failed.
- Cory had a hard time finding ppl that he could trust. Letting go of responsibilities, they become bigger
Houston Robotics Market TODO https://builtin.com/companies/location/houston/type/robotics-companies
- Square Robots, Underwater Robotics
- Nauticus Robotics Underwater Robotics
- Rugged Robotics construction robotics
- Cognitive Space: aerospace
- Arix Tech, might be
- Coco, silicon valley
========================================================================
========================================================================
- What do you get:
- Investment banking?
- MBB consulting, senior manager, director, corporate ladder
- Northwestern has collaboration with these schools
- Consulting, boring
The role is between product manager and engineers. Planning -> Dependency Management -> Tracking Progress - Program Engineer: closer to the engineers.
- Product manager:
- Pain points
- measuring success
- What are some of the "big goals" you'd like to achieve in say 10 yrs, 5 yrs, and 2 yrs?
- Do you have side "fun" projects
-
Robotics is cool. But it
- has high dependency on hardware. Getting it to work with hardware is difficult, expensive, and long way to go?
- Market share of commercial robotics is not high?
- I think it's high, for domestic / personal robotcs (TODO: how high in each market)
- but even with a system like saycan, we don't have proven / reliable technology that does: go to sink, retrieve something. (TODO: checkout saycan? How does it work?).
- Therefore, our fields of choices are still specialized, which means limited market share in general. Unless you do those with clear applications & relatively easy implementations, like autonomous driving, auto-lawnmowers, Otherwise, it's difficult to be profitable.
- Diligent Robotics is one of the "lucky" start up that has a clear application with "proven" way to implement, and potential to grow into a more "generic". Accel robotics, also had a good
-
What about buying up accel robotics? The business DOES have a "proven" way to succeed; a potentially large market share; (retail + real estate??)
- Need to meet with creditors of the business;
-
Andrew Stout:
- 1999 - 2003 bachelors, swarthmore college. UM amherst for masters, and unfinished PhD;
- Alderbaran: 11-13, pepper; boston studio manager, 2013 - 2016;
- Anki:
- Vector: voice controls, characterful reactions; behavior tree viz tools; app integration
- circadian rhythm; sleep schedule;
- leaning behavior choices; software architecture design, natural language processing
- In general I'm happy to hear the discussion points at the meeting!
- I think the ambition to become a unicorn is exciting. This sounds like a good goal for a successful start up
- Project blackberry is ambitious, but a reasonable step towards higher levels of autonomy we try to reach
- Vivian spent a few slides talking about communication. I think creating a culture of sharing achievements, and trusting others to fix problems together is really key.
- We have achieved a lot! (doors, elevators are not easy tasks)
- It's great that we are actively summarizing our learned lessons. In Q1, we learned that:
- a major feature rollout takes a lot of coordination to interrupt regular services
- Simple vs complex sites ratio estimation was off originally.
- Time of day is vastly in our favor
- Hardware reliability can shift a site's readiness greatly
- To mitigate cash burn, we introduced project blackberry (establish remote ROC station at hospitals). CRT (Clinical Robot Technician). I think this is a smart move: providing more incentive for people, while reducing our cash burn.
- Peter: Too young for the job; interview for job in columbus. Book Loft: bookstore
- Marcel: amazon robotics; 11 month contract; massachutes; grilling at amazon, warehouse robots at work walkways AMR. Robot arm;
- Alex: chilli's; alex: recycle old system76
- Drive by wire
- https://en.wikipedia.org/wiki/Drive_by_wire
- Michael Popchuk: the host, very familiar with running restaurant costs
- Eric Han: Engineering Manager at Instabase. Senior Engineer, now a manager. Language building app.
- Zhen Song: Siemens Guy in building, senior data scientist;
- Hemesh: Manager, instacart delivery designs; Going thru web bootcamps in 2019, know ruby, react, etc.
- Preston: the paper aggregation & auto-recommendation guy.
- The recommendation system guys:
- william ? (blond hair mexican)
- Aaria ? (mongo db Atlas), also got experience in flast, docker, MongoDB
- The immigration paper guys
- Sujith: UCLA ms in ECE; ADAS
- Gianmarco: eVoTL, his immigration paper idea is interesting.
- Ramiro: sales, northwestern kellog
- Nick Marcatelli: Digital Marketing, SEO
- Sanjay: 20 years old, programmer, impressive!
- My team
- Michael
- Sherry: UX researcher. User Painpoints.
-
Pick your mentors:
- Nav: Shangzhou -> Matt ->
- Deep Learning: Nhat? Harjatin? Adam?
- Meet with them often.
-
Pick your team: doing courses alone is comfy, but having a team can help you check your sanity, and a lot more fun. They can provide potentially more creativity. Also, they can join your lifelong network.
-
Ask how they would like to tackle the projects:
- Machine learning: what kind of networks they would use to tackle your intended project,
- go about finding data sources
- what tooling they are comfortable with (Colab, AWS)
-
Some people do end up quitting.
- Problem, Solution, Key metrics, unique Value Proposition. Unfair Advantage (can't be easily bought/copied), target segments. Customer Segments (early adopters, be specific about the demographics).
- unfair advantge: professor, make a name there.
- Better to be monopoly in a small market, serving the most underserved.
- Criteria
- MOST IMPORTANT: DID YOU talk to customers? Are you solving a problem? Have you identified a target market?
- A mvp or prototype? Show your wireframe;
- Business plan:
- How do you plan on making this a success
- Are you solving a problem?
- Is your idea unique?
- Resource: https://www.techstars.com/communities/startup-weekend/organize-a-startup-weekend/plan-your-event/tools-for-participants
- I know you guys sometime cook cultural foods. Tell me how you shop. (OPEN-ENDED QUESTION! )
- If starting with: How many times in a week do you have trouble finding ingredients from your local grocery store? People may likely say no
- If they mention "couldn't find ingredients", How do you get around the problem of not finding the ingredients.
- Goal: see if this business case is valid.
- Michaels' Startup spent > $100000 in first year, just because they didn't talk to customers.
- How's life coming along? He didn't like the company, a super early startup. The founder Eason was a far colleague who wanted talents, but senthil didn't know him while he was there. Senthil had 40 interviews, They all had some caltech connections. The company founded by Eason's PhD student. They are familiar with deep learning, foundamental skills. Short interview process.
- Why switching to LLM? He was not passionate about LLM, but he's a founding member, and have more machine learningexperience. He had interviews with metropol, fulfil (josh, James) were working there, and agricultural robotics; Mujin:
- Nuro: Deep learning object detector
- Leetcode: almost all every company. Should solve a medium one without big struggle.
- What do you think would be the future of SLAM? Or a relevant money making area?
- Tractor, lawn mowers are "On/off road autonomy"
- Early companies don't do a lot of that.
- SLAM built by ARKit by google; SLAM is tied to sensor calibration: where they are positioned, their properties. As companies mature. Brenna wanted auto-calibration in SLAM (intrinsics, distortion params).
- Now the market has high demand and low demand due to layoffs. 700 within 24 hours for a front end role.
- It's a good idea to take more time, 1 month, see where you want to grow
- Industry findings
- Tesla asks you to drive 15 mile. It's autocalibration, they already have good camera position estimates, orientation poses. Autocalibration
- Let's start something, or a curriculum.
- Built visual odometry for Ghost.
- Building Connections.
- Senthil Network:
- Interviews; went to CVPR; SLAM pipelines, within the two companies.
- Senthil Network:
- Math:
- Moving forward what to do?
- Next projects:
- Breadth vs Depth
- Stereo Cameras
- Monocular SLAM
- ICP vs PnP
- Backend
- Different sending modalities
- G2O: Analytical Gradients
- Breadth vs Depth
- Deep Learning
- Front End: feature matching: matching networking
- FEature Detection production
- End 2 End frameworks: how well it works: https://arxiv.org/pdf/2108.10869
- Fuse Multiple views
- How to eliminate people (HD mapping, they need to remove.)
- Waymo: 3D Lidar;
- can fuse with radars, acoustic devices.
- Iphone has lidar enabled dip?
- RGBD:
- Senthil doesn't work with hardware:
- Math:
- Eventually you have to go through them.
- 1h 40 days a day. It doesn't click immediately;
- Next projects:
========================================================================
========================================================================
- Jack Quinn's LinkedIn I am an algorithm scientist/engineer with nine years of industry experience applying machine learning, computer vision, and deep learning methods to a wide range of real-world problems. During this time I broadened and deepened my technical skills by constantly thinking about what needs to be done to advance the project or company, and then doing it (and quickly learning along the way if needed). Aside from my main focus on algorithm development, this often means: creating software tools, collecting data sets, doing experiments, building prototypes, troubleshooting electronics, optimizing code, debugging, etc. Other times it means keeping up to date on the latest developments in AI and adapting them if it leads to improved performance on a relevant real-life problem. Sometimes it means inventing new methods – I am a co-author on 17 patents (10 granted, 7 pending).
I was one of the key contributors in building autonomous store technology at Accel Robotics. I was one of the first few employees and was involved in all stages from early prototypes all the way to publicly-deployed stores. I also led a collaboration across several teams (hardware, electronics, firmware, cloud, algorithms) to successfully build a product onboarding box.
In addition to my practical experience, I also have a solid foundation from my time working on a PhD in computational physics. Because of this background, I am not afraid to dive deep into the math when needed. I have published 7 papers in academic journals.I am an algorithm scientist/engineer with nine years of industry experience applying machine learning, computer vision, and deep learning methods to a wide range of real-world problems. During this time I broadened and deepened my technical skills by constantly thinking about what needs to be done to advance the project or company, and then doing it (and quickly learning along the way if needed). Aside from my main focus on algorithm development, this often means: creating software tools, collecting data sets, doing experiments, building prototypes, troubleshooting electronics, optimizing code, debugging, etc. Other times it means keeping up to date on the latest developments in AI and adapting them if it leads to improved performance on a relevant real-life problem. Sometimes it means inventing new methods – I am a co-author on 17 patents (10 granted, 7 pending). I was one of the key contributors in building autonomous store technology at Accel Robotics. I was one of the first few employees and was involved in all stages from early prototypes all the way to publicly-deployed stores. I also led a collaboration across several teams (hardware, electronics, firmware, cloud, algorithms) to successfully build a product onboarding box. In addition to my practical experience, I also have a solid foundation from my time working on a PhD in computational physics. Because of this background, I am not afraid to dive deep into the math when needed. I have published 7 papers in academic journals.
- Mason Markee
Hi Mason,
My name is Ruotong and I was previously working at Diligent Robotics as an autonomy engineer. I recently left previous job and moved to Houston to reunite with my family after a few years of separation. I saw that you guys are doing some interesting work in construction robotics, so I'm wondering if you guys are looking to get some addition help with autonomy? (I didn't see such postings on your website, just confirming)
My specialty is in robot navigation related technologies. In the past I've worked on 2D and 3D SLAM and navigation systems, camera calibration, as well as other infrastructure work such as behavior trees and task arrangement. Please check out my resume for more about my experiences.
If there is such an opportunity recently, I'd be very happy to know more!
Thanks,
Ruotong