• Talk to experts before and while you explore a new field.
• The only way to understand and analyze a system/algorithm is by making it deterministic. This means you should brutally cull any sources of noise and stochasticity until you can make sense of what you are seeing. This is true even for research in statistics.

Resources

1. Guide to ML Research: John Schulman
2. How to Write a Technical Paper: Michael Ernst
3. How to Make Paper Videos

Steps

1. Identify an interesting/challenging area and lit review like crazy.
2. Test the existing approaches on different hard/easy problems
3. Identify and log their successes and failures.
4. Construct a real-world problem that cannot be solved satisfactorily by any of them.
5. Think of a way to modify the existing strategies to solve it.
6. If you cannot, think of a different direction of attack.
7. If you cannot, go to [4].

Test Problem Design

This is probably the most underrated but by far, the most crucial step of research. By a test problem (or a set of them), I mean a problem that is complex and challenging enough to closely mirror the real-life problem, but tractable enough to allow you to try different existing approaches as well. If this problem is poorly constructed- say if you make unprincipled assumptions or simplifications, it will slow down your research considerably.

The performance of an algorithm on your test problem should be a good indication of its performance on the real problem. A good way to enforce this is to lock the problem before you start doing research. Ask yourself- "Will I want to publish a paper with experimental results based solely on this problem." And this includes visualization and problem complexity as well. If the answer is no, then do no process until you have such a problem.

Keep at least two set of problems: (1) Easy (2) Hard. Easy problems are those can be solved satisfactorily by existing methods and Hard problems are those that can't be. Whenever you come up with a new solution, first do a sanity check on the easy set. If it does not work well on that, then it is probably not a good idea.

How to Write a Paper

1. Just start. The most important thing to do is to get an imperfect first draft. Do not try to edit/improve as you are writing the first draft. It is a waste of time.
2. Make an outline with a list of figures and tables.
3. Start with the Experiments and Results section.
4. The Introduction is the hardest to write and hence starting here is an invitation to get blocked.
5. Write the Reference section at the end.

What to Focus On

1. Convince the reader that the problem is worthwhile, unsolved and that you solved it.

Reference

How to Write Research Code

1. Write as little code as possible. The complexity and hence the probability of bugs increases exponentially with the number of lines of code.

How to Meta-Study

1. Identify two or more topics that you feel are connected at a deeper level.
2. Find one book for each topic and study them in parallel.
3. Make an entry in your google doc.
4. Create a blog/latex doc to summarize the learning.