I have this feeling that understanding state-of-the-art algorithms in any field is very hard. I keep reasoning that this is because the brightest minds in the field are working on it, and I’m not the brightest of minds. In this article, I think about it more closely and breakdown the misconception I have been having.

While it feels like state-of-the-art is very hard to understand (and that it needs the best mind), historically, it is clearly not true. This is put well in the book, Men of Mathematics, where the author talks about how a once “hard to understand” concept is understood easily by present day high school students.

"Newtonian law of gravitation was an incomprehensible mystery to even highly educated in the early 1700s. Any present day educated person accepts it as a simple truth." - Men Of Mathematics (Book) [rephrased for clarity]

This idea is furthered by a centuries of work we learn in very little time at an advanced level grad course (example). These are ideas that researchers have worked on for years and we’re getting a good overview of them in a week. There’s no reason to believe we wouldn’t be able to do the same with present state-of-the-art research.

I speculate that the following are the reasons why Newtons laws feel much simpler in the present day.

• Having prerequisite background: Present day student knows advanced math, so the math in Newtons laws comes very easily.
• Access to good condensed explanations: Present day student has access to good intuitive explanations of the Newtons laws (from books, lectures, etc.). They also have condensed versions
• Psychological advantage: Newtons laws are presented as the simplest of concepts in textbooks (presented in initial chapters). This has the psychological impact of students thinking that these ideas are easy, they are not intimidated anymore. Also, Newtons isn’t seen as the guy who has advanced knowledge on everything.
• Memorizing key results without understanding: We also use some formulas as facts without thinking closely about them. This helps us learn advanced concepts more easily but I’m not sure if it’s a good thing. It’s sad how few people can derive the formula for area of circle from scratch (I can’t do that either).

Therefore, if you want to get into learning present state of the art concepts quickly, I believe you could use the following tips.

• Have strong background of all prerequisite courses to understand the concept.
  • This includes both conceptual background and sufficiently advanced math to parse the ideas more easily.
  • Conceptual background: You don’t need to know everything that came up previously to this idea in the field, you just need to know the ones on top of which this idea was developed. Try to create a story of how these concepts came up. For example, If you’re reading about CNNs, you don’t need to know all about every previous NN architecture proposed. You just need to know the references in the main CNN paper well. Also, you don’t need to know all ideas in the related paper (tricks/hacks used to get better results are not necessary). You just got to understand the big picture and flow of the concepts well.
  • Mathematics background: Be so good at equations presented in the paper such that it’s easy for you to visualize what each equation is doing. (might not always be practical, use your judgment on the extent that you want to do this)
• Read it from the right sources.
  • Often, more condensed versions of the original papers are helpful to understand concepts quickly and well. Find such resources. They need to obey the following qualities: concise, explains intuition behind,
  • If there aren’t any, use it as an opportunity to create such a presentation of the concept.
• Get over the fear of state of art.
  • Freeman Dyson took 6 months to understand Feynman’s state-of-the-art work on QED theory and anything 6 months to use his background in math to much further these ideas. Have the clarity that under the right guidance, it wouldn’t take as long to learn completely new topics. Try to setup similar guidance in your life.
  • Realize that a few decade from now, people will learn these ideas in a page of two in their textbooks. These are not really complicated ideas, we just complicate them.
  • Understand that coming up with ground-breaking ideas is hard, and that we need to brightest minds for that. However, to understand those ideas is much easier.

In future, if time permits, I would like to think more closely on these ideas by imagining myself back in the 1700s and reading the original paper of ground breaking concepts which we presently deem as simple. Would be quite an experience, don’t have the time to do it now.

Did I miss anything here? Let me know by leaving an email.