The Idea Machine

Learning at Harvard and MIT in the Age of Artificial Intelligence

Background

One of my favorite books is The Idea Factory: Learning to Think at MIT by Pepper White. It tells Pepper’s story as a student earning his Master’s in Mechanical Engineering at MIT in the 1980s, capturing his experiences as a student and as a researcher, including his constant feelings of inadequacy. Forty years later, as a Master’s student at Harvard conducting research in MIT’s Brain and Cognitive Science Department, these posts are my stories, inspired by Pepper and in tribute to those who came before. To highlight my favorite quote, “If I could see […] an insight, a new way of looking at [a problem] that would maybe, just maybe, find its way into future generations […] In the Eiffel tower of technology, I would be a rivet.”

A Paper From Start to Finish in 2 Weeks

Early in January, I approached Weiwei Pan, a postdoc who I had TA’ed for in her graduate Applied Math 207 Bayesian Methods course about a possible project that I needed her thoughts on. Through my discussions with her, I was able to clarify the research question I wanted to ask and answer: how can one efficiently perform inference in latent variable time series with Bayesian non-parametric priors over the latent states?

About a week after I had a clear question in mind, I was able to derive a result that I wanted and implement rudimentary simulations showing that my result held numerically. I asked Weiwei what would be a reasonable venue to target for publishing and we quickly realized that there were only two reasonable choices: NeurIPS, with a deadline three and a half months from now, and UAI, with a deadline in a week.

I had never heard of UAI but after some quick investigation, it seemed legitimate. Looking at 2020, it was sponsored by Google and the Vector Institute, and I recognize many organizers (David Sontag, Ryan Adams, Roger Grosse, David Duvenaud, Doina Precup, Caroline Uhler) as well as senior authors of accepted papers (Peter Dayan, Bernard Schölkopf, Philippe Rigollet, David Blei, Aapo Hyvarinen, Maria-Florina Balcan, Shimon Whiteson, Josh Tenenbaum, Finale Doshi-Velez). After about a half hour’s reflection, I realized I have too many interesting projects to pursue such that waiting three and a half months for NeurIPS to roll around wasn’t acceptable. I decided I’d drive for UAI 2021.

To make it, I’d need help. I had already been chatting about this research direction with my labmate Mikail Khona and my friend Blake Bordelon, and when I explained this deadline dilemma, they both agreed we’d might be able to make it working together. We created a list of what needed to be in the paper, prioritized the hell out of it and got to work. We met many times that week. To complicate matters, Blake lives in Texas which was hit with a winter storm and they lost power and water for a couple of days. By the end of the week, after many meetings and little sleep, we had solid results and submitted. After the UAI paper was submitted, unfortunately, I had no time to rest. My COSYNE poster was due the very next day and I had barely started it.

Of course, the paper is nowhere near perfect, so I’ll continue working on it to address predictable criticisms. By the deadline, I was admittedly exhausted, but I was surprised by the feeling of euphoria. The question on my mind moving forward is how to sustain such productivity in the long term. I think there were a few key ingredients:

1. Short term deadline
2. A clear, immutable TODO list
   • Immutability ensures you, your supervisor and your collaborators don’t lose focus of what matters
   • Immutability reduces exploration, which I think necessarily introduces drag
3. Motivated collaborators, which adds (1) camaraderie and (2) motivation to be a positive contributor
4. Frequent (daily) communication to make course corrections as necessary
5. Complementary strengths amongst collaborators