The Idea Machine

Learning at Harvard and MIT in the Age of Artificial Intelligence

Background

One of my favorite books is The Idea Factory: Learning to Think at MIT by Pepper White. It tells Pepper’s story as a student earning his Master’s in Mechanical Engineering at MIT in the 1980s, capturing his experiences as a student and as a researcher, including his constant feelings of inadequacy. Forty years later, as a Master’s student at Harvard conducting research in MIT’s Brain and Cognitive Science Department, these posts are my stories, inspired by Pepper and in tribute to those who came before. To highlight my favorite quote, “If I could see […] an insight, a new way of looking at [a problem] that would maybe, just maybe, find its way into future generations […] In the Eiffel tower of technology, I would be a rivet.”

Riding the NeurIPS Rollercoaster

This was my first experience submitting to the Conference on Neural Information Processing Systems, more commonly known as NeurIPS, and I had a rollercoaster of a time. We submitted the paper back in May without much hassle (I think we submitted a day early), and I put a reminder on my calendar that we’d hear back in mid-August. On August 7th, Ila and I received an email on that our reviews were accessible so we hopped on a call to walk through them: 7, 7, 8, 6 with confidence scores of 4, 4, 5, 4. Nice! After reading the reviews in depth, we found ourselves in agreement with almost all the comments and put together a short list of key revisions we’d make before the resubmission deadline next week.

Before continuing with the story, you might need some background. For those who aren’t familiar, NeurIPS Author Responses are a bit counterintuitive. I thought that we were given a week to (1) read our reviews, (2) decide what feedback we’d try to address, (3) make the appropriate changes and then (4) resubmit. Unbeknownst to me and Ila, only (1) and (2) are allowed; you can make whatever changes you like, but you only submit a 1 page promise of the changes you intend to make if accepted. You don’t actually make the changes and resubmit.

So for a week, I worked my ass off. In typical fashion, the TODO list seemed manageable when we first wrote it on Friday. By Monday, we had new results that merited further investigation. Then, on Wednesday morning, we were digging even deeper. By Thursday morning, with five hours left to go, the TODO list was four times longer than it had been a week ago. Ila had meetings all throughout the day, so we were calling again and again, whenever she could fit me in, to keep iterating. With 50 minutes left to go, I launched the jobs to regenerate our final plots that would take at least 30 minutes to run. As I sat there waiting, I thought there must be something productive I could do with my time. I decided I should practice submitting the paper to preempt any issues that might arise.

When I logged into the CMT submission system, I couldn’t find where to resubmit the paper. There was only one submission form, which permitted a single page PDF of the author response. Something was wrong. I was starting to panic. I texted all my friends who might know what the issue was. When I didn’t hear back within the minute, I started calling people. No one was answering. They were all busy with their own submissions. Finally, I got through to one of my friends. She laughed and said we were only supposed to submit the author response. I laughed at her and told her to stop fucking with me. But she was serious - and correct. She sent me the FAQ and I felt like a puppet with its strings cut. All the stress vanished and I collapsed into my chair.

NeurIPS Reviews

I had been told that the NeurIPS review process can feel arbitrary and even though I thankfully received solid scores, I had a taste of how capricious the process can be. One of my friends, who I think is utterly brilliant, had submitted a fantastic piece of work about how to apply a theoretical approach in practice. He received terrible scores from reviewers who didn’t see any value in such a contribution, even though figuring out how to do what the paper proposed wasn’t at all obvious. In comparison, I had another friend who called me the night before the NeurIPS deadline, on the brink of tears, because one of her supervisors told her he didn’t feel comfortable submitting the paper because it was half undone. She woke up that morning to a text from that supervisor, telling her that he had a change of heart. Her reviews were 8, 8, 9, 8.