Let's Talk About How We Talk About Science

A while ago, Brian Burg commented on Twitter that he would like to see more discussion of marketing in academia. I decided I'd rather write a meta-post about how we need to talk about how marketing is affecting our evaluation of science.

Beyonce.

Kim.

If you want to be on the cover of Glamour magazine, you know what to do. Put your hair in glamorous waves, wear something small, and stare directly at the camera with slightly open lips. It helps if you have the Look. (Has anyone else noticed that Beyonce and Kim are being airbrushed to look more and more like each other all the time?)

If you want to be on the cover of a glamour journal, things are not much different. Open with a deep-sounding but incontestable vision of where you think the world is going. Hone in on a specific problem. Make the problem sound hard. Make your solution easy for a casual reader to understand. Write with the voice of a winner. It helps to have picked a topic that a science journalist might drool over. Oh, and if you are going for the cover: make sure to have good images.

But, you might say, fashion magazines are frivolous, and science is Serious*. I'll be the first to agree that the investigation of the fundamental truths of reality is a worthy endeavor requiring brilliance, hard work, persistence, and all kinds of other positive qualities. (Side note: beauty is also hard work, and used to oppress women.) But people determine what science is higher-profile than other science. People live in society, and it is widely acknowledged that society is superficial. Many a fairy tale involves a causal relationship between the changing of clothes and the changing of fortune. In Thomas Carlyle's satirical novel Sartor Resartus, religion itself is a matter of clothing.

In fact, a major part of my metamorphosis into a Real Researcher has involved accepting that appearance matters. When my advisor and I used to get papers rejected in the beginning of my PhD, we would spend a long time thinking about how to make the work so good that the paper was not rejectable. I have come to realize that this is the equivalent of failing to impress on a first date and hoping that soul-searching will address the issue for the future. Looking deeply into one's soul, while usually good in the long term, often does not address the problem of first impressions.

Sure, part of preparing one's research for wider dissemination involves doing what everyone would expect of good communication: having a clear description of the goals, clear explanations of the solutions, and a clear explanation of the context with respect to previous work. Good logical reasoning goes a long way. Good evaluation of results does as well. But if we look at the papers that do--and don't--make it into the "glamour" conferences and journals, we begin to suspect that there are other factors at play.

If we look more closely, we can see that American** science replicates patterns of elitism and gatekeeping that we see in the rest of American society. In Privilege: The Making of an Adolescent Elite, Columbia sociology professor Shamus Khan reports on behavioral traits that characterize the new elite. Khan describes how, rather than stemming from family prestige, the social status of the boarding school students he observes comes from an ease of moving through social situations and a cultural omnivorousness (embracing both the high-brow and the low-brow). Especially since these behaviors are learned at elite institutions, they serve a gate-keeping function similar to explicit markers of socioeconomic status. People look for this ease and this omnivorous, for instance when interviewing candidates, justifying their choice with some idea that such traits somehow make people more deserving. There is also a mythology about hard work that serves more as a justification than an explanation for elite status: students feel that they are receiving the benefits they do from society not because they were born into it, but because they "worked so hard to get there."

As it turns out, the training of elite scientists also involves learning gatekeeping behaviors. In science there is, a similar mythology about hard work being responsible for differential success. In Computer Science, the privileged behaviors I've observed include having research vision (as opposed to making solid technical contributions), being aggressive about imposing that research vision upon others, and having a "genius quality," which involves pattern-matching on similarities to previous successful scientists (often white men). Like ease of interaction and cultural omnivorousness, these traits are often associated with people deserving of recognition, but their presence does not mean the work will be good. I would not be surprised if having research vision and exhibiting genius quality were more correlated with being educated in an elite American institution than with potential for long-term scientific impact. With this premise, the recipe for academic fame involves not only marketing one's work as making positive contributions to science, but also demonstrating a combination of privilege and flash. The privilege here is more subtle than that of having cover-girl looks, but it is a very real kind of privilege nonetheless.

But how, you may wonder, do people not see through the shiny exterior? Those who have been following American politics in the last year may be familiar with the answer: insufficient attention. Publications are reviewed by researchers under increasingly high demands to pass quick judgments. Between December 2015 and February 2016, for instance, I had accidentally agreed to be on two concurrent major conference Program Committees, and had a reviewing load of over 60 full-length (12-page, 9-10 pt font) papers. (And I am not the only person who had such reviewing volume!) Had I only been on one Program Committee, the reviewing load would have still required me to evaluate, on average, a paper every two days over the course of two months. Under such reviewing pressure, it is easy to succumb to flash judgments, emotional first responses to a paper's Introduction section. It is easy to accept the paper with the good story over a paper with a deeper but more subtle result.

Despite all this, I believe in the future of science, and that we can shift back to a situation where we are making space for "real" science, what science looks like before the makeup and airbrushing. To do so, we need to wage a similar campaign to the one people waged on unreasonable beauty standards. We need to teach people to recognize--and be skeptical--of "Photoshopped" results: all that is too slick, too inspiring, and too good to be true, in both individual papers and in the story of a scientist's career. We need to raise more awareness about what "real" science looks like: the incremental results required on the way to big discoveries, the science that is foundational, necessary, and often with subtleties difficult to communicate to non-experts. Making structural changes that reduce reviewing loads and allow for deeper evaluation also reduces the incentives that have led to the proliferation of these current practices.

Elite institutions are much more than a finishing school for scientists, but we have been moving to a model where the marketing is coming to dominate the science. To protect the pursuit of truth, we need to admit that people can be shallow when it comes to evaluating science. We need to talk about how we talk about science so we can make space for science that is slow, science that is subtle, and science that is outside the mainstream.

With thanks to Seth Stephens-Davidowitz, who told me my first draft lacked a cohesive point, and Adeeti Ullal, who very patiently helped me with the last paragraph.

* I don't believe fashion magazines are blanket frivolous, but you might.
** I don't have the depth of experience to comment on how this generalizes to other cultures.

What Professors Can Do About the Collaboration Problem

A few weeks ago, I wrote a blog post about the "collaboration problem" that sparked a significant amount of discussion among my colleagues in academic computer science, in large part because many people had observed the same problem, without ideas for great solutions. Here are some emails I've exchanged with Ben Zhao, a professor of Computer Science at UC Santa Barbara, and my colleague Claire Le Goues in the School of Computer Science at CMU about how to address the problem in the courses we teach. (Ben recently posted this article on social media and had quite extensive discussion with many people in the field about how to address the problem.

I hope this will generate even more discussion that brings us closer to solutions.

--

from:		Ben Zhao
to:		Justine Sherry, Jean Yang
date:		Tue, Dec 13, 2016 at 1:50 PM
subject:		looking for advice

hey Justine, Jean.

Random email out of nowhere, hope you’re both doing great, and happy holidays!! :)

So I’ve been thinking and reading a fair bit on group dynamics in CS classes, esp. w.r.t. female students, with a fair bit coming from you guys. There aren’t that many in my classes (I teach undergrad networking and OS, so they’re almost all juniors/seniors by the time they make it to my class). So I’d like to make sure that I’m not contributing any more to the gender imbalance.

I need help. As strong women in CS, would love your take on a couple of key questions (but also would love any general advice you want to share, period).  And I know you’re busy super busy, but hopefully this is something that won’t take too much time. Either way, your advice would immediately impact 10s of female students in the coming quarter...

Key questions on my mind right now are:
- How should I do group assignments for larger classes with moderate to heavy projects? About half of my networking class homeworks are in groups of 2, and nearly all of my OS class homeworks/projects are in groups of 2-3.
- From what I have thought about and seen in past classes, I think my past practice of letting students choose their own groups doesn’t work. I recall something like 1/2 of all groups with at least 1 female student experiencing some type of malfunction, either due to the male student(s) flaking out or just failing out.

Right now, I’m considering something like the following:
  - Beginning of quarter, I reach out individually to all female students in the class (maybe 10-15, 20 if I’m lucky), and I ask them to attend an open discussion with me on campus.
  - I ask them for their experiences and concerns in the class, and esp. for group projects
  - I lay out what I think are challenges that they could face
  - I give them the option to find partners within the group, before the overall group formation process starts.

What do you think? Would this work? Would female students react negatively to being singled out? What happens if they don’t care and don’t show up?

Thanks in advance, and again, I’d love to hear any thoughts on this or on any other topic..

thank you thank you!
Ben

--

from:		Jean Yang
to:		Ben Zhao
cc:		Justine Sherry, Claire Le Goues
date:		Wed, Dec 14, 2016 at 12:16 PM

Hi Ben,

  Thanks for writing! These are great questions, I'm glad you're asking them. I'm looping in Claire Le Goues, another professor at CMU, because we've been talking about how we could address some of these collaboration issues with curricular changes, and about proposing an audit of the curriculum to make sure students are learning collaboration skills.

  Here are some things I learned from people after my blog post about the collaboration problem:

- It's important to keep in mind that all students have trouble with collaboration. It may disproportionately affect female and other minority students because 1) there are already so many factors that wear away at their desire to participate, and 2) students without strong social ties within Computer Science may not have access to as desirable of a partner pool. But an important take-away is that all students struggle with learning how to collaborate well, that we don't teach it in lower-level courses, and that in upper-level courses collaboration ability becomes important for academic success all of the sudden.

- There is strong evidence that self-selected groups are not as good as instructor-assigned groups.

- There are many resources out there for helping students work in teams more effectively. I was given this as a starting point:

http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/Oakley-paper(JSCL).pdf

- There are ways to get students to more actively work on their collaboration skills. Claire addresses this in the software engineering course she teaches. Some professors have reported having students assess how collaborations went, and docking points for students who didn't collaborate well.

- Several women, including myself, said that their best collaborations during undergraduate were with other women. I'm still not sure what to make of this in the context of other findings.

  Given this, I have the following thoughts about your proposed plan:

- I like the idea of talking to students about collaboration issues, but there are two main reasons I wouldn't do it only with the women. First, collaboration is an everyone problem, and not just a women's problem. It also affects people along lines of race, sexual orientation, socioeconomic background, etc. Second, even if the problem were only one of gender, it's a problem to be addressed by people of both genders. I've long believed that in order to solve the gender problem, we need to address the stereotypes associated with both femininity and masculinity. Only involving one of the genders in the conversation places all of the burden on that gender, and when it's the women, we are burdening an already burdened group. For these reasons I'd encourage a discussion about collaboration with the entire class, and then support to ensure collaborations are going as smoothly as possible throughout the semester.

- It doesn't hurt to check in with women and minority students, but without making them feel like they are being singled out, or because you are interested in them primarily because of the women in CS problem. My undergraduate professors paid a lot of attention to me, and I always assumed it was because I was a woman, and in fact this made me feel like I was less deserving of attention.

- I do like the idea of making it easier for minority students to find each other, but I don't know that it's your place to do it as the instructor. I don't know if there's a non-awkward way to bring this up during the whole-group discussion. Also, based on what people say it actually seems better to assign the partners as the instructor, and then it would not seem appropriate to assign people to work together based on their minority status. I'm still really not sure how to think of partner choice vs. partner assignment, and welcome discussion about this!

  Curious to hear your thoughts after your Facebook post about this topic blew up. :)

Best,

Jean

P.S. This discussion is interesting. What do you think about me posting this to my blog, maybe after Claire/Justine chime in?

--

from:		Ben Zhao
to:		Jean Yang
cc:		Justine Sherry, Claire Le Goues
date:		Wed, Dec 14, 2016 at 1:27 PM

hey Jean.

Great thoughts.

I’ve been learning a lot from the various viewpoints on the FB post, but I’m slightly frustrated by the lack of consensus as to the right solution. First, I agree with all the viewpoints that the problem is broader (re: your point on everyone having collaboration issues and others’ points about male students sharing in the solution), and any effort to address it should be more inclusive. That I think is very doable: I can talk about the issue early on in the class with some of Sarita’s slides she shared on the FB post. Hopefully I can do it in a tactful way that doesn’t alienate any group.

By beyond that, I’m sort of torn. It’s clear that different personalities play into how different women reacted to my suggestions. Some, like my senior colleague Linda Petzold, reacted fairly negatively because (I think at least in part) she has a really strong personality, and perhaps had less of an issue handling those situations herself. Perhaps I’m generalizing too much based on a sample set of maybe 2-3, but I’m guessing there might be an inverse relationship between a student’s own ability to deal with these challenging situations and their sensitivity to being singled out. In other words, is it possible the women (or other minority groups) who are most vulnerable to the negative situations (because they’re less assertive or more introverted) would be less concerned about being singled out as a group?  I don’t want to downplay comments from you or Linda (and a couple others on the FB thread) about being singled out, but do you think that then sensitivity might be less of an issue for less assertive students? Given my slightly biased sample of strong female colleagues, I’m not quite sure how far off I am on this line of thinking.

My overarching concern is that a broader discussion, while very positive and definitely much better than nothing, is still not quite enough. I worry that individual students will find it difficult to reach out to me the professor to discuss group issues. This has been very much my experience in the past, that students don’t want to appear like they’re a hassle, and no matter how I try to make myself approachable and less intimidating, there’s always a high barrier to overcome (especially for those more shy/introverted students). So all those comments/suggestions that involve groups reaching out and giving me feedback about their own individual group dynamics, I think they’re somewhat naively optimistic.

So I will definitely do what I can for the broader class. But I worry that won’t be enough. Beyond that, I can do random group assignments. But there I foresee lots of complaints by students unable to work with their friends, and any personality conflicts will be blamed on me (which is ok). There I worry that the disruption to the class group formation as a whole will produce more issues, and I haven’t convinced myself that random assignment is a better solution in general.  The other option is more proactively reaching out to female students. There the question is do the ends justify the means: would the potential benefits of helping women students form self-selected groups outweigh the initial discomfort of being “singled out”?

Any/all thoughts welcome, and thanks for spending your time on this. I’m fine with whatever you want to post on your blog about the issue. I think more exposure can only help, as I’m pretty sure that most (if not all) of my male colleagues in the dept have no idea group dynamics is even an issue.

thanks,
Ben

--

from:		Claire Le Goues
to:		Ben Zhao
cc:		Jean Yang, Justine Sherry
date:		Wed, Dec 14, 2016 at 5:46 PM

I don't think that the following response is by any means complete, but here are some offhand thoughts:

I also wouldn't single out female students. I do think you can signal that you are a supportive ally in various subtle and not-so-subtle ways, especially at the start of class.  For example, giving the students a survey wherein you ask for their names, preferred names/nicknames, and pronouns indicates that you are a person who understands that pronouns are a thing worth asking/caring about.  This can indicate to marginalized students that you are more likely to be educated/aware of gender dynamics overall and thus they may feel more comfortable approaching you with concerns.  

My take is: Many women or other members of underrepresented groups *know* that life can be challenging as a woman in homework groups.  Having someone tell me those challenges neither solves them, nor makes me feel much better.  On the other hand, having you publicly talk to everyone, men included, about challenges that groups face, covering elements like subconscious bias, diversity/groupthink, etc, and the ways those forces hinder effective teamwork, might frankly resonate more with the women than singling them out, and might actually get the guys to think about their lives/privilege/behavior a little bit.

What you might do, if you don't necessarily want to go the random route, is ask *all* the students at the beginning (as part of a start of semester survey) if they have someone they want to work with.  You can say something like "I haven't decided yet how to assign groups but I am willing to entertain suggestions, so let me know by filling out this form; I will not share your answers with anyone."  If all the women pick someone reasonable, and all pairs are matched (like I say Jean and Jean says me), then you let them pick their own.   That way you're neither saying "HEY WOMEN YOU ARE BEING SINGLED OUT" but you can still get at the information you want.

We do assign students to groups pseudo-randomly, which is honestly pretty consistent with the literature.  We ask for schedule availability (there's an online tool for this I can dig up) and use that to assign groups, looking to maximize times they can work together while honestly trying, when we can, to split up known cliques.

This is all made easier by the fact that I teach a class explicitly about software engineering, including teamwork and process, and so we can very easily and truthfully say: You will go work for a company and get assigned to work with a team of people you do not know, and so being able to do that effectively is one of the learning goals of this class.  The students are generally receptive to this argument, even though there are always a dysfunctional team or two.  Teaching a systems-y class lends itself to the same line of argumentation: either they're going to industry or academia, and regardless, they need to learn to work with people who are not their friends.

The literature is mixed on group composition.  There is some evidence that putting members of underrepresented groups together is good in early classes (100-200 level), and additional evidence that past that, it doesn't really matter (because if you haven't dropped out yet, group composition is unlikely to be the deciding factor?).

Other thoughts on what we do: 

(a) We provide opportunities for individual assignments/assessments, ideally with each group assignment or milestone (so the first part is group work, and a smaller component is to be done individually).  This lets us identify malfunction and ensures that we are actually assessing individual as well as group performance, 

(b) We do not do peer grading by default but reserve the right to start if teams report serious problems.   Peer grading skews incentives within groups in a way that interferes with our particular learning goals in an SE context. However, it might work in your context where the "learn to work in teams" is less explicitly a goal of the course. 

(c) (speaking to your concern about students being hesitant to surface or discuss issues with you) At various points in the semester we specifically survey the groups about how they think they're doing and aggregate the feedback from everyone to send it back to them (these are teams of 3--5, so it's easier to do anonymously than teams of 2).  The form we used is based on literature on assessing group performance; I can find it if you're interested.  We then reach out to students who are reporting problems.  We talk to them individually and then also reach out to the whole group as appropriate (if the individual student says they're OK with it).  We do encourage them to try to sort it out by talking about it amongst themselves, and regardless, we follow up with the individual students to see how they feel after a week or so.  We do not use those feedback forms for grading in any way (and we tell them so).  I have found that about half the time, the frustrated students just want to vent for a half hour and then say "yeah I feel better, no need to do anything else." ;-)  When we bring them in as groups, we try to make them do as much of the talking as possible.  Like "Hey everyone, how's it going?  What do you think you're doing that's working well?  What are you doing that's not working well?  How can you fix it?"

(d) We cover effective teamwork in class at the start of the semester; practices like "have specific roles that you rotate; document agreements and assignment of responsibilities" (also from the literature---I think the Oakley article Jean linked).  I'd emphasize both the explicit assignment as well as the "rotation" aspect of the roles---otherwise, the female students tend to get "scribe" duties all the time. We've debated ways to enforce those practices (like asking for documentation of who does what), but have never formally done so.  

Honestly, I've never heard anyone say that having the students pick their groups worked out particularly well, frankly.  There are profs who say "Oh, but they complain if we assign them randomly!" And perhaps I'm too cavalier, but my response is: so?  They also complain if the tests are hard and if we give them too much homework, and we do it anyway because it serves our pedagogical goals.  I feel the same way about assigned groups.

Assigned groups don't solve all sources of team dysfunction, of course, and so I think we should as a curricular point do more to mitigate the risks that groupwork poses particularly to marginalized students and to teach students how to work together.  They spend their childhoods being told they're not allowed to work with others, and then we throw them into teamwork situations with no training, and then are surprised when they're terrible at it.  I think covering those challenges and strategies to mitigate them and proactively paying attention to team dynamics over the course of the semester is important to help them learn, though I by no means think we have a complete answer on that.

-CLG

On Avoiding Stress Culture

I've been at Carnegie Mellon University as an Assistant Professor for a little over a month now, and the students tell me we're approaching "Deep Semester." The glow of summer vacation has worn off. People are skipping classes and skipping meals in pursuit of Excellence. A pall of Seriousness has descended upon the Gates Hillman Complex. (Many days this week, the Seriousness has physically manifested as heavy rain.)

Now is a good time to remind myself that I can stay out of it*. Jim Morris, the former Dean of CMU's School of Computer Science, once told me, "Stress culture is worst among junior faculty. Avoid it." Jim seems to live by this advice. He teaches a course called Campus Stress as a Wicked Problem**. My friend Chinmay, who is in the Human-Computer Interaction Institute with him, tells me Jim is one of the few professors on campus who isn't "so busy" all the time.

At first glance, it may seem difficult to avoid stress culture as a junior faculty member. After all, everyone knows that being junior faculty means working all the time, never sleeping, and having so little of a life outside of work that you can only keep plants alive if they are in the office. But people also seem to think being a PhD student means working all the time, and there are many examples of successful people who did not work all the time as PhD students. Thus I'd like to posit the hypothesis that the idea that one must work "all the time" as junior faculty comes more from a culture of stress than necessity for success.

But why, you might wonder, would this stress culture exist among junior faculty members if it were unnecessary? I speculate below:

• Your responsibilities are much more divided than they were before and it's difficult to juggle. It's possible to spend all time doing any of the following: teaching, advising, writing grant proposals, and attending committee meetings. Also note that this list does not include the reason you presumably became faculty in the first place: doing research. The solution is not to implode, but to compromise.
• The closer you get to the top of a hierarchy, the more intense people get. When I go out into the real world people find me to be a total megalomaniac. My academic peers don't seem to think the same thing about me.
• A lot of people who made it all the way to becoming faculty did get there by working all the time. Though not the only way, this is a legitimate way of working.
• As humans we're not engineered to say "no" too often, and there are infinite things to say no to as a faculty member. If I said yes to every meeting and answered every email I'd die of not eating and not sleeping very quickly. (This might be a harder thing for women because we're socialized to be agreeable.)

In support of my hypothesis that stress culture is something to be eschewed rather than embraced, I present a list of my role models when it comes to finding space and balance:

• My undergraduate professor Radhika Nagpal. This recent excellent profile of her talks about how, as junior faculty, she avoided politics and made it a rule not to check email on weekends. She wrote the most-read post on Scientific American's website about her approach to the tenure track called "The Awesomest 7-year Postdoc."
• Turing Award winning MIT professor Barbara Liskov, who famously worked only 9 to 5 on weekdays, working an evening here and there only if there was a deadline.
• The aforementioned Jim Morris, and also my friend Chinmay, who seem to make time to do the things they want to do.
• My postdoc advisor Walter Fontana, who lives by the Goethe quote "Do not hurry; do not rest." He seems to have always found the space to do the science he wants to do. He once told me it is important to have a "strong internal compass" and know when you believe your work to be good so you can avoid pressures to hire more and publish more.

Stress culture might not be bad for everyone***, but it certainly is not productive for me. (My friend Seth once observed that I seem to work best in the complete absence of pressure.) So though I could be doing more work, right now I'm going to go read a book. Good night.

* In Influence, Robert Cialdini says if you want to do something, tell the entire world. Then you'll feel more accountable and be more likely to do it.
** The course focuses on problems at CMU, but in terms of pressure CMU is not so different from the other elite higher-education institutions I've experienced.
*** A student once told me I needed to put more pressure on him so he would get more work done!

Counter-Advice for the PhD

Recently I attended the Programming Languages Mentoring Workshop, a program to introduce advanced undergraduates and early-stage PhD students to research in general and research in our field. (By the way, this is a fantastic workshop and I highly encourage students to attend!) While listening to advice from other academics and talking to the students about their questions, I realized that I have come to disagree with much of the conventional wisdom and advice for PhD students, some of which I have been guilty of re-dispensing. I express my dissent here.

Clarification: The workshop was not the source of all of the quotes! It was simply what got me thinking about the dangers of taking any one piece of advice too seriously. (The workshop itself is great for showcasing different points of view.)

--

"To decide what to work on, read lots of papers and then choose the best problem."
One of my undergraduate professors once told me, "Take all advice with a grain of salt. Most advice is highlights and wishful thinking." This was one of the best pieces of advice anyone has ever given me. It is easy for people to give this kind of advice about choosing research problems after they have learned what makes a good research problem. The advice is far more difficult to follow for people who are still developing their research taste. While some people probably have chosen research problems this way and while it is helpful to more deeply understand your area, early-stage researchers sometimes just have to jump in, do things, and learn from the confusion.

Also, in the early stages of a PhD, developing research skills (project management, time management, and communication of results) can be far more important than working on the best problem. In this case, I would recommend working on a problem that a mentor is sufficiently invested in to help you gain the skills you need.

--

"Choose an advisor you are completely compatible with."
This advice goes in the same category as the previous one. Once you have gone through the PhD and developed a deep understanding of who you are and who your advisor is, it is easy to think that a good situation can be easily recreated or a bad situation could have been more easily avoided. While you should look enough into your soul and do enough due diligence to make sure there are no glaring red flags, you should not worry if you do not feel like you know enough about your working style or preferences to choose a perfectly compatible advisor. Advisor-advisee relationships, like all other human relationships, depend a lot on many factors, only some of which are under the control of the two main participants, and also can evolve quite a bit over time.

--

"Do a PhD because you are in love."
 I completely agree that doing a PhD out of love of learning, love of discovery, or love for a discipline is a much better reason than doing a PhD for the money, fame, or glory. But I've seen many students get stuck in the "passion trap," the idea that you need to be completely in love with something before you invest significant amounts of time and energy into it. According to Cal Newport, who has written extensively about this, passion is something that often comes later, after you have become an expert and people recognize you for your contributions.

What I have noticed is that people often differ more in the narratives they have about their relationships with their work than in their actual relationships with their work. In my Quora answer to the question "How common is it for PhD students to do work they are not passionate in?" I talk about how one's relationship with a project often follows a trajectory similar to a romantic relationship: infatuation, followed by a steady state that comes sometime later, often much later, with a period of confusion and negotiation in between. I've seen every researcher I know well experience the confusion phase, but some researchers are more open than others to talking about it.

A side observation is that relationships with research seem to vary culturally: for instance, being blanket positive about one's own research seems to go along with the American tendency to be blanket positive about one's own life.

--

"Superstars are born, not made."
No one has said this specific phrase to me, but many people have implied it with the qualities they value in students. Once a professor told me that some students "just can't cut it." I've seen professors pick favorites based on internal metrics they have (often, it seems, based on how much a student reminds them of themselves). I've seen students decide someone is the smartest among them because of confidence, or some other "star" quality that doesn't necessarily correlate directly with research skill. While there is a baseline level of intelligence, curiosity, drive, and tolerance for uncertainty that someone needs to be a good researcher, many of the qualities that make a great researcher--discipline and persistence, to name two--are not entirely innate and definitely not strongly correlated with the confidence and charisma that seem to build many star reputations. (Note: confidence and charisma can also be learned.)

--

"The PhD is lonely without a significant other, especially if you are a woman in a male-dominated field."
I was surprised that people have told me this--and that, at least when I was starting my PhD, there was a common conception that having a romantic partner was somehow necessary for enduring the trials of the PhD. While it is important to nurture healthy relationships with supportive people, a significant other does not need to be one of them. Especially in relationships between people of similar levels of ambition, it can become tricky to negotiate coevolution and colocation, thus adding unnecessary pressure to the PhD experience. (And, unfortunately, because of society's insistence on holding on to gender roles, women who date men often find themselves with more pressure to conform to their partner's desires.) During my PhD, I had many friends, some of them women in male-dominated fields, many who ended up becoming stars in their fields, who were happily single for significant portions of their PhD.

--

"The most successful PhD students work all the time."
See my answer (and other answers) to the Quora question "Do Ph.D. students have time for hobbies?" Nurturing a relationship with a significant other also counts as a hobby, so you can do that too if that's what you want.

--

"If it's not hard, it's not worth doing."
There is something to be said for only doing things that help you grow in some way, but there is necessary challenge and then there is unnecessary challenge. During the PhD, it is necessary to come to terms with uncertainty, confusion, and possible rejection of your ideas from the community. This is crucial for one's development into a full-fledged researcher. What is less necessary, however, is depriving yourself of food or sleep, always working to the point of exhaustion, or mismanaging your time so that you are always under deadline pressure. For some people it may be necessary to endure toxic advisor or collaborator relationships, but I would encourage those people to seek ways out of that if possible--abuse does not need to go hand-in-hand with growth. Self-inflicted struggle only makes the necessary struggle more difficult.

--

I hope you realize by now that there is no single right way to do the PhD and that there are many valid--and sometimes conflicting--views on what a good path is. Have fun with the confusion. :)

Why It's Not Academia's Job to Produce Code That Ships

Note: The images in this post are inexplicably broken due to some kind of Blogger bug. If someone is reading this at Google, please help!

My scientist friends often scoff at crime show writers' creative interpretation of technology's limits.

The technology shiny world of CSI: Cyber.

"Let's zoom in here," a character says in an investigation room with floor-to-ceiling screens showing high-definition maps of the show's major metropolitan area. A flick of the fingers reveals an image of the suspect at mouth-watering resolution.

In another scene, the characters listen to a voice mail from the suspect. "What's that in the background?" one investigator asks. Using an interface that deadmau5 would kill to have, the hacker of the bunch strips out the the talking, the other sounds. They say some words like "triangulation" and, eureka, they deduce the suspect's exact location.

Yes, real police technology is nowhere near this sophisticated. Yes, nobody (except maybe the government, secretly) has technology like this. But those who criticize the lack of realism are missing the point.

The realities that art constructs take us out of our existing frames of perception--not only for fun, but also for profit. Many important technological advances, from the submarine from the cell phone, appeared in fiction well before they appeared in real life. Correlation does not imply causation, but many dare say that fiction inspires science.

Some complaints against academic Computer Science.

This brings us to the relationship between academic Computer Science and the tech industry. Recently, people in industry have made similar criticisms of academic computer science. Mike Hoye of Mozilla started the conversation by saying he was "extremely angry" with academics for making it difficult for industry to access the research results. This unleashed a stream of Internet frustration against academics about everything from lack of Open Access (not our faults) to squandering government funding (not entirely true) to not caring about reproducibility or sharing our code (addressed in an earlier blog post).

At the heart of the frustration is a legitimate accusation*: that academics care more about producing papers than about producing anything immediately (or close to immediately) useful for the real world. I have been hearing some variation of this criticism, from academics as well as industry people, for longer than I have been doing research. But these criticisms are equivalent to saying that TV writers care more about making a good show than being technically realistic. While both are correct observations, they should not be complaints. The real problem here is not that academics don't care about relevance or that industry does not care about principles, but that there is a mismatch in expectations.

It makes sense that people expect academic research results to work in companies right away. Research that makes tangible, measurable contributions is often what ends up being most popular with funding sources (including industry), media outlets, and other academics reviewing papers, faculty applications, and promotion cases. As a result, academic researchers are increasingly under pressure to do research that can be described as "realistic" and "practical," to explicitly make connections between academic work and the real, practical work that goes on in industry.

In reality, most research--and much of the research worth doing--is far from being immediately practical. For very applied research, the connections are natural and the claims of practicality may be a summer internship or startup away from being true. Everything else is a career bet. Academics bet years, sometimes the entirety, of their careers on visions of what the world will be like in five, ten, twenty years. Many, many academics spend many years doing what others consider "irrelevant," "crazy," or "impossible" so that the ideas are ready by the time the time the other factors--physical hardware, society--are in place.

The paths to becoming billion-dollar industries.

In Computer Science, it is especially easy to forget that longer-term research is important when we can already do so much with existing ideas. But even if we look at what ends up making  money, evidence shows that career bets are responsible for much of the technology we have today. The book Innovation in Information Technology talks about how ideas in computer science turned into billion-dollar ideas. A graphic from the book (on right) shows that the Internet started as a university project in the sixties. Another graphic shows there were similarly long tech transfer trajectories for ideas such as relational databases, the World Wide Web, speech recognition, and broadband in the last mile.

The story of slow transfer is true across Computer Science. People often ask me why I do research in programming languages if most of the mainstream programming languages were created by regular programmers. It we look closely, however, most of the features in mainstream languages came out of decades of research. Yes, Guido Van Rossum was a programmer and not a researcher before he became the Benevolent Dictator of Python. But Python's contribution is not in innovating in terms of any particular paradigm, but in combining well features like object orientation (Smalltalk, 1972, and Clu, 1975), anonymous lambda functions (the lambda calculus, 1937), and garbage collection (1959) with an interactive feel (1960s). As programming languages researchers, we're looking at what's next: how to address problems now that people without formal training are programming, now that we have all these security and privacy concerns. In a media interview about my Jeeves language for automatically enforcing security and privacy policies, I explained the purpose of creating research languages as follows: "We’re taking a crazy idea, showing that it can work at all, and then fleshing it out so that it can work in the real world."

Some may believe that all of the deep, difficult work has already been done in Computer Science--and now we should simply capitalize on the efforts of researchers past. History has shown that progress has always gone beyond people's imaginations. Henry Leavitt Ellsworth, the first Commissioner of the US Patent Office, is known to have made fun of the notion that progress is ending, saying, "The advancement of the arts, from year to year, taxes our credulity and seems to presage the arrival of that period when human improvement must end." And common sense tell us otherwise. All of our data is becoming digitized and we have no clue how to make sure we're not leaking too much information. We're using software to design drugs and diagnose illness without really understanding what the software is doing. To say we have finished making progress is to be satisfied with an unsatisfying status quo.

The challenge, then, is not to get academics to be more relevant, but to preserve the separate roles of industry and academia while promoting transfer of ideas. As academics, we can do better in communicating the expectations of academic research (an outreach problem) and developing more concrete standards of expectations for "practical" research (something that Artifact Evaluation Committees have been doing, but that could benefit from more input from industry). As a society, we also need to work towards having more patience with the pace of research--and with scientists taking career bets that don't pay off. Part of the onus is on scientists for better communicating the actual implications of the work. But everyone else also has a responsibility to understand that if we're in the business of developing tools for an unpredictable future--as academics are--it is unreasonable to expect that we can fill in all the details right away, or that we're always right.

It is exciting that we live in a time when it is possible to see technical ideas go from abstract formulations to billion-dollar industries in the course of a single lifetime. It is clear we need to rethink how academia and industry should coexist under these new circumstances. Asking academics to conform to the standards of industry, however, is like asking TV writers to conform to the standards of scientists--unnecessary and stifling to creativity. I invite you to think with me about how we can do better.

With thanks to Rob Miller and Emery Berger for helping with references.

* Note that this post does not address @mhoye's main complaint about reproducibility, for which the response is that, at least in Programming Languages and Software Engineering, we recognize this can be a problem (though not as big of a problem as some may think) and have been working on it through the formation of Artifact Evaluation Committees. This post addresses the more general "what are academics even doing?!" frustration that arose from the thread.

--

Addendum: Many have pointed out that @mhoye was mainly asking for researchers to share their code. I address the specific accusation about academics not sharing code in a previous blog post. I should add that I'm all for sharing of usable code, when that's relevant to the work. In fact, I'm co-chairing the POPL 2017 Artifact Evaluation Committee for this reason. I'm also all for bridging the gaps between academia and industry. This is why I started the Cybersecurity Factory accelerator for turning commercializing security research.

What I'm responding to in this post is the deeper underlying sentiment responsible for the misperception that academics do not share their code, the sentiment that academics are not relevant. This relevance, translating roughly into "something that can be turned into a commercial idea" or "something that can be implemented in a production platform" is what I mean by "shipping code." For those who wonder if people really expect this, the answer is yes. I've been asked everything from "why work on something if it's not usable in industry in the next five years?" to "why work on something if you're not solving the problems industry has right now?"

What I'd like is for people to recognize that in order for us to take bets on the future, not all research is going to seem relevant right away--and some if might never be relevant. It's a sad state of affairs when would-be Nobel laureatees end up driving car dealership shuttles because they failed to demonstrate immediate relevance. Supporting basic science in computer science involves patience with research.

What The Bachelor Teaches Us About Choosing a PhD Advisor

This post was an experiment in simultaneous co-writing with Claire Le Goues, fellow CMU Computer Science professor, fellow undergraduate alumna of advisor Greg Morrisett from the time he was at Harvard University, and fellow reality TV enthusiast.

--

In academia, many PhD students are lucky enough to receive offers of admission from several different programs, and thus the opportunity to work with a handful of different advisors. During the decision-making process, the prospective advisors engage in Skype calls and often a recruitment open house filled with meetings and fun social events to compete for the favor of each PhD student. The end goal is the PhD student’s decision to work with the advisor.

 A couple of months is a short time to decide to be someone’s student for up to ten years. Especially when you only see each person briefly--and at their best. Especially when you’re trying to make a decision between so many people. Especially when you might not even be sure what you want. Especially when you’re tired from all the travel.

That’s right: the PhD student recruitment process is like The Bachelor.* On the popular reality show The Bachelor, a couple dozen eligible bachelorettes go on dates around the world over the course of twelve weeks to compete for the “final rose,” usually accompanied by a proposal of marriage, from the protagonist bachelor. In our world, the PhD student is the protagonist, and the potential advisers are the bachelorettes. (For those concerned about the gender asymmetry: there is also an inverse show, The Bachelorette.**)

It is not surprising that both on The Bachelor and in PhD-land, there are mistakes. On the 14th season of The Bachelor, Jake Pavelka famously chose Vienna Girardi despite the warnings of all other contestants. While Jake saw only the fun side of Vienna, the other contestants saw how self-centered and unpleasant she could really be. Many contestants were sent home for trying to warn Jake that Vienna “may not be here for the right reasons.” Similarly, on the 16th season, Ben Flajnik was so dazzled by Courtney Robertson’s beauty that he dismissed all hints that she was “different in the house than she is with you.” Viewers across America watched as Courtney charmed Ben while bullying the other contestants, telling them that she was “not here to make friends.”

It is interesting that both on The Bachelor and in academia, the multiple suitors often have access to information that the protagonist does not. The suitors all live together and interact with one another without the protagonist; professors are part of a wide social/professional network and often know one another from graduate school, conferences, and the like. In both cases, however, the person in a position of choice has plenty of reason to doubt the motivations of one suitor saying something negative about another.

Indeed, the smart suitors typically avoid saying negative things about competitors, as doing so typically results in a swift diminishment of their own prospects. Similarly, you will not hear much that is negative from your potential advisors during the PhD recruitment process. All other potential advisors are “brilliant researchers” who “do great work.” But if there’s anything we can learn from The Bachelor, it’s that it pays to pay attention to red flags. There is no such thing as an abusive advisor, only an “incompatible advising style.” The advisor who requires students to work weekends becomes “hands-on”, or someone who “drives students hard.” The advisor who only pays attention to students after they’ve proven themselves becomes “hands-off.” The advisor who barely pays any attention at all to students becomes someone who “gives students a lot of freedom to find themselves.” These ways of describing potential advisors can certainly be confusing. On the one hand, a potential advisor and their students may not have the potential student’s best interests in mind. On the other hand, potentially coded words may actually be the truth.

Fortunately, a key difference between PhD recruitment and The Bachelor is that PhD students are allowed to talk to whomever they want about potential advisors. This means they can talk to current students (which may be like talking to the family of potential bachelorettes in that they have a conflict of interest), graduated students, and former students who ended the advisor-advisee relationship before or without graduating. Though the tabloids do not cover prematurely terminated advisor-advisee relationships the way they covered the breakups of Jake and Vienna or Ben and Courtney, it is possible--and often very productive--to find out about these situations if you ask around. (We also strongly recommend asking a trusted mentor for what should be considered a red flag in a potential advising situation--there are more than we can list!)

When doing due diligence, it’s helpful to ask for concrete facts. How was Ben Flajnik supposed to know that “one way with you, but another way in the house” really meant “Courtney walks past us creepily when we’re having fun and berates us?” Besides, one person’s nightmare might be another person’s ideal. Maybe Ben wants a partner to do these kinds of activities with. (Though apparently not, since he broke things off with Courtney after watching footage of his season.) Similarly, when asking about potential advisors, it is helpful to know whether “hands-on” means “they give me guidance whenever I want it” or “they check in every day” or “they require me to work weekends whether I like it or not.” Daily checkins might be perfect for one student and terrible for another. Asking concretely about requirements and availability can go a long way in resolving possible confusion. Plus, concrete details are much more difficult to fabricate believably.

It’s also important to trust, but verify. On the most recent season, Leah lied to Ben Higgins by saying that obvious frontrunner Lauren B was different in the house than with him. But Leah was the only person who said this, Lauren’s shocked reaction when confronted was very believable, and all other available evidence suggested that Lauren was as warm and friendly with the other women in the competition as she appeared to Ben. Ben sent Leah home and ultimately proposed to Lauren. The lesson is that if you do hear something negative about a potential advisor, asking them--or their students--directly about it, as well as observing the other evidence at hand, are good ways to see if the rumor is true.

We can also learn from Bachelor history that the success of the whirlwind matching process is not sufficient to ensure a successful future. The handful of successful bachelor couples probably succeeded because they actually dated/got to know each other after the final rose and before getting married. Similarly, it is helpful for PhD students to take as much time as they can within the parameters of their program before confirming an advisor. This may mean Skyping with the advisor after the open house or rotating with the advisor’s group before deciding to join. It is also important to remember that the final rose doesn’t determine your fate forever. Just as Jason Mesnick ended up marrying his second choice from The Bachelor season 13 after the show ended, many successful researchers switched advisors at least once during their PhDs***.

Finally, remember that JoJo could, after a couple of months, reflect on her unexpected breakup with Ben Higgins with detached but understanding affection and is now going on to star on her own season of The Bachelorette. Similarly, your potential but ultimately unsuccessful advisors will generally understand---even if not in the moment---that the process works in mysterious and unpredictable ways. Just like the Bachelor alums who are now friends in real life, feel free to approach and talk to your rejected advisers at conferences and in other settings, and consider collaborating with them in the future. After all, you already know you have interests in common.

* We consider this analogy uncreepy only because we do not actually advocate professors developing romantic relationships with their students.
** We acknowledge the problematic nature of The Bachelor/The Bachelorette along lines of heteronormativity, race, gender roles. But so many lessons.
*** Our undergraduate advisor Greg Morrisett has said that when he first went to Carnegie Mellon University, he thought he was going to work with a professor who ended up going to Brown. Then he wanted to work with another professor, who ultimately had too many students at the time to take on another. He worked for a while with a professor who did not ultimately end up advising his Ph.D. thesis. Greg did not end up with his final advisor until late in his graduate career. He has since had an illustrious, award-winning research career and is now the Dean of CIS at Cornell.

What My PhD Was Like

What it looks like to do a PhD: the view from my desk.

Recently, I have been talking to many prospective Computer Science PhD students about choosing graduate institutions and advisors. In addition to excitement, there is fear and confusion. The confusion comes not only from not knowing what to expect, but not even knowing how to think about what to expect.

One prospective student put it well when she said that she knows how she is in other work situations, but she has never done a PhD before, so she does not know what it will be like. She had been curious about things like my relationship with my advisor, my research trajectory, and whether I had hobbies. She mentioned speaking to other people at various points during the PhD, but not having the whole picture much. After this conversation I decided it would be helpful for more people to talk about their PhD experiences in entirety. Now that I am a few months out of my Computer Science PhD at MIT and am about to begin an Assistant Professor position at Carnegie Mellon University, I can talk about my PhD experience in whole. This is my version of Philip Guo's The Ph.D. Grind.

Like many other academics, I will say that graduate school was one of the best periods of my life. I grew immensely, made many friends, and learned about the research process. During my PhD, I largely got to determine my own research agenda and day-to-day schedule. I developed my tastes as a researcher. I developed my tastes as a human being. Also important to me is that I developed some of the most intellectually and emotionally fulfilling friendships of my life. I loved graduate school so much that I decided to stay in academia.

But this does not give the entire picture. "Loving graduate school" is different from loving a good meal, a thought-provoking film, a pet, or another person. A PhD is about training you to become a researcher. The further you are from being in an experimental field, the further this growth is about concrete skills and the more it is about existential questions of how you want to be. If you want to stay in academia, you must also balance the growth with ensuring you have enough to show at the end for the intellectual coming-out party that is the academic job market. This growth is painful and anxiogenic. Even if you are in a supportive environment and have great work-life balance, a PhD is not supposed to be a comfortable ride. The flip side is that, as my friend Alex describes it, during your PhD every day is "epic."

Enough philosophizing. Here, concretely, is what my PhD was like.

Summer 2008. At the end of the summer, I move into my office at MIT, where I am to be co-advised by Mike Ernst and Saman Amarasinghe. Saman pays for me to start early. I spend a couple of weeks reading papers, mostly about parallelism, and report to him every morning about what I've read. He gives me lots of advice about what life is going to be like. Mike Ernst has been in Europe this whole time. The last week of the summer, Mike Ernst calls me and tells me he is moving to the University of Washington.

2008-2009 (Year One). I'm recovering from a repetitive stress injury in my neck, so Saman advises me to take hard theory courses that involve little computer use. I take Advanced Algorithms and Machine Learning in the fall and Advanced Complexity Theory in the spring.

I get involved with and become lead author on a PLDI submission with Saman and Martin Rinard about a programming language for trading off power and accuracy. This paper is pretty good for having come into existence over the course of 1.5 months, but not objectively a great paper--and it gets rejected.

Ras Bodik, a professor at Berkeley who tried to recruit me, emails me telling me his student Armando Solar-Lezama is starting a professorship at MIT. I meet Armando and he sends me many papers to read. Soon we are meeting and working so much I no longer have time for the other project, which was a suboptimal fit anyway for various reasons. I like how much bandwidth Armando has for technical discussions, especially since he has no other students or teaching duties at the time. I am nervous that this high meeting frequency is unsustainable, so I try hard to recruit other students to Armando's group.

Armando and I submit a paper to OOPSLA but it is not good because we don't start writing it until three days before. I also submit a paper to the Haskell Workshop about my senior undergraduate thesis, but it is not good because I don't know how to write papers. My friend Jean-Baptiste who reads it asks why my blog is so good but my paper drafts are so bad.

Since I am still figuring out what I want to do, I schedule many meetings with professors and ask them how to do research. In addition to Saman's group meeting I also attend Daniel Jackson's group meeting. I go to many PhD defense and job talks. Both Martin and Saman tell me that in order to become a good researcher, I have to flounder and fail. I have never failed before. I take it as a challenge to do work risky enough to fail.

January of this year I attend my first conference, POPL. Martin pays for me to go after I email him asking how I can get to go to POPL. I meet many people, including many people at Microsoft Research. This eventually leads to my first internship.

Outside of work, I try to avoid using my computer due to my repetitive stress injury. Because of the injury, I am also aggressive about working 8-hour days and taking breaks on weekends. I am too lazy to make many friends so I spend my free time reading, cooking, doing yoga, and running.

Summer 2009. I intern at Microsoft Research in Redmond with Chris Hawblitzel. I had seen Chris's talk at POPL about verifying garbage collectors using Boogie and thought it was one of the coolest projects I had ever seen. I had not thought I was qualified to intern with him, but some of the people I met at POPL recommended me to him. This summer, we build Verve, an operating system verified for type and memory safety. I had never verified anything or built an operating system before. I work hard, 10-12 hours a day, because there is so much to learn and because the project is so fun. I work closely with Chris and he teaches me an incredible amount about working with low-level code. Outside of work, I hang out with the other interns. I live in Redmond but on weekends I go into the city, where I have two friends I stay with.

2009-2010 (Year Two). I continue to work with Armando, on the project that eventually becomes the Jeeves programming language. My first year we started with a vague idea that dynamic constraint-solving was useful for handling corner cases that static program synthesis could not handle well. We apply this to data imputation. We learn over the course of 1.5 years of paper submissions that people don't care much about data imputation, especially since our results are not particularly good. By the end of my second year, we start thinking that security might be a better application domain. This cumulative work is my masters thesis. We still have many things to figure out at this point so I am not that excited about my thesis. I try to ask for an extension but Armando tells me to just turn it in.

In the spring, Saman and I start what becomes an annual off-site retreat of all the research groups related to programming languages at MIT. People give research talks and we have philosophical discussions about the future of research, publishing, and education. At the first one, we realize there are many more people who have common interests than previously thought. After the first meeting, the professors meet together and decide to do more activities together from now on. From this point forward there is more cohesion and activities like weekly PL-related meetings.

At the beginning of this year, I start telling people that I wish I had more female friends, since I realize that there are many fewer women around me than before. My friend Neha Narula tells me to respond to a call for people interested in starting a graduate women's group and I meet Kay Furman, a PhD student in the Health Science Technologies program. Along with a couple of other graduate women we start Graduate Women at MIT. Such a group had never existed and our proposals are met with a fair amount of skepticism, more from students than from administrators. At this point I still don't have many friends, so I spent many evenings and weekends working on building the organization: recruiting, fundraising, inviting speakers, writing materials about why we need such a group, etc.

As part of my acquisition of more female friends I join a feminist discussion group, also invited by Neha. We get together about once a month for a dinner hosted in one of our homes. These discussions and my friendships with the women in the group shape many of my views over the next few years.

During this time, I look into acquiring more hobbies. I take some break-dancing classes at the Dance Complex but decide it's not my thing. I take up acrobatics (silks, rope, and static trapeze), for which I continue weekly classes on and off for about a year until I realize I am not serious enough about it (and travel far too much) to make continuing worthwhile.

Summer 2010. I intern with Nikhil Swamy and Juan Chen at Microsoft Research Redmond. They have a dependently typed language called Fine for verifying security properties. I ask to intern with them because I want to take my work in this direction. I spend most of my time working on a tool for stuffing proofs into bytecode and taking them back out. I learn a lot, but the proofs are too big and we never quite manage to get it to work. I also participate in many meetings and calls about the F* language, which is like Fine but supports proof erasure. I learn a lot about dependent type systems and security concerns.

I live in Seattle this summer, in a studio apartment on Capitol Hill. For traffic and productivity reasons, I take the 7am bus to work and stay at work until 7pm. There is a heated vinyasa yoga studio I go to after work most days. I love having a routine and Seattle in the summer and the energy of Capitol Hill at night, so I am very happy with my life. This summer, thanks to Chris's nomination, I also participate in the Lindau Meeting of Nobel Laureates, where Nobel laureates give talks about science, the research process, and the future of the world. I make friends with scientists from around the world and find the talks inspiring.

Chris and I have a paper on Verve that appears at PLDI. We win Best Paper Award. I am too young to realize that not all projects go this smoothly and not all papers are this good, so I am confused. Since this is the first paper I ever published, this becomes my new standard for paper goodness. It takes me years to realize that it is possible to do much less and still publish a paper.

2010-2011 (Year Three). Now I am done with classes, have no more routine, and have to return to working on my project I am not that excited about, so I am less happy than before. I have this OCaml interpreter implementation for proto-Jeeves that I have been working on. To make things scale, I implement a C backend and a garbage collector. It integrates with the Yices constraints solver. It is hard to do anything with this interpreter because it requires a lot of work to experiment with language features. I do not feel much hope about the project. I shop around for other projects. I consider doing an internship in hardware synthesis with David Bacon at IBM Research, but after talking to some people decide I should not switch to this area.

In the fall, I TA a new program analysis course taught by Arvind, Martin, and Armando. The professors are excited for me to help revamp the course. There are four course staff total and six registered students. Most of the work involves doing the problem sets to make sure they work. It takes a surprisingly amount of time to grade six assignments, since each assignment is likely to make different design decisions. I hold weekly recitation where three people attend. All three students are strong, so most of the time we talk about fun PL topics.

To take a break after going straight from turning in my masters thesis to doing a internship, I treat myself by taking a two-week vacation. I take a road trip through Romania (and also parts of Serbia and Hungary) with my friend Florian who I met at some conferences. My friend Kate, who I met through the feminist discussion group and who has moved to Europe for an Erasmus masters program, meets us in Vienna, where Florian is studying.

In February I attend my first workshop at Dagstuhl, a German castle that hosts focused computer science workshops. I have involved discussions many computer scientists working in similar areas. I find it excitiyng. Afterward I meet my friend Kate for a weekend in Brussels and we have adventures as usual.

This year, I also start to make more friends. I become good friends with my "cohort," the other students interested in programming languages and software engineering who I started with. I also become friends with many female scientists. I never had many female scientist friends before and this makes me happy because I feel much more understood.

Summer 2011. I decide to stay around MIT to focus on research. My groupmate and officemate Kuat Yessenov tells me it would be simple to implement my language as an embedded domain-specific language in Scala. Kuat makes a prototype implementation and begins to collaborate with me on the project. This changes everything. All of the sudden, the language has hope of becoming practical. A nice story about Jeeves emerges: it is a convenient and practical programming model that lets you factor out information flow policies. It seems natural and like the way to go--but formalization is not entirely straightforward and there are some interesting theoretical properties too. It is great to collaborate with Kuat, who is very smart. We submit a paper to POPL. We work hard, staying long nights in the office. I briefly become nocturnal in response to receiving feedback from my advisor in the evening and feeling compelled to make the changes before I sleep.

It rains a lot this summer and we both become sad. One day, Kuat leaves a fortune cookie fortune on my desk that says "It cannot rain every day."

2011-2012 (Year Four). Armando starts putting pressure on Kuat to return to his project so Kuat stops collaborating with me. I am now collaboratorless and still haven't published anything on this language, now called Jeeves, so I become more and more hopeless. A couple of my friends, Marek Olszewski and Michal Depa, leave school to do startups. Marek invites me to join him. I'm not ready yet, but I have significant FOMO (Fear Of Missing Out). I take a class about startups and become friends with Tiffany Kosolcharoen, then a student at Sloan. Tiffany and I spend hours together every week and she teaches me an immense amount about the startup world.

My friends begin to worry I will quit graduate school. My groupmate Rishabh Singh stops teasing me about my inferiority for the first time, looks at me seriously, and says, "Miss Yang! You cannot quit. You are an intellectual." Someone leaves a fortune cookie fortune on my desk that says "It does not take guts to quit." Sasa Misailovic, another one of my cohort, has an intervention lunch with me telling me that I like graduate school the most out of all of us, so if I quit then what does it mean for everyone else. I tell him everything is meaningless.

Our paper gets into POPL and everyone is relieved. I am relieved to confirm there is life after Verve. Armando is relieved for obvious reasons. My friends are relieved because they stop thinking I will quit graduate school.

During the spring I build a small conference management system using Jeeves, not because I want to but because Armando says it is important to build things to see what breaks. This takes a lot out of me because Jeeves doesn't really work yet and I previously knew little about building web applications. We run out of memory all the time and I have to wake up in the middle of the night to restart the server. For weeks after, my friends make fun of me by sending emails telling me my server is down. For weeks, Armando and I wake up with nightmares in the middle of the night that our server is down. During this process I also realize that in order for language-based solutions to work, we have to have a better story with interfacing with the database. I feel proud that I convince Armando this is an important direction of future work. I also concede that it is important to build things.

This spring I do some traveling. I win a Facebook Fellowship and travel to the Bay Area to give talks at Facebook, Google, and Berkeley. I attend another workshop at Dagstuhl and spend the weekend in Paris beforehand and meet my friend Kate in Barcelona after.

My social life becomes fantastic this year. I find the "party circuit" orbiting a nucleus of social Harvard and MIT graduate students. I meet many interesting and brilliant people.

Summer 2012. I intern at Facebook to gain "privacy street cred." I start in July because it's after the POPL deadline. The other interns starting at the same time are all Eastern European college boys. This becomes my friend group for the summer. I live in the city and commute to Menlo Park every day, staying on campus for 10-12 hours. I work hard, building a verifier for privacy policies using Haskell. I like the work but find it isolating because I'm the only person working on my project and it's not clear what I am allowed to talk about outside of work. I take daily walks with my friend Pieter Hooimeijer, who I knew from research and went to Facebook after his PhD. I work harder than I want to because of the potential high impact of this project: it is tied to a pre-launch project and if it launches well, this could be a really nice example of verification in production use at a large company. Armando flies out to Facebook at the end of the summer to discuss my project under NDA. I am so happy to see him.

2012-2013 (Year Five). I return to school but continue contracting for Facebook. I work on my verifier one day a week and fly out to the Bay Area every few weeks for a few days at a time to continue working on my project. I maintain two social lives, one in Cambridge and one in San Francisco. The bigger project I'm a part of gets cancelled around February, for political and not technical reasons. I feel very disappointed, especially because my verifier had been very close to being put into production. I also feel relieved, because I'm exhausted from having two jobs and two social lives. I make some small efforts to try to publish on what I built, but Armando is not excited and the team at Facebook has scattered. We file a patent.

The rest of my research is not satisfying this whole year. I work on extensions to Jeeves that Armando tells me are necessary for the work, but that I am not excited about. We work on integrity extensions to Jeeves that I don't think make sense. What I really want to do is figure out how to extend the Jeeves programming model to the database, but I can't work on this until after we finish this other stuff. I submit an uninspired paper to CAV. One of the reviews says, "I cannot believe there are four authors on this paper and so many typos." I am not excited about anything by the end of this year.

In the spring, I take a class in the Media Lab with Ethan Zuckerman about new media. Half the students are technologists and half are mid-career journalists at Harvard and MIT on various fellowships. This gets me thinking much more about media, online communication, and civic participation.

Summer 2013. I stay around school to work on the integrity story for Jeeves, which I continue to not believe in. Armando tells me it is time for me to graduate and go on the academic job market. I tell him the academic job market is a big commitment because 1) I don't know that working alone forever is the best way to achieve the things I want to achieve and 2) the work/life balance doesn't seem great. He says 1) is a result of how he chose to run his group, but if I don't want to put in the work of being a professor then I should wrap up my work as quickly as possible and move on with my life. We are both sad.

2013-2014 (Year Six). I spend a month having in-person and Skype meetings with people about other career paths, for instance working at LogicBlox or Jane Street Capital. After my mother reminds me I don't have to do computer science anymore, I consider becoming a writer or an artisan specializing in linocut block prints. Rocky, who works on my floor, tells me that usually everyone on the seventh floor can hear when I laugh, but he has not heard me laugh for six months.

During this time, I also visit Andrey Rybalchenko and Byron Cook at Microsoft Research Cambridge. I enjoy myself immensely and have many stimulating meetings and become exposed to the field of executable biology. (Later I find out this was a secret interview.) I determine that nothing else is as exciting to me and even though academia is a big commitment it seems worth it. I tell Armando I'm ready to go on the job market and he says he already told everyone I wasn't anymore and plus it's better for me to wait. I am more than fine with this, since a big issue had been that I don't want to leave yet.

After I determine that one of my main issues with my project is that nobody uses Scala for prototyping web apps, I decide to switch to Python. Armando lets me take on an MEng (masters) student to convert our implementation to Python. I find Travis Hance. He finishes the conversion in the fall and we spend all spring collaborating on building a web framework based on Jeeves. I am really excited to finally be thinking about the problem of extending the programming model to the database, especially as this is something I had wanted to do for two years. I love having Travis as a collaborator. We decide to submit to OSDI. During the weeks before OSDI, we work together for 10-12 hours a day. I introduce him to many Kendall Square restaurants. During our breaks, we create Haskell Ryan Gosling.

During this time I also decide to take my interest in writing more seriously. The previous year I had come across a former problem set partner Carl Schoonover on Business Insider's list of sexiest scientists in the world and discovered he had been achieving my science writing dreams: writing a book, speaking about science, and even starting a group called NeuWrite to promote collaborations between scientists and science writers. I had brought him to MIT to speak, which led to two things: 1) me assembling an RSVP list of Harvard and MIT graduate students interested in science writing and 2) Carl deciding I was a good person to take over the NeuWrite Boston group, as group membership was shrinking and the current leaders were moving away. One of the journalists from my new media class introduces me to science writer Amanda Gefter and we take over the group. We build a new group from the event RSVP list and Amanda's writer friends. I spend much of my free time 2014-2015 building the group, thinking about writing, and writing. I start publishing not only on my own blogs but also in more public venues. A couple of my Quora answers get published on Forbes and my pieces for The Muse get chosen to run in Forbes and Newsweek.

Summer 2014. PLDI is in Edinburgh at the beginning of June and my friend is getting married in Croatia at the end of June. Since Armando is paying for the trans-Atlantic flights, I scheme to remain in Europe for the month. I set up another visit to Microsoft Research Cambridge and a visit to the Max Planck Institute in Kaiserslautern, Germany. I get good feedback on my work during both visits. I then spend a long weekend in Berlin visiting my friend Kate and a week wandering around various parts of Croatia for vacation purposes. I see Oslo for a day on the way back. I find the entire trip relaxing but find it difficult to return to normal life after.

Our OSDI submission gets rejected, so I spend the rest of the summer cleaning up the paper and doing more work to submit to PLDI. I feel an incredible amount of angst about my uncertain future as I prepare for the job market. I spend one day a week doing extreme exercise, cycling 50+ miles or running 11+ miles, to obliterate all thoughts. For various reasons I also spend a lot of weekends in New York and make more friends with people working on data and journalism.

2014-2015 (Year Seven). I spend most of the fall preparing a submission about the Jeeves-based web framework to PLDI. I also work on my job applications. I go on the job market, interview, etc. I get some offers, accept a position at Carnegie Mellon University, and defer a year to collect myself before beginning and also to explore my interest in applying programming languages and biological modeling.

To keep myself from thinking about my uncertain future all the time, I start a cybersecurity accelerator, Cybersecurity Factory, with my friend Frank Wang, with the goal of helping research-minded people start companies. After Frank and I ran out of research things to talk about the natural next step was to develop a project of mutual interest and we're both interested pushing academic research into the world.

I don't have much of a social life this year. For a couple of my interviews I stay with friends beforehand and they listen to my talks. My friends Alison and Tobi make sure to force me out of my house for non-work reasons every once in a while. Every few weeks my friend Angela comes over and cooks me dinner. I remain grateful for my friends.

Summer 2015. I spend the first part of the summer working on writing up the Jeeves-based web framework work, which has continued to be rejected from every conference. I get the framework to be much faster and work with my collaborators on a cleaner semantics. We submit to POPL and then I work on my thesis. The introduction and related work take much longer than expected, maybe about a month.

During this time I spend maybe about a third of my time working on the accelerator and doing public communication about cybersecurity. We accepted two teams for our Cybersecurity Factory pilot program and we work closely with them to make sure they are getting the support they need. I spend a good amount of time on the phone with media people getting press for our program. I write an op-ed for the MIT Tech Review.

Somewhere in there I take a 36-hour trip to Singapore to give a talk at a data privacy workshop Saman is organizing. Saman gets the people to fly me business class so I can sleep in transit. It is my first time flying business class. I am confused about what to do with such frequent deliveries of hot towels to my seat.

I don't have much of a social life during this time either. Several people text me "you have to eat sometime" after I repeatedly decline meal invitations.

2015-2016. I defend! I take two weeks off and then start a postdoc at Harvard Medical School. I slowly ease myself back into having hobbies and seeing friends. Our POPL submission gets rejected and I finally spend a lot of time thinking about how to sell the work (rather than how to do the work). I learn that despite pressures to make the work better until the last minute, I need to fight to spend a long time on the writing, especially if the proposed idea is outside of the mainstream. A nice story emerges about a new and practical web programming model that allows programmers to factor out information flow policies across the multiple runtimes of the application and database. Our paper finally gets accepted at PLDI. I feel relieved to publish the final part of my thesis. And here we are now.

For those wondering what to take away, it is important to note that every PhD experience is unique influenced by factors such as the advisor relationship, institutional requirements, the funding situation (both professional and personal), and the academic and social environments. I am privileged to have had a relatively supportive environment and few additional pressures. My experience is most likely to generalize for other computer science PhD students at top schools, where the funding situation tends to be good and advisors tend to give students a fair amount of freedom.

You may also be interested in other posts I've written about my PhD:

• On Quora: Do Ph.D. students get time to pursue their hobbies?
• Reasons to Pursue a Ph.D.
• The Life of an Academic, Explained
• How Science Really Works

--

Addenda:

• You may be interested in reading the comment thread for this post on Hacker News.
• My friend and former groupmate Nadia pointed out that during 2014-2015, I claim to have no social life "but we went to so many cool parties." It's true. Over the course of the year I did go to some cool parties.
• My friend Michael, who I once dated, said I should mention the existence of romantic relationships during the PhD to provide a more complete picture of my life. Chris Martens also says in her blog post about her PhD that she is frustrated when people don't discuss the interplay between romantic partnerships and work. Over the course of my PhD I was in a few committed romantic relationships, mostly with other academics, mostly in other fields. Some of the time I was in relationships coincided with periods of extreme work. This was difficult to balance, but other academics tend to understand. I was never in a relationship where someone else was routinely doing any portion of the domestic labor. I was also single for a significant portion of my PhD.
• Berkeley PhD student Stephen Tu has begun compiling a list of PhD reflection here.