Why We Need to Talk About the Collaboration Problem

Today I spoke with a Computer Science professor who is finishing a semester of teaching a notoriously challenging advanced undergraduate course.

"I figured out the problem with my female students," he told me. "It's their partners."

All semester, this colleague--let's call him Albus Dumbledore--had been telling me about the strange phenomenon of drama with his female students and their project partners. The course has a significant project component, and successful completion of the project usually depended on both partners pulling their weight. Mediating partner disputes became the responsibility of the instructor. And what the instructor noticed was that an alarming fraction of the disputes seemed to happen when one of the partners was female.

After wondering all semester how bias might contribute to the drama of the female students' partners, Albus had a relevation. The female students complaining about their partners all seemed to have better overall grades than their partners. Not only did the partners have lower GPAs, but many of them were from outside of Computer Science. Albus surmised that these partners were, in fact, probably not pulling their weight, and that the students had every right to complain.

"But why would these strong students choose such bad partners?" he asked.

That female students had bad partners was, to me, not surprising. After all, nobody had asked me to work on any problem set until the second semester of my sophomore year, and a fellow student only asked me after obtaining an unprotected copy of course grades on our department servers and discovering I had the second-highest midterm score in one of our courses. I told Albus about how a friend once confessed to me that before she had gotten to know me, she had forbade her boyfriend from working with me. I told him about how problem set partners often preferred to solve problems for me rather than with me. My best collaboration in college had been with another woman, and she had been so initially skeptical of my abilities that it took me at least half of a semester to win her over with how fast and how correct my code was.

"So it's not by choice," Albus concluded. "What can we do about this?"

Important question. For my first few years of college, the collaboration problem had left me feeling so isolated and so much in doubt of my abilities that I often thought about switching away from Computer Science. If not for a chance encounter with a friend, one year behind me and facing similar problems, I might have left. What began as a quick hello as our paths intersected on the way back from class turned into a long discussion about the difficulties we both had in finding people who would collaborate with us. I had graded this woman's homework in multiple classes, so I knew the problem was not that she was not capable. This was when I began to realize that the problem may not be with me, but with the way people perceived me--and other women.

Years later, when I was starting Graduate Women at MIT, this conversation led me to put together a panel on collaboration--specifically, on collaborating as women in male-dominated fields. I felt so validated when the panelists--three women at various stages in their careers, each at the top of her field--said what I had observed for years, but had never dared to say out loud. It can be hard to collaborate with men, one panelist said: they often talk at you rather than to you because they are socialized to impress women. It can be harder to collaborate with two men, another panelist said: they will often talk only to each other while trying to impress you. (I don't like to make blanket statements about all people of a gender, just like I don't like to make blanket statements of all people from a culture, but these kinds of conversations can be helpful for recognizing patterns.) While much of this advice was unsurprising, and also depressing, it felt incredibly powerful to hear someone else say these statements out loud. Talking about this explicitly seemed like the first step towards solving the problem.

In the intervening years, I've collected much more evidence of the problem than I have solutions. It is undeniable that collaborations account for much of people's success in technical settings. Albus talked about how, in his class, the students with subpar partners struggled to complete their projects. A recent study I read* cited female academics' ability to travel for international collaboration as one of the biggest determinants of their success. Yet collaboration seems to remain a problem. At a recent lunch of Women@SCS in my department, I spoke about my experiences with Graduate Women at MIT, including about the collaboration panel, and the student kept returning to the issue of collaborating in a male-dominated field. Students asked about how to find collaborators who would take them seriously. Students asked about what to do in groups when people may not be listening to them. A student asked what to do if she has had so many negative collaboration experiences she is reluctant to collaborate anymore. A student said that she, too, felt like male collaborators were often trying to impress her rather than work with her, but she had thought it was in her head.

After the recent lunch, a student asked me about the benefit of talking explicitly about these issues. Wouldn't it be better, she asked, to not draw attention to gender and wait for the problems to go away? I, too, would love to live in a post-gender world where people can just be people. Unfortunately, it seems that collaboration is a topic we need to address explicitly. Not only do these cross-gender/culture problems not seem to be going away on their own, but they also seem to be increasing certain inequalities. Especially in Computer Science, smart people have done an excellent job of solving many other problems of gender equality. I have full confidence that once we recognize this as a problem, we can find good solutions. I would love to hear your ideas.

* In the process of looking for this citation... Let me know if you have it!

Why It's Not Academia's Job to Produce Code That Ships

Note: The images in this post are inexplicably broken due to some kind of Blogger bug. If someone is reading this at Google, please help!

My scientist friends often scoff at crime show writers' creative interpretation of technology's limits.

The technology shiny world of CSI: Cyber.

"Let's zoom in here," a character says in an investigation room with floor-to-ceiling screens showing high-definition maps of the show's major metropolitan area. A flick of the fingers reveals an image of the suspect at mouth-watering resolution.

In another scene, the characters listen to a voice mail from the suspect. "What's that in the background?" one investigator asks. Using an interface that deadmau5 would kill to have, the hacker of the bunch strips out the the talking, the other sounds. They say some words like "triangulation" and, eureka, they deduce the suspect's exact location.

Yes, real police technology is nowhere near this sophisticated. Yes, nobody (except maybe the government, secretly) has technology like this. But those who criticize the lack of realism are missing the point.

The realities that art constructs take us out of our existing frames of perception--not only for fun, but also for profit. Many important technological advances, from the submarine from the cell phone, appeared in fiction well before they appeared in real life. Correlation does not imply causation, but many dare say that fiction inspires science.

Some complaints against academic Computer Science.

This brings us to the relationship between academic Computer Science and the tech industry. Recently, people in industry have made similar criticisms of academic computer science. Mike Hoye of Mozilla started the conversation by saying he was "extremely angry" with academics for making it difficult for industry to access the research results. This unleashed a stream of Internet frustration against academics about everything from lack of Open Access (not our faults) to squandering government funding (not entirely true) to not caring about reproducibility or sharing our code (addressed in an earlier blog post).

At the heart of the frustration is a legitimate accusation*: that academics care more about producing papers than about producing anything immediately (or close to immediately) useful for the real world. I have been hearing some variation of this criticism, from academics as well as industry people, for longer than I have been doing research. But these criticisms are equivalent to saying that TV writers care more about making a good show than being technically realistic. While both are correct observations, they should not be complaints. The real problem here is not that academics don't care about relevance or that industry does not care about principles, but that there is a mismatch in expectations.

It makes sense that people expect academic research results to work in companies right away. Research that makes tangible, measurable contributions is often what ends up being most popular with funding sources (including industry), media outlets, and other academics reviewing papers, faculty applications, and promotion cases. As a result, academic researchers are increasingly under pressure to do research that can be described as "realistic" and "practical," to explicitly make connections between academic work and the real, practical work that goes on in industry.

In reality, most research--and much of the research worth doing--is far from being immediately practical. For very applied research, the connections are natural and the claims of practicality may be a summer internship or startup away from being true. Everything else is a career bet. Academics bet years, sometimes the entirety, of their careers on visions of what the world will be like in five, ten, twenty years. Many, many academics spend many years doing what others consider "irrelevant," "crazy," or "impossible" so that the ideas are ready by the time the time the other factors--physical hardware, society--are in place.

The paths to becoming billion-dollar industries.

In Computer Science, it is especially easy to forget that longer-term research is important when we can already do so much with existing ideas. But even if we look at what ends up making  money, evidence shows that career bets are responsible for much of the technology we have today. The book Innovation in Information Technology talks about how ideas in computer science turned into billion-dollar ideas. A graphic from the book (on right) shows that the Internet started as a university project in the sixties. Another graphic shows there were similarly long tech transfer trajectories for ideas such as relational databases, the World Wide Web, speech recognition, and broadband in the last mile.

The story of slow transfer is true across Computer Science. People often ask me why I do research in programming languages if most of the mainstream programming languages were created by regular programmers. It we look closely, however, most of the features in mainstream languages came out of decades of research. Yes, Guido Van Rossum was a programmer and not a researcher before he became the Benevolent Dictator of Python. But Python's contribution is not in innovating in terms of any particular paradigm, but in combining well features like object orientation (Smalltalk, 1972, and Clu, 1975), anonymous lambda functions (the lambda calculus, 1937), and garbage collection (1959) with an interactive feel (1960s). As programming languages researchers, we're looking at what's next: how to address problems now that people without formal training are programming, now that we have all these security and privacy concerns. In a media interview about my Jeeves language for automatically enforcing security and privacy policies, I explained the purpose of creating research languages as follows: "We’re taking a crazy idea, showing that it can work at all, and then fleshing it out so that it can work in the real world."

Some may believe that all of the deep, difficult work has already been done in Computer Science--and now we should simply capitalize on the efforts of researchers past. History has shown that progress has always gone beyond people's imaginations. Henry Leavitt Ellsworth, the first Commissioner of the US Patent Office, is known to have made fun of the notion that progress is ending, saying, "The advancement of the arts, from year to year, taxes our credulity and seems to presage the arrival of that period when human improvement must end." And common sense tell us otherwise. All of our data is becoming digitized and we have no clue how to make sure we're not leaking too much information. We're using software to design drugs and diagnose illness without really understanding what the software is doing. To say we have finished making progress is to be satisfied with an unsatisfying status quo.

The challenge, then, is not to get academics to be more relevant, but to preserve the separate roles of industry and academia while promoting transfer of ideas. As academics, we can do better in communicating the expectations of academic research (an outreach problem) and developing more concrete standards of expectations for "practical" research (something that Artifact Evaluation Committees have been doing, but that could benefit from more input from industry). As a society, we also need to work towards having more patience with the pace of research--and with scientists taking career bets that don't pay off. Part of the onus is on scientists for better communicating the actual implications of the work. But everyone else also has a responsibility to understand that if we're in the business of developing tools for an unpredictable future--as academics are--it is unreasonable to expect that we can fill in all the details right away, or that we're always right.

It is exciting that we live in a time when it is possible to see technical ideas go from abstract formulations to billion-dollar industries in the course of a single lifetime. It is clear we need to rethink how academia and industry should coexist under these new circumstances. Asking academics to conform to the standards of industry, however, is like asking TV writers to conform to the standards of scientists--unnecessary and stifling to creativity. I invite you to think with me about how we can do better.

With thanks to Rob Miller and Emery Berger for helping with references.

* Note that this post does not address @mhoye's main complaint about reproducibility, for which the response is that, at least in Programming Languages and Software Engineering, we recognize this can be a problem (though not as big of a problem as some may think) and have been working on it through the formation of Artifact Evaluation Committees. This post addresses the more general "what are academics even doing?!" frustration that arose from the thread.

--

Addendum: Many have pointed out that @mhoye was mainly asking for researchers to share their code. I address the specific accusation about academics not sharing code in a previous blog post. I should add that I'm all for sharing of usable code, when that's relevant to the work. In fact, I'm co-chairing the POPL 2017 Artifact Evaluation Committee for this reason. I'm also all for bridging the gaps between academia and industry. This is why I started the Cybersecurity Factory accelerator for turning commercializing security research.

What I'm responding to in this post is the deeper underlying sentiment responsible for the misperception that academics do not share their code, the sentiment that academics are not relevant. This relevance, translating roughly into "something that can be turned into a commercial idea" or "something that can be implemented in a production platform" is what I mean by "shipping code." For those who wonder if people really expect this, the answer is yes. I've been asked everything from "why work on something if it's not usable in industry in the next five years?" to "why work on something if you're not solving the problems industry has right now?"

What I'd like is for people to recognize that in order for us to take bets on the future, not all research is going to seem relevant right away--and some if might never be relevant. It's a sad state of affairs when would-be Nobel laureatees end up driving car dealership shuttles because they failed to demonstrate immediate relevance. Supporting basic science in computer science involves patience with research.

I'm Using Python Now

I have a confession. All these years I've been evangelizing strongly statically typed languages, I've been going home at night and using Python. Not all the time. It's more like the hungry vegetarian graduate student who comes to the free lunch and discovers only meat dishes. Sometimes you have to do what you have to do.

It started with small things here and there. Because of its libraries, Python has been my go-to language for web scraping jobs. Because of its relative concision to Java and its relative principles to PHP, Python has become my go-to language for web backend programming. And as the web is getting fancier and fancier, I've been doing more of this. As the web has been getting fancier, my research has also involved more of this. Web backend programming. And also Python.

It's true. I'm now doing research programming in Python*. For my PhD, I've been developing Jeeves, a new programming language (and soon-to-be web framework) for automatically enforcing privacy policies. Jeeves makes the programmer's life easier by making the language runtime responsible for keeping track of the privacy policies policies. As this approach happens completely at run-time rather than compile-time right now, it is a great fit for embedding into Python, which does most things dynamically (at run time). And so we switched from the Scala implementation we had been maintaining for a couple of years to Python.

The initial reason for switching to Python was that it is my favorite popular language for web backends. Popularity matters: a web framework is a big piece of software. Lots of people using it usually corresponds to lots of people developing and maintaining it, as well as more documentation for how to use it. I had found both of these to be issues with Scala web frameworks: the two popular Scala web frameworks, Lift and Scalatra, have learning curves that are quite steep. The people-power behind development, maintenance, and documentation also means that it is likely that lots of people will continue using it.

Another reason for the switch was that Python is what the (MIT undergraduate) kids are learning these days. I had thought that I could push Scala upon undergraduate research assistants using the force of my charisma, but this is harder than it might sound. It can take upwards of half a semester to teach even a motivated, bright undergraduate Scala. And given that undergraduates tend to be around for about a semester, that does not leave much productive work time.

After switching to Python, I learned that playing with language features in Python is much faster than in Scala. Much of the time spent coding in the Scala implementation involved an intricate dance with Scala's type system. While it's thrilling never knowing if your new tricks will allow you to save yourself from writing type-casing boilerplate, it takes a lot of time to convince the Scala type-checker that you're doing something reasonable in all cases. Sure, people might object that writing in Python is like swimming without a life jacket or something, but if you're an experienced swimmer trying out different moves in a small body of water, a life jacket is just going to hinder you. Same effect for experienced programmers prototyping a research language...

Something else that I've discovered is that people outside of programming languages seem much more excited about using my language when I tell them it is embedded in Python. These people include computational biologists, computer scientists working in areas other than programming languages (systems; the web), and undergraduates who are trying to work on our research project. A computational biochemist I spoke to who has been using logic programming was quite excited that we were embedding our language in Python. There are already many biological modelling toolkits written in Python, he said, so he could easily envision people picking up our tool.

I'm not saying everyone should use Python instead of Scala. I still stand by everything I say in my other blog post praising Scala: Scala is a less pretty version of the ML family of languages that is potentially way more usable because of its interoperability with Java. If you're trying to do quick-and-dirty language prototyping or trying to build a web framework, however, Scala may not be the tool for the job.

I know I'm not going to be able to check my Python programs before I run them. And I know I'm never going to be able to run my programs super fast. But I've looked into the tradeoff space and made my choice. Good thing Python is for girls.

* In the past few decades, the fashion has been to conduct programming languages research using statically typed languages with strong type systems. The idea is that since we have the compute power to check our programs before our run them, it is irresponsible not to. Thus using something like Python is considered somewhat scandalous.

What is Computer Science?

Every now and then, a non-technical friend--or stranger at a party--will ask me, "What is computer science?" As your typical antisocial computer scientist, I am often too tired to do justice to this question. From now on, I will carry around QR codes pointing to this post.

Computer science is the study of what machines can do for us. Among computer scientists there are theorists, applied logicians and mathematicians study what is even possible for a machine to compute. There are systems scientists who study how to get the power we want out of our machines. Then there are artificial intelligence researchers who help computers make unsupervised decisions--they are the ones responsible for the robots and the cool demos. Computer science researchers take the risk out of building systems; software engineers build the things we use.

At the core of computer science is a pile of logical foundations. People like to study how hard problems are: whether you can compute solutions in steps ("time") that are a polynomial function of size of the input, an exponential function, or something even worse. People talk about different classes of problems: for instance, P, the class of problems with polynomial-time algorithms, and NP, a class of problems for which polynomial-time algorithms are not known. It is not known whether P and NP are the same, but it would change everything if we discovered they were: all of our banking software is encrypted based on the assumption that P != NP (P not equal to NP). (A couple of years ago there was big news when somebody claimed to have proven P = NP.  Fortunately, it turned out to be a false alarm.) People across subfields talk about what is decidable: what it is possible to develop a terminating computational routine to do. This allows us to guarantee (haha!) that the software powering your space shuttle will not spontaneously freeze from getting stuck in an infinite loop.

Then there are the machines themselves. Incredible things. The invention of the transistor allowed for everything by making high (1) and low (0) signals, called bits, stable enough to build everything else on top. Computer hardware runs on bits combined into instructions and a set of logic gates (for instance: "and," "or," "not") for decoding these instructions. Instructions encode a unit of computation: for instance "add" or "store in memory" along with operands, either as constants or in as locations in memory. Memory just contains more of these bits (just 1's and 0's!). All a computer is doing, really, is reading instructions from code memory and reading to and writing from data memory. There are also details like interacting with a keyboard and a screen. One of the key ideas in systems is abstraction: building higher-level, simpler systems on top of existing ones to simplify reasoning. Programming languages shield the programmer from having to reason about bits and operating systems shield the programmer from reasoning about details like the interaction with the keyboard and in what order application processes should run. On top of all this we have the mobile app that let you read this post. Underneath, it's just bits.

And there is also the sexy artificial intelligence stuff. Back in the day, people thought they might create machine intelligence by modeling how the mind works. Now most of the routines powering robots (and spam filters and everything else) is based on statistical heuristics. Bayesian inference and fancier stuff. People who work on these kinds of things spend a lot of time thinking about how to get more accuracy out of their algorithms (for detecting faces, for predicting whether a message is spam, for planning the route a robot will take) and/or how to make these algorithms run faster. It is pretty amazing that all of the obscure stuff you might have learned in probability theory is the reason that your Facebook friend feed works.

As for me, I focus on programming languages. Programming languages protect people from having to reason about bits, or machine-level code, or sometimes even complex details of execution. Programming languages researchers range from logicians who try to make programs more correct and hackers who try to make people's lives easier. The specifics of this will be the subject of another blog post, as we seem to be reaching the limits of your attention span.

Oh, you were just trying to make polite conversation? Sorry. Maybe you can pass this along to a friend.