Tag Archives: litvak

Viewpoint: Why social science grad students make great product managers

Posted on by
Reply
Litvak
A couple of months ago we featured Paul Litvak from Google in our Outside the Matrix series. After his interview, his inbox was inundated with questions from readers and he recently wrote a response on his own blog which we thought was so fantastic we wanted to republished it on InDecision as well. So, this week Paul shares his views on why social science grad students make excellent product managers. Note: even if you’re not a grad student yourself, it’s worth reading Paul’s views in case you’re ever in a position to hire one! 

After my interview with InDecision Blog, a number of graduate students emailed asking me about careers in technology (hey, I asked for it). They were a very impressive lot from top universities, but their programming skills varied quite a bit. Some less technically minded folks were looking at careers in technology aside from data scientist. Enough of them asked specifically about product management, so I thought I would combine my answers for others who might be interested.

What does a product manager do?
Brings the donuts. The nice thing about social science grad students for whom reading about product managers is news is that we can skip over the aggrandized misconceptions about product management that many more familiar with the technology space might harbor. The product manager is the person (or persons) that stands at the interface between an engineering team building a product and the outside world (here includes not only the customers/users of the product, but also the other teams within a given company who might be working on related products). The product manager is in charge of protecting the “vision” of the product. Sometimes they come up with that vision, but more often than not, the scope of what the product should be and what features it needs to have today, next week, or next year is something that emerges out of interactions between the engineers, the engineers’ manager, the product manager, company executives, etc etc. The product manager is really just the locus of where that battle plays out. So obviously there is a great need for politicking at times as well.

But wait, there’s more! Once the product is actually launched, it is typically still worked on and improved (or fixed). So the product manager is also the person that gets to figure out how to prioritize the various additional work that could be done. But how do they figure out what needs to be changed or fixed? This is one of the places where research comes in! So someone like me might do analysis on the data of people’s actual usage of the product (the product manager prioritized getting the recording of people’s actions properly instrumented, right? RIGHT?). Or a qualitative researcher might conduct interviews of users in the field and try and abstract an understanding from that. Either way, the product manager has to make sense of all this incoming information and figure out how to allocate resources accordingly.

Why would social science graduate students be good at that?
Perhaps you can see where I’m going with this. Products are increasing in scope. Even a simple app has potentially tens of thousands of users. Quantitative methods are becoming increasingly important for understanding what customers do. In such an environment, being savvy about data is hugely advantageous. In the same way that many product managers benefit from computer science degrees without coding on a daily basis, product managers will benefit from knowing statistics, along with domain expertise in psychology, sociology, anthropology even if they aren’t the ones collecting and analyzing the data themselves. It will help them ask the right questions and to when to trust results, and when to be more skeptical. It will help them operationalize their measures of success more intelligently.

The soft skills of graduate school also translate more nicely. Replace “crazy advisor” with “manager” (hopefully a good one) and replace “fellow graduate students” with “other product managers” and many of the lessons apply. Many graduate social scientists will have plenty of experience with being part of a lab and engaging in large-scale collaborative projects. Just like in graduate school, a typical product manager will spend hours fine tuning slide decks and giving high stakes presentations meant to convince skeptical elders of the merit of a certain course of research (replace with: feature, product, or strategy).

Finally, building technology products is a kind of applied social science. You start with a hypothesis about a problem that people are having that you can solve. Of course, as a social scientist, the typical grad student understands just how fraught this is! Anthropologist readers of James Scott and Jane Jacobs and economists who love their Hayek will have a keen appreciation for spontaneous order (“look! users are using this feature in a totally unexpected way!”), as well as the difficulties of a priori theories of users’ problems or competencies. In fact, careful reading of social science should make a fledging PM pretty skeptical of grand theories. For instance–should interfaces be simpler or more complicated? How efficient should we make it to do some set of common actions? If everything is easily accessible from one click on the front page, will there be overload of too many buttons? Is that simpler or more complicated? These sorts of debates, much like debates about the function of particular social institutions or legal proscriptions, are not easily solved with simple bromides like “less is always better”, or “more clear rules, less discretion” (I am reading Simpler: The Future of Government by Cass Sunstein right now, and he makes this point very well with respect to regulations). The ethos of the empirical social scientist is to look for incremental improvements bringing all of our particularist knowledge to bear on a problem, not to solve everything with one sweeping gesture. This openness is exactly the right mentality for a product manager, in my opinion.

Conclusion
I hope I have at least partially convinced you that as an empirical social scientist, you would make a great product manager. Now the question is, how do I convince someone in technology of that? The short and most truthful answer is, I’m not 100% certain. It might take some work to break into project management, but I see lots of people with humanities background doing it, so it can’t be that hard (One of my favorite Google PMs is an English PhD). One thing I would suggest is carefully framing your resume to emphasize your PM-pertinent skills–things like, group project management, public speaking experience, making high stakes presentations, etc. You might also consider making a small persuasive deck to show as a portfolio example of a situation where you convinced someone of something (your dissertation proposal could work?). This would be a great start. Another thing is consider more junior PM roles initially–as a PhD coming out of grad school you are still going to make a fine salary as an entry-level product manager. If you apply these principles I have no doubt that you will quickly move up.

Read Paul’s original interview here.

Outside The Matrix: Paul Litvak

Posted on by
11

LitvakPaul Litvak is currently a Quantitative Researcher working on the Google+Platform team to improve people’s social experiences online. Prior to that he was a Data Analyst at Facebook working on fighting fraud, tracking the flow of money and improving customer service. He also has a PhD in Behavioral Decision Research from Carnegie Mellon and his dissertation was on the impact of money on thought and behavior. During graduate school he co-founded a boutique data science consulting firm, the Farsite Group, which is consulting for some of the largest retailers and private equity firms to improve their data-informed decision-making processes. Through these various activities he’s managed to keep a foot in both the academic decision science and business data science worlds for the last 6 years. 

Tell us about your work: how does decision making psychology fit in it? I work at Google as a quantitative user experience researcher–I use quantitative methods to try and understand how people are (or aren’t using) features of Google products with the hopes of recommending ways to improve upon them. Often times this involves running an experiment but can also often involve correlational analyses instead. Sometimes the sample sizes are so large (millions or even billions!) you don’t need to run any statistics at all–you just count the rate at which some event happened.

Decision-making psychology fits into this work in at least three ways. First, in hypothesis generation and testing, knowing which  effects from psychology are relevant in a situation gives you great product intuition. For example, you might be analyzing how users bid on ad space and remind the engineers and designers of how much the anchor matters. Second, it’s useful in designing and conducting good experiments. In online experiments you are always weighing the pros and cons of different operationalizations of user constructs (e.g. what is “engagement” or “satisfaction” in the context of a particular website?). Being able to operationalize a variable intelligently is the difference between an experiment that convinces a Product Manager to change things accordingly and one that is totally ignored. Third, decision science lets you think clearly about analytic problems that come up a lot in software design. Nowadays it is common to use some machine learning algorithm to classify some otherwise messy data. In doing so, it is crucial to be able to think clearly about false positives and false negatives, and tradeoffs between the various costs of being wrong versus not making predictions for some cases. Fundamental statistical reasoning concepts (e.g. Bayes rule) never go out of style!

Why you decide to go into industry instead of continuing in academia? For me, it was a combination of factors. First, for many reasons (some outside my control), my research hadn’t been as successful as was needed to secure a good tenure track job. In order for me to have continued I would have had to have taken a postdoc for some number of years and continue working hard in the hopes that I could get sufficient papers published. I felt some amount of despair over my floundering career. (In retrospect, I’m not sure how overblown that was.)

Also, I had always had some interest in technology and business. I majored in computer science (and philosophy–I contain multitudes!) and had an interest in technology since I was a 10-year-old programming BASIC in my friend’s basement. Meanwhile I had co-founded a boutique statistics consulting group, Farsite (http://farsitegroup.com), that had had some early successes. Through trying to sell a variety of large businesses on consulting services (which I did in between running lab studies for my dissertation) I learned more and more about the business world. We even won a few contracts! More and more, I was enjoying applying the same scientific thinking I was using in research to solve business problems, like where to put pharmacies.

There were also quality of life issues. I wanted to have a life outside my job, and that seemed close to impossible as an academic. I noticed my advisor, who was a young tenure-track faculty, worked like a madman, seemed very stressed and unhappy. (He seems better now, and might dispute my contention that he was unhappy then.) Consequently, when a job opportunity came along to work for Facebook, pre-IPO, in Austin, Texas, where my best friend was living, it was nigh impossible to turn down.

What do you enjoy the most in your current role? By far the thing I enjoy the most about my role is having a large impact on the world. While I worked for Facebook, my analyses and code affected literally millions of dollars of revenue, and helped keep the site clean of a lot of bad content that would have made people’s daily experience much less pleasant. At Google, my research has launched whole product initiatives, determined whether to keep or get rid of product features, and literally affected what millions of people see across all of Google’s products every day. I have a huge amount of flexibility to work on research projects that interest me, in part because I love working on, and am good at formulating impactful research.

Do you see any challenges to the wider adoption of decision making psychology in your field? Yes, there are at least three challenges:

1) Because of disproportionate incentive to produce positive results and an increasing amount of researchers chasing fewer dollars and jobs, I do think the pressure to cut corners has increased significantly. This is impacting the quality of research that is being produced. Not just in terms of replicability and p-hacking, but also in terms of theoretical comprehensiveness. I read a lot of papers and I can’t help but feel like decision science isn’t very cumulative. Most researchers are chasing individual findings instead of trying to integrate our understanding of decision-making into a cohesive model or theory. It feels like it’s stagnated a bit to me–the best papers I read were written in the 70s, 80s, and 90s. I think the grab-bag nature of our findings makes it difficult to know which findings to apply in a given new context.

2) Another related problem is interactions. Social scientists uncover many many effects, but in real life many different effects could be active at the same time. It’s hard to know if all these effects should be additive, or what will win out when certain psychological antecedents suggest opposite effects. Perhaps more experiments at large scale can help this.

3) A third problem is entrenched attitudes toward experiments. I’ve definitely seen companies and executives resistant to the idea of running experiments. Sometimes they are worried about what will happen if the press finds a weird version of a product or feature. Sometimes they object to a lack of uniformity and vision in a product offering. Sometimes they are just ignorant about statistics, and have basic skepticism about generalizability and research. I’m happy to say that I think this has changed a great deal over the last 5 years. Nate Silver has done some good work in this area.  🙂

How do you see the relationship between academic researchers and practitioners? I see the relationship as fundamentally symbiotic.

Academics help practitioners in at least 4 ways (even setting aside direct collaboration, which is quite common nowadays): creating new methods, discovering findings in the lab that can then be applied, creating new theories from which to base products on (e.g. Goffman’s work on self presentation and different identities could affect the sharing model in social networks), and giving a sense of context and history. The last one is particularly important for various techno-utopists out there who think that they can use technology to fundamentally alter social relations without considering the results of previous attempts to do just that.

Practitioners help academics as well; they provide lots of data and invent useful technology. Have decision scientists and psychologists started thinking yet about what Google Glass will do to transform research? Imagine field studies were you could record what the subject is seeing when they make their choice? Or think about what the second screen could offer in terms of real time experience sampling or extra information to alter a choice. The possibilities are endless. Finally, and most obviously, practitioners often have access to lots of money… which is helpful, I’m told.

What advice would you give to young researchers who might be interested in a career in your field? Three things:

1. Come talk to me. 🙂

2. Learn some programming. R, then SQL, then Python, or some other scripting language. The more programming you learn the higher up the food chain you can go. If you know a lot of programming, you aren’t limited by what data exists, but only by what data you can create. This is hugely empowering, and increases your impact considerably. However, if all you learn is R, that is still incredibly useful,and will still get you into a variety of jobs.

3. Be curious! So many useful insights come from a broader curiosity about the world. This applies to both academic and worldly knowledge. Very random papers have led me to business/product insights. Similarly, keeping curious about what’s going on in the world is what enabled me to get into technology in the first place. Keep learning!

Want to read more? Try these…