After a long break we return to the Outside the Matrix series with Jolie Martin, a quantitative user experience researcher at Google. She received her PhD in Science, Technology, & Management at Harvard through a joint program between Harvard Business School and Computer Science department, and did post-docs both at Harvard Law School Program on Negotiation as well as the Social and Decision Sciences department at Carnegie Mellon. Prior to joining Google, she was also an Assistant Professor in Strategic Communication at the University of Minnesota.

Tell us about your work: how does decision making psychology fit in it? My title for the last year or so has been Quantitative User Experience Researcher at Google. However cumbersome, all the words are necessary to indicate what I do. Like my colleagues who do “regular” (qualitative) user experience research, my goal is to understand when users successfully satisfy their information needs using Google products. In my case, working on the Search Analysis team, I specifically develop metrics that describe how users interact with features on the Google search results page. The key distinction from other user experience researchers is the data source I draw upon, and as a result the types of analyses I do. Rather than running lab studies or even large online studies through tools like mturk, for the most part I rely on data recorded in logs to tell me how real users behave under natural conditions. The benefit of this approach is massive amounts of data. Nearly everything of interest is significant, sometimes even with very minor tweaks to the product that are imperceptible to the average user. The drawback – although it’s sometimes the fun part – is that I have to draw inferences from behavioral signals about users’ preferences, intentions, and satisfaction.

Judgment and decision making to the rescue! My theoretical background in this field has been extremely helpful in formulating hypotheses about why users search the way they do, from the queries they enter to the sequence of clicks that they take. For example, in considering ways to improve the user experience with exploratory tasks that require large amounts of subjective information (say, choosing where to go on vacation), I need to be mindful of contrasting interpretations of a user’s behavior. If she spends more time and clicks more links, this could be a bad signal that she simply didn’t find the information necessary to make a decision, that she suffered from information overload, or that she was distracted and continued browsing to procrastinate on a more worthwhile task. On the other hand, it could be a good signal that we offered her a rich set of information sources – increasingly tied to her personal characteristics and social networks – that offered insights worth delving into. To tease apart these interpretations requires testing mental and behavioral models of an extremely diverse set of users.

Why you decide to go into industry instead of continuing in academia? Unlike many of my academic colleagues – and even many people I know in industry who jumped ship – I never embarked on a PhD specifically to pursue a career in academia. In fact, I was clueless that this was the expectation of my advisors until several years into my PhD program! I was operating under the assumption that building theoretical knowledge and methodological skills would serve me well in any career. At some point right around my third or fourth year of grad school, I did become somewhat indoctrinated to the notion that academia is the “highest calling” and we should leave the actual implementation of our ideas to others. And of course I realized how difficult it would be to return to academia should I leave, so with this in mind, I gave it the old college + MBA + PhD + 2 post docs + assistant professorship try before finally divesting myself of those sunk costs. I liked each of my academic positions, but often felt as if I was spinning my wheels to achieve an objective (publishing in journals read almost exclusively by other academics) that I didn’t really care about, so when Google contacted me, I figured it couldn’t hurt to interview. During the process, I was surprised to find many other people like me with PhDs and interests in “pure” research. These were very smart people, and all had various personal and professional reasons for leaving academia, but it became clear to me that it was a choice, not necessarily indicating that someone couldn’t make it in academia.

That said, I am a firm believer that people enjoy things that they are good at, and where they can continue improving over time. I thought Google would offer exactly this for me. I have always loved building cool stuff, which is really the core of what we do. At the same time, there would be a lot to learn. When I accepted the offer at Google, I took a one-year leave from my assistant professorship (which was extremely generous of my department chair to offer), and it was nice to have that safety net should I dislike my new job. During the week of orientation with mostly software engineers, I thought more than once that I might need to use it. Just about everything flew over my head. But once I settled in with my teammates, I realized that everyone was willing to help, and no one had all the answers; doing logs analysis from end to end is complex by its very nature, and no one could step into the role as an expert. The expectations of me were that I be persistent and keep asking interesting questions. After a year in my position, the torrent of learning opportunities hasn’t tapered off in the least.

What do you enjoy the most in your current role? The main appeal of my job is the rapid pace that I can have impact on products that improve people’s lives in a tangible way, sometimes just through offering them a whimsical break from a busy life. I love working for a company that takes this mission seriously, and always holds it above monetary factors. Of course, this is not true of every company, so I feel lucky in that regard. I also have a nice variety of projects that result from mutual selection, and work with people in just about every role. There are only about 10 of us across the company in the Quantitative User Experience Researcher position, and our ability to glean insights from large data sets is highly valued by others. There is no prescribed way to perform these analyses, so we have freedom to use novel methods in distributed computing, machine learning, and natural language processing, among others. Last but not least of what makes my work stimulating is the chance to witness the evolution of cutting edge new technologies, such as riding in a self-driving car, wearing Glass, and seeing a prototype of a balloon that may one day provide internet in developing countries. Making these products useful requires not only tech savviness, but also political and legal knowhow.

Do you see any challenges to the wider adoption of decision making psychology in your field? Google and many other large companies are quite receptive to using decision making psychology in some ways. For example, I was involved in a “20% project” (whereby we can spend 20% of our time on something completely unrelated to our job function) running consumer sentiment surveys during the Democratic National Convention and presidential debates. I’m now working on another 20% project that draws upon academic research to test how environmental and informational factors shape food choices in our cafes. Similar studies have been conducted at Google to examine how defaults affect 401K allocations, and programs have been implemented based on the findings, with material effects on employee well-being.

However, for several reasons, there is more resistance to using basic research in the creation of products for end users. First, many companies in the technology industry are comprised mainly of software engineers (at last count, about 75% of Google employees) who may not consider psychology relevant. They often expect that users are “rational” in the sense of taking optimal actions given the set of options and information at their disposal, whereas we know this is rarely the case. Second, what research we do has focused on user response to specific technologies, with little ability to then generalize to a broader set of stimuli or outcome measures. This is related to the fast product development cycle I mentioned previously; we simply don’t have time to test fundamental psychological principles or the product will be launched and onto v2 before we have anything to say about it. This is changing gradually as the value of longer-term focus is realized. Third, while publishing is encouraged, there are not huge incentives to do so, especially given the more rigorous hoops we have to jump through in obtaining approval. Even in cases where we have interesting findings applicable to psychology more broadly, we often can’t disclose them for proprietary or privacy reasons.

How do you see the relationship between academic researchers and practitioners? In my opinion, the ideal relationship between academics and practitioners is one that takes into account the comparative advantages of each. While academics are usually more in touch with trending or provocative research topics that are likely to interest audiences and gain traction, practitioners are more aware of the available data sources and product use cases. Similarly, in terms of resources, academic connections provide legitimacy and wider dissemination of research findings, while those of us in industry can potentially be more useful in supplying funding, a sample population for experiments (be they users or employees), and analysis infrastructure (i.e., computing power). Collaborations would be more synergistic if there was greater engagement in both directions, with academics developing research questions based on real business or social issues, and practitioners making the additional effort to share findings via peer-reviewed conferences and journals.

What advice would you give to young researchers who might be interested in a career in your field? I’d suggest that students contemplating a transition to industry try a temporary or part-time internship; it’s a relatively low risk way to test the waters, and realistically, given the scarcity of professorships at top research universities, your advisors should support your consideration of other options. However, also be aware that one company isn’t going to fully represent all of industry, the same way stepping into a random graduate program or postdoc could be quite different from the one that is the best fit for you. I interned at a hedge fund during grad school and knew pretty quickly that it wasn’t for me, but it was a valuable experience nonetheless.

Perhaps more feasible for faculty members who are dissatisfied with certain aspects of their careers (e.g., working weekends and responding to emails at 3am), consider reaching out to people at companies of interest to you. You will likely find that they are excited to talk to someone with the wherewithal to do in-depth analysis of their users, and may even be open to handing over data or running experiments with you. Ask if you can present at company meetings to get a sense of the culture and style, or invite industry folks to present at your university. And don’t just build your network, but also maintain it by staying in touch with people you’ve worked with in the past. Referrals from a company’s current employees will make a big difference if you decide to apply!