Viewpoint: The role of revealed research preferences

As a young person in decision research you may find it hard to figure what your “thing” is. However, you are not alone. Many successful professors did not initially decide on a research agenda, but instead, through the process of research, their agenda gradually developed.

Though our field is quite firm in arguing that preferences are constructed not revealed, I have found that time and time again professors explain that their research preferences have “revealed themselves.” Professor Adam Waytz has  said that he discovered what he was most interested through the process of research rather than directly deciding what his research interests were. Others also may help to reveal your preferences to you: professor Mark Leary said that one day a colleague explained him what Leary’s research focus was. Leary realized that he had never thought about framing his interests like that before, but when it was framed as such, it was clear: that was what he was interested in.

To characterize Professors Waytz or Leary as just idly bouncing around before their moments of revelation would be wrong, as they are outstanding examples of organized research. However, comments like these offer an interesting insight into the research development process: rather than trying to decide what you should do, think about “what do I keep coming back to every time I come up with ideas?” You may also be wise to pay attention to your colleagues as we come to know how our colleagues’ minds work and what theories our colleagues tend to gravitate towards. One’s friends can be a great resource for personal insight and they probably know you better than your advisors do.

Finally, senior researchers overwhelmingly advise you to follow your passion. So when going forward, why not go up to your advisor and say “here is what I am interested in on a personal level, how can we go forward with it?”. If they disagree to this approach, just remind them that nearly every professor at doctoral conferences tells students to follow their passion and that deciding on your “thing” is not a simple logical decision that can be made in a 30 minute meeting. Tell your advisers it is going to take you time to understand yourself as a researcher… though maybe phrase that a little more gently and less hippie–like!

Advertisements

Viewpoint: Focusing through a multi-pronged approach

multiprongIf you are a researcher, then there is a near 100% chance that you have been told at least once to “focus, focus, focus your research!” However, it is difficult to figure out how to focus, especially when even the best people in the field seem to have such “unfocused” research agendas.

At a recent mentorship lunch, psychologist Mark Leary used a physical metaphor that helped to clarify to me how one can be topically diverse but theoretically focused.

Leary explained how he did research by holding out an open hand with his fingers extended. He explained that the core theoretical interests rested in his palm, but that he explored that idea through many different prongs (his finger). This is the “multi-pronged approach.”

This approach permits a researcher to investigate different substantive or psychological questions, but with a firm foundation in a topic they hold expertise in.

For a young researcher the multi-pronged approach permits controlled exploration. For instance, a person focused on self-control might be interested in construal level or motivated cognition. Instead of completely jumping ship to a different topic, the researcher can ask, “What do I know about self-control that might be interesting to look at in conjunction with construal level or motivated cognition?”.

Science is best when it is cumulative—when multiple people do multiple projects on the same idea. With few exceptions, researchers are also best when they are internally cumulative—personally doing multiple projects on the same idea.

In sum, the multi-pronged approach provides an example of “programmatic research” that is less linear, but is not disorganized. Linear “programmatic research” may take almost a decade to manifest, but the multi-pronged approach can take much less time, may work for better for a graduate student, and arguably is what many researchers in our field actually do.

Viewpoint: Are you spending your research time wisely?

Editor’s note: In this post, Troy goes beyond the surface to see if professors really have as few regrets as they claim, whether all research is equally valuable and how young researchers should spend their time. These important issues will also be explored in our week-long special “The 7 sins of consumer psychology”, starting on 7th April with each day uncovering a new sin. Stay tuned!

time

After many mentor lunches, doctoral consortiums  and interviewing professors for this blog, I have come across a common theme: many professors actually regret a lot.

Though the “Research Heroes” series on this blog has been full of interviews of professors saying more or less “I regret nothing,” this sentiment is not shared by many in our field. Since, it is unlikely that even 2% of the people reading this post will be as successful as the Research Heroes we have previously interviewed, it may be useful for us to listen to what other professors are saying (many who are also very successful).

So why do so many professors regret how they spent their time?  It comes down to one phrase: All research is valuable, but some research is more valuable. Many professors regret working on projects that had little impact theoretically or substantively, but were safe publications. Today those publications have few cites and no practitioner has ever lifted an eyebrow at the findings.

For anonymity’s sake I will not reference the specific projects professors deemed as their “unimportant projects” but the projects they mentioned shared a number of qualities. These qualities are: 1) the findings were idiosyncratic to a domain, 2) it was difficult or pointless to apply the findings in a practical way, and 3) the findings neither answered nor stimulated any interesting questions. Oddly enough, these weaknesses could have been identified before starting the project.

Literature, parents, and religion have taught us to spend our limited time on Earth wisely. And though we may apply this wisdom to how we live our lives, we are not always good at applying it to how we do research, often chasing an idea because we think “it will work” rather than thinking, “does it matter if it works.”

Maybe try this test next time you have an idea. First tell a decision researcher and see if they think is theoretically interesting and distinct. Second, go tell a friend, significant other, or parent about your idea and see if they think it is interesting. If they don’t, you may want to reconsider the idea.

Outside The Matrix: Paul Litvak

LitvakPaul Litvak is currently a Quantitative Researcher working on the Google+Platform team to improve people’s social experiences online. Prior to that he was a Data Analyst at Facebook working on fighting fraud, tracking the flow of money and improving customer service. He also has a PhD in Behavioral Decision Research from Carnegie Mellon and his dissertation was on the impact of money on thought and behavior. During graduate school he co-founded a boutique data science consulting firm, the Farsite Group, which is consulting for some of the largest retailers and private equity firms to improve their data-informed decision-making processes. Through these various activities he’s managed to keep a foot in both the academic decision science and business data science worlds for the last 6 years. 

Tell us about your work: how does decision making psychology fit in it? I work at Google as a quantitative user experience researcher–I use quantitative methods to try and understand how people are (or aren’t using) features of Google products with the hopes of recommending ways to improve upon them. Often times this involves running an experiment but can also often involve correlational analyses instead. Sometimes the sample sizes are so large (millions or even billions!) you don’t need to run any statistics at all–you just count the rate at which some event happened.

Decision-making psychology fits into this work in at least three ways. First, in hypothesis generation and testing, knowing which  effects from psychology are relevant in a situation gives you great product intuition. For example, you might be analyzing how users bid on ad space and remind the engineers and designers of how much the anchor matters. Second, it’s useful in designing and conducting good experiments. In online experiments you are always weighing the pros and cons of different operationalizations of user constructs (e.g. what is “engagement” or “satisfaction” in the context of a particular website?). Being able to operationalize a variable intelligently is the difference between an experiment that convinces a Product Manager to change things accordingly and one that is totally ignored. Third, decision science lets you think clearly about analytic problems that come up a lot in software design. Nowadays it is common to use some machine learning algorithm to classify some otherwise messy data. In doing so, it is crucial to be able to think clearly about false positives and false negatives, and tradeoffs between the various costs of being wrong versus not making predictions for some cases. Fundamental statistical reasoning concepts (e.g. Bayes rule) never go out of style!

Why you decide to go into industry instead of continuing in academia? For me, it was a combination of factors. First, for many reasons (some outside my control), my research hadn’t been as successful as was needed to secure a good tenure track job. In order for me to have continued I would have had to have taken a postdoc for some number of years and continue working hard in the hopes that I could get sufficient papers published. I felt some amount of despair over my floundering career. (In retrospect, I’m not sure how overblown that was.)

Also, I had always had some interest in technology and business. I majored in computer science (and philosophy–I contain multitudes!) and had an interest in technology since I was a 10-year-old programming BASIC in my friend’s basement. Meanwhile I had co-founded a boutique statistics consulting group, Farsite (http://farsitegroup.com), that had had some early successes. Through trying to sell a variety of large businesses on consulting services (which I did in between running lab studies for my dissertation) I learned more and more about the business world. We even won a few contracts! More and more, I was enjoying applying the same scientific thinking I was using in research to solve business problems, like where to put pharmacies.

There were also quality of life issues. I wanted to have a life outside my job, and that seemed close to impossible as an academic. I noticed my advisor, who was a young tenure-track faculty, worked like a madman, seemed very stressed and unhappy. (He seems better now, and might dispute my contention that he was unhappy then.) Consequently, when a job opportunity came along to work for Facebook, pre-IPO, in Austin, Texas, where my best friend was living, it was nigh impossible to turn down.

What do you enjoy the most in your current role? By far the thing I enjoy the most about my role is having a large impact on the world. While I worked for Facebook, my analyses and code affected literally millions of dollars of revenue, and helped keep the site clean of a lot of bad content that would have made people’s daily experience much less pleasant. At Google, my research has launched whole product initiatives, determined whether to keep or get rid of product features, and literally affected what millions of people see across all of Google’s products every day. I have a huge amount of flexibility to work on research projects that interest me, in part because I love working on, and am good at formulating impactful research.

Do you see any challenges to the wider adoption of decision making psychology in your field? Yes, there are at least three challenges:

1) Because of disproportionate incentive to produce positive results and an increasing amount of researchers chasing fewer dollars and jobs, I do think the pressure to cut corners has increased significantly. This is impacting the quality of research that is being produced. Not just in terms of replicability and p-hacking, but also in terms of theoretical comprehensiveness. I read a lot of papers and I can’t help but feel like decision science isn’t very cumulative. Most researchers are chasing individual findings instead of trying to integrate our understanding of decision-making into a cohesive model or theory. It feels like it’s stagnated a bit to me–the best papers I read were written in the 70s, 80s, and 90s. I think the grab-bag nature of our findings makes it difficult to know which findings to apply in a given new context.

2) Another related problem is interactions. Social scientists uncover many many effects, but in real life many different effects could be active at the same time. It’s hard to know if all these effects should be additive, or what will win out when certain psychological antecedents suggest opposite effects. Perhaps more experiments at large scale can help this.

3) A third problem is entrenched attitudes toward experiments. I’ve definitely seen companies and executives resistant to the idea of running experiments. Sometimes they are worried about what will happen if the press finds a weird version of a product or feature. Sometimes they object to a lack of uniformity and vision in a product offering. Sometimes they are just ignorant about statistics, and have basic skepticism about generalizability and research. I’m happy to say that I think this has changed a great deal over the last 5 years. Nate Silver has done some good work in this area.  🙂

How do you see the relationship between academic researchers and practitioners? I see the relationship as fundamentally symbiotic.

Academics help practitioners in at least 4 ways (even setting aside direct collaboration, which is quite common nowadays): creating new methods, discovering findings in the lab that can then be applied, creating new theories from which to base products on (e.g. Goffman’s work on self presentation and different identities could affect the sharing model in social networks), and giving a sense of context and history. The last one is particularly important for various techno-utopists out there who think that they can use technology to fundamentally alter social relations without considering the results of previous attempts to do just that.

Practitioners help academics as well; they provide lots of data and invent useful technology. Have decision scientists and psychologists started thinking yet about what Google Glass will do to transform research? Imagine field studies were you could record what the subject is seeing when they make their choice? Or think about what the second screen could offer in terms of real time experience sampling or extra information to alter a choice. The possibilities are endless. Finally, and most obviously, practitioners often have access to lots of money… which is helpful, I’m told.

What advice would you give to young researchers who might be interested in a career in your field? Three things:

1. Come talk to me. 🙂

2. Learn some programming. R, then SQL, then Python, or some other scripting language. The more programming you learn the higher up the food chain you can go. If you know a lot of programming, you aren’t limited by what data exists, but only by what data you can create. This is hugely empowering, and increases your impact considerably. However, if all you learn is R, that is still incredibly useful,and will still get you into a variety of jobs.

3. Be curious! So many useful insights come from a broader curiosity about the world. This applies to both academic and worldly knowledge. Very random papers have led me to business/product insights. Similarly, keeping curious about what’s going on in the world is what enabled me to get into technology in the first place. Keep learning!

Want to read more? Try these…

Passion – The Converging Theme of the SCP Doctoral Consortium

ImageJim Bettman, Punam Killer, Greeta Menon, Barbara Kahn, and many of the other speakers and round table leaders all touched on one common theme: “one should do work on what you are passionate about.”

This doesn’t mean ignoring the necessary unfun pieces one has to do to have a successful career, but it means making passion the center of one’s career. The faculty also made the point that success comes first and foremost from passion.

Jim Bettman kicked off the day of advice by putting passion as the first principle of his research plan (passion-ownership-impact). Overall, the faculty converged on the idea that passion helps motivate good work and supports a good life.

This advice stands in stark contrast to the advice graduate students often hear about being strategic. One roundtable panelist noted that “One can be over strategic” and another suggested that doing a paper just for an easy “A” publication is bad idea. Other speakers noted that one should follow their passion even if that passion leads them out of academia and into industry.

Jim Bettman summarized that that passion is a way to inspire, drive, and sustain programmatic research.

SCP Doctoral Consortium Advice Highlights: Part 1

IMG_20130228_084125

Greeta Menon and Barbara Kahn recommend the 2 – 2 – 2 pipeline plan.

The professors advised students to have a pipeline of work with a few projects at each of the stages (e.g. review, writing, data collection). As a rule of thumb they offered that one should try to have 2 items at each stage.

Be Your Own Brand

Geeta Menon explained that ““People recognized you for you, not where you go, so you can be your own brand.” She mentioned how no matter where you can go you can use your own work and your own web presence to shape how others see you.

The Punam Keller Goal List

Keller explained that she sets specific academic goals each year (~5). The goals must have actionable steps. Then whenever she considers doing an activity she simply asks “does it fit the goals?” If it does not, she puts the activity and her “say no” list. She says this keeps her focused and her “say no” list allows her to feel okay saying no to things. She also keeps a personal goals list in a similar way and accordingly keeps a great work life balanced. “I have a fabulous life,” Keller told students on Thursday.

“Play around with Facebook Ads”

Zak Tormala recommends students look into the opportunities with Facebook advertising to test hypothesis with cell sizes approaching the millions.  The use of field data was echoed by many throughout the conference. Punam Keller presented a nuanced view of field research. She said she uses the field when it is appropriate and the lab when it is appropriate. She advised against testing hypothesis in the field just because, noting “there needs to a reason” to use the field.

Highlights: Jim Bettman’s SCP Career Talk

ImageJim Bettman brought his blend of insight and humor to his career talk this morning at the Society for Consumer Psychology Doctoral Consortium. Here’s what he advised students.

His guiding principles: “P.O.I.”

Passion – “Love it or leave it.” Only do things you believe in.

Ownership – Do your own ideas, not your advisors’. Or take your advisors ideas and make them your own as his former student Mary Frances Luce did by applying emotion to Bettman, Payne, and Johnson’s established decision research.

Impact – “Go for the gold.” Aim to change the field.

Reading advice: “Make generating study ideas a focus of whatever you’re reading.”

 If you ever get the pleasure of getting a manuscript of yours reviewed by Bettman, you will find that nearly every paragraph has a comment about a potential future direction or connection to another literature. This persistent focus on generating ideas is what has allowed Bettman to be so wildly impactful. Though Bettman often appears very programatic, this should not be taken as sign of him not thinking diversely.

On what to follow up on: “Be ready for lightning bolts.”

Bettman said be open to new ideas and when you find one you are passionate about embrace it and then follow up on it. Inspiration comes first and then the programmatic process begins.

Caveat: “Not a once size fits all.” 

Bettman indicated that even though be believed in his research style, there have been many people in the field who have succeeded in following a different style of research.

Funniest quote: Good research is like “ideas having sex.”

Bettman noted that good ideas come when ideas are pieced together with other ideas, when ideas are allowed to cross-fertilizer and different combinations of idea chromosomes come together.

Second funniest quote: “When talking about how Mary Frances Luce suggested emotion be brought into decision research, “We had never discussed emotion, or been emotional.”