In The Wild: Tom Wein

35b5aeaIn our first In The Wild interview of 2014, we speak to Tom Wein who is a behavioural change consultant who has led major primary research projects to tackle counter-radicalization, aid security sector reform, plan public diplomacy efforts and design communication strategies. He currently works on behavioural change for national security, principally for the consultancy SCL. He read War Studies at King’s College, London, and has also worked for the European Defence Agency in Brussels as a communications consultant.

Tell me about your work: how does decision making psychology fit in it? We conduct research projects for the US and UK militaries in fragile and conflict-affected states, and design interventions to reduce violence. The one thing we’re always trying to explain is that just asking people about their attitudes isn’t enough – you need to examine their psychology in order to change behaviour. So we measure concepts which psychologists will be familiar with, like self-efficacy, motivations and reward structures, to build up a much deeper picture of a group; that way we can come up with much more effective ways of to solve the problem.

Of course, nobody will pay us to do research in Switzerland – our projects are invariably in places where high quality research is difficult. We can partly solve those problems through good recruitment and training, and through building in redundancy, but crucial to the way we research is a process of triangulation. Some problems are inevitable, given the challenges, but two research strands are unlikely to go wrong in identical ways, so we focus on those findings that are confirmed by several sources. We generally use a mixture of semi-structured depth interviews and surveys (containing scales), plus a few focus groups and more free-form interviews with experts at either end of the process, to inform that process.

How did you first become interested in decision making psychology? Like a lot of people studying conflict, I was frustrated with the crudeness of the military’s tools in fighting the deeply complex wars in Iraq and Afghanistan – wars that were defined by our ability to win over the very people we kept accidentally killing. At the same time, I was shocked (I still am!) at how much money was being spent on projects and policies with only the flimsiest evidence base. Those two ideas were crystallized when I came to work for SCL, and found that there was a better, more intelligent way of doing things.

What type of research do you find most interesting, useful or exciting? I am always, always looking for field trials. Hypotheses are great, and laboratories are wonderful places, but I want you to prove that your thing could work in the messiness of the real world (and that doesn’t mean testing on American college students!). No doubt lots of the readers will be familiar with the work of Chris Blattman, whose work in Liberia and Uganda is magnificent stuff. The younger members of the development industry really do ‘get’ evidence and research, even if they’re still sometimes fighting their elders. When I argue that you’ve got to look at groups, rather than humans in general, constantly in the background is the work of Stathis Kalyvas, who has written powerfully about the impact of very local conditions on the conduct of wars.

Other than that, I am always more excited by elegantly written work, and by work that is open access. Those factors are much more important than the field a paper comes from. I’m also suspicious of the validity of findings in different contexts, so I’m often looking for research conducted in the country I’m studying at the time.

Do you see any challenges to the wider adoption of decision making psychology in your field? There’s an awful lot of persuading still to do. In the UK, the Behavioural Insights Team has been invaluable in persuading people that they ought to do research before taking a decision, but in the US there’s a complete focus on very simplistic attitude surveys, if they do research at all. Part of the problem is that comprehensive research projects in warzones are really expensive – it’s a lot cheaper to just do a quick poll.

How do you see the relationship between academic researchers and practitioners? We’ve been quite lucky in that respect – there is a reasonable-sized cohort of academic researchers who have been doing some exciting research in this field, and they’ve been generous with their time, especially when we’re trying to learn about and plan research in a new country. As I hinted above, I can get quite frustrated with the academic system, but that hasn’t prevented us from working well with individual academics.

What advice would you give to young researchers who might be interested in a career in your field? The first thing is to learn some quantitative skills. There are lots of people who can write essays out there; you’re far more likely to get an interesting job if you can also analyze data. The second, rather depressing, thing to say is that there are fewer and fewer full time jobs where you’ll get trained up – you may well have to fight for a series of short term projects before you get hired properly. Therefore, make contacts, network, and use your time at university effectively (including begging professors for introductions) – you’ll never have so much time again. Finally, if you’re in London, go to the monthly behavioural economics networking drinks!


Happy New Year from InDecision!

It’s been little over a year since we started this blog, with the hope of attracting a couple of hundred readers. Instead, we’ve had over 70,000 hits with over 35,000 visitors from 155 countries. The top 10 countries for visitors included:

  1. visitors globallyUnited States
  2. United Kingdom
  3. Canada
  4. Germany
  5. The Netherlands
  6. Australia
  7. India
  8. Singapore
  9. Sweden
  10. Switzerland

So much, so predictable! But who does JDM research in Honduras, Kyrgyzstan, Mongolia, Vanuatu, Sudan, Rwanda, Ghana, Nicaragua, Bermuda, Bhutan, Barbados or Bolivia? If that’s you, get in touch – we’d love to speak to you and hear about JDM research in your country. This year, we’ll address one of the issues highlighted by professor Dan Ariely in his interview and start to look at what impact culture might have on decision making science through a series of interviews focusing on the challenges (and opportunities!) cross-cultural psychology might pose for JDM.

In case you missed them the first time, the top 10 posts from the year are:

  1. Research Heroes: Richard Thaler
  2. In The Wild: Rory Sutherland
  3. Outside The Matrix: Paul Litvak
  4. The Seven Sins of Consumer Psychology
  5. Research Heroes: George Loewenstein
  6. Viewpoint: The role of revealed research preferences
  7. Outside The Matrix: Jolie Martin
  8. In The Wild: Kelly Peters
  9. Research Heroes: Colin Camerer
  10. Research Heroes: Gerd Gigerenzer 

We’ve been incredibly lucky in being able to interview some amazing people in our field, and we can’t thank them enough for giving their time to answer our questions. On behalf of all the people who have thanked us for running the blog, please know that your contribution is widely appreciated and makes a big difference to young researchers around the world.

The original aim of the blog was to give young researchers a voice. We’ve taken some steps in that direction by growing the team with sub-editors Caroline Roux, Shereen Chaudry and Leigh Caldwell as well as our dedicated contributor Troy Campbell. In 2014, we’ll start to feature young researchers more regularly through a new interview series. We’ll also widen our net for career advice to include researchers who have are making waves early on in their career and shaping the field as they go.

Lulu+-+Something+To+Shout+About+-+CD+ALBUM-427621We’d also welcome submissions from readers: if you’re a young researcher and have just published an awesome paper you want to tell the world about, get in touch. Since subtle hints and words of encouragement have so far fallen on deaf ears, let us put this bluntly: blatant self-promotion is OK, and strongly encouragedOne of the main goals of this blog is to give young scholars a platform to share and discuss their work, but we cannot achieve this goal without your contribution!

Finally, one of the emerging trends in our field is the rising popularity of field studies and applying the science both in the policy and commercial worlds with many of our Research Heroes highlighting the need to connect our work with the outside world. However, such work is extremely challenging and we have much to learn from the pioneers, so in 2014 we’ll also be speaking to those who have made early inroads into taking decision making science out of the lab and into the Real World.

As always, we welcome your feedback and contribution – please don’t hesitate to get in touch and let us know what you think!

We hope that you’ll enjoy the next year with us.

Elina & Neda

Research Heroes: Shlomo Benartzi

20110427_1192As one of the last posts this year, we’re featuring our 28th Research Hero: professor Shlomo Benartzi from UCLA Anderson School of Management, a leading authority on behavioral finance with a special interest in household finance and participant behavior in retirement savings plans. His most significant research contribution is the co-development of Save More Tomorrow (with Richard Thaler), a behavioral prescription designed to help employees increase their savings rates gradually over time. Professor Benartzi has also supplemented his academic research with both policy work and practical experience through advising government agencies in the U.S. and abroad as well as helping to craft numerous legislative efforts and pension reforms.  In addition, he has also worked with many financial institutions as an academic advisor. His latest initiative is where he’s exploring new digital interventions that will help consumers, businesses and policymakers leverage behavioral research. 

I wish someone had told me at the beginning of my career… that you should only do research you are really passionate about. Research often requires years and years of sustained effort, so unless you have a passion for these ideas, then you will almost certainly give up. (It’s like a marriage in that sense.) There’s also something magical that happens when you are passionate about the research. Not only is the work more fun, but it somehow gets published. Don’t ask me how it happens.

I most admire academically… I’m going to cheat and give you three names. The first person is Danny Kahneman. Not only is he super brilliant, but he’s also very insightful about questions outside his area of expertise. He never gives in to pressure, and always does what he thinks is right academically. Richard Thaler I admire for the diversity of his research program, and also his ability to see the big picture. John Payne is incredibly humble, yet an unusually deep thinker.

The project that I’m most proud of is… Save More Tomorrow, a little idea Thaler and I came up with that led more than 4 million people to double their savings rate. We weren’t particularly brilliant, but we were persistent and with a bit of luck we made a big difference.

The one project that I should never had done… I’m still trying to forget that.

The most amazing or memorable experience when I was doing research… I was salsa dancing at the boathouse in Santa Monica and chatted with my friend Brian Tarbox who worked in the finance industry. I told him about my idea for Save More Tomorrow; I didn’t even think he was listening. Several years later I hear from him again and he hands over an excel spreadsheet with all the data. He said I have some good news: I tried out the Save More Tomorrow idea and it works. Your program quadrupled the savings rate of these low-income people.

The one story I always wanted to tell but never had a chance… what I’d really love to do is follow-up with those people in Brian’s spreadsheet. The company insisted on being anonymous, and Brian passed away, but I’d love to know how those individuals are doing now. Are they still saving more? Have they managed to retire with dignity?

A research project I wish I had done… I had this hunch that automatic enrolment in a retirement savings plan would get a lot more people to start saving, but that it might also lead to a decrease in aggregate savings, since the default saving rate is typically very low, often around 3 percent. I wanted to test out my hunch, but Brigitte Madrian tested it out first and did a superb job.

If I wasn’t doing this, I would be… an unhappy architect. I love good architecture, but if it was my profession then it would no longer be a fun hobby. I would have to pay the bills and deal with clients.

The biggest challenge for our field in the next 10 years… is increasing our impact. How do we take these proven behavioral insights and scale them up? How do we solve big societal problems around health care or retirement savings or education? In my future work, I’m going to explore how we can use the digital revolution to accelerate the pace of change. With, I want to test out new digital interventions that will help consumers, businesses and policymakers leverage all of this new research. I think that smartphone in your pocket represents a tremendous opportunity to help people think better and make better choices, but we have to get it right.

My advice for young researchers at the start of their career is… not to listen to me! Every young researcher needs to tailor their journey to their particular set of skills, interests and weaknesses. Find your own passion. Don’t follow mine.

Departmental website |

SJDM 2013: InDecision team recommends…

SJDM toronto picGreetings from Toronto and the annual conference for the Society for Judgment and Decision Making! With the help of the InDecision team, we’ll be covering the best bits of the conference for you if you couldn’t make it (and even if you are here, we’ll have something for you, too). With dozens of great sessions on offer this weekend, choice overload is pretty much guaranteed. But fear not: we’ve scoured the program and selected the best ones to help you make the most of the conference. Here’s where you’ll find the InDecision team this weekend…

caroline new resizedCaroline’s picks

Research and Academia (Session #7) Questionable research practices. Misunderstanding or misuse of statistics. Lack of reproducibility. Many academic fields are currently going through several research-related crises and controversies. Different solutions are being proposed to improve the ways we conduct research, but I sometimes find it hard to keep up with all the suggested improvements for our different research practices. That is why I am always looking forward to conference sessions that can help me stay up to date with the most recent developments. The four papers presented in the session cover important issues, such as the replicabilitiy and reliability of behavioral research findings, and, most importantly, provide interesting solutions that I am really looking forward to learning more about. How to find it: Sunday, November 17, 2:45-4:15 pm, Civic South

The Relationship Between Altruism and Personal Benefits (Session #4) The existence of altruism, or whether humans can ever transcend self-interest, is an age-old question that is constantly being debated across different fields. It is a debate that I find quite interesting, so I am always drawn to conference sessions that provide new ideas, or revisit old ones, on the topic. I find this session particularly interesting because it explores the interplay between altruism and personal benefit and provides interesting findings about how perceived self-interested motives or outcomes can taint the judgment of seemingly altruistic behavior, among others. I am really looking forward to learning more about how this impacts people’s judgment and performance of altruistic or prosocial behavior, and whether there are any ways to overcome these effects. How to find it: Saturday, November 16, 3:15-4:45 pm, Simcoe/Dufferin

elina resizedElina’s picks

Applying Behavioral Economics in the Field: Nudging Customers to Pay their Credit Card Dues The fact that this session is talking about a large-scale field experiment makes this session unmissable for me to two reasons. The primary reason is that for me field experiments represent an exciting new phase for the field itself: after years spent in the lab it’s time to migrate to the outside world to see how our ideas perform in reality. It’s risky because we can’t control everything so the level of noise is likely to be high, and we have to find partners for it which brings its own complications. This bridge between academia and practice is one that I feel we need to cross to ensure the relevance of our work to the outside world, which ultimately defines the value of our work through funding. On a personal level I’m also interested in hearing about the practical challenges of running such studies as it’s close to my own research interests both as a PhD student and practitioner so I’m hoping to get some great ideas and inspiration from this talk. How to find it: Saturday 16th November, 3.15-4.45pm, Session #4 Track Ι: Choice Architecture 2 – Willow East

The Impact of Comparison Frames and Category Width On Strength of Preferences This session is definitely one I’ll be listening to with my practitioner hat on: understanding the strength of consumers’ preferences is at the heart of my work as a market research consultant. We know already that how options are presented to people changes how they perceive them, but when it comes to a real-life client scenario, it’s absolutely crucial to understand the nuances of how consumers make these comparisons to help advise our clients to emphasise the right attributes of a product. This might seem manipulative or even sinister, but just think for a moment about a product you really like: what if the “wrong” communication would have meant you’d never discovered that product? How to find it: Monday 18th November, 9.45-11.15am, Session #8 Track 2: Consumer Decision Making – Civic South

leigh resizedLeigh’s picks

Are risk and delay psychologically equivalent? Testing a common process account of risky and inter-temporal choice Research that unifies previously disparate effects is always interesting to me – because my instinct as a mathematician is to work towards ever more general and simpler (and therefore more powerful) models. If inter-temporal choice can be explained by the same process as probabilistic decisions, it takes us one step closer to understanding decisions in a coherent way. And this does seem a logical step: some accounts explain hyperbolic discounting as a rational response to the riskiness of a delayed reward – maybe if I hold off on eating the marshmallow and wait to get two of them, some unknown event will intervene and I won’t get any! However, it seems that these researchers have found evidence to counter this unification. I’ll be intrigued to hear what alternatives they put forward. How to find it: Saturday 16th November, 8.30-10am, Session #1 Track 2: Risk 1 – Essex

Partitioning option menus to nudge single-item choice This talk is interesting for me both for my consulting work with some commercial clients, and also because it feels like it could help understand how we compose small intermediate steps into larger decisions. Many complex decisions have various parts, and forcing people to unpack those individual steps (for instance by listing individual options separately rather than allowing people to integrate them into one bigger choice) may reveal some of the internal processes that are not directly observable. Classical decision theory (as used in rational economic modelling) assumes that separate choices can simply be added up to come to an overall totality of decisions, but the results of this paper seem to provide more confirmation that this doesn’t work. Seeing the differences between low-level and high-level choices may help us figure out a better way to put individual decisions together in a model and predict social behaviour. How to find it: Saturday 16th November, 10.30am-12pm, Session #2 Track 1 Choice Architecture 1 – Willow East

shereen resizedShereen’s picks

As a behavioral decision researcher, I am interested in finding behavioral solutions to policy-relevant problems. Indeed, I learned at APPAM this past weekend that there is a lot of room for behavioral research in the policy arena. With that in mind when looking at the SJDM program, I am focusing on talks that (1) investigate the practical elements that influence choice, or (2) identify either a behavioral problem or behavioral solution in a policy-relevant domain. For now, the talks in choice architecture (both of them) and financial decision-making are on the top of my list.

The first session on choice architecture addresses abstract but broadly relevant topics in choice architecture. These talks seem key to understanding basic concepts in this area such as defaults and choice sets. The second session on choice architecture delves into more area-specific interventions on choice and their effectiveness. These choice architecture talks have more direct relevance for policy, marketing, or other applications. With the recent formation of the Consumer Financial Protection Bureau (CFPB) in 2011, it is clear that policy-makers are concerned about the way people make financial decisions. The talks in the financial decision-making session speak directly to this concern with a series of experiments that either identify the obstacles people face in considering their finances and/or provide some way to mitigate these problems.

How to find them: Choice Architecture I – (Saturday, Nov 16, Track I, Session #2, 10:30am-11:50am); Choice Architecture II –  (Saturday, Nov 16, Track I, Session #4, 3:15pm-4:35pm); Financial Decision Making – (Saturday, Nov 16, Track III, Session #5, 5:15pm – 6:35pm)

troy resizedTroy’s picks

Cruel nature: Harmfulness as an overlooked dimension in judgments of moral standing “Cruel Nature” promises to be a great talk and not just because of its slick title. The talk will tackle an already controversial topic (the basis of morality) and throw an additional wrench into the puzzle (people respond to animals with moral emotions). The talk will be big in scope, have a good literature review, and will to lend itself to fiery conversation (or at least that’s how the talk played out when it was presented in multi-school Moral Research Lab). Piazza and colleagues propose that “harmful intent cannot be reducible to agency.” They use scenario studies featuring non human subjects like sharks to test and show this. This talk will ultimately try to critique the Agency-Patient model of morality, a model that is already a very new critique of also still relatively new Moral Foundations model of morality. With sharks, controversy and morality, even if you disagree with the speakers’ claims (and probably many people will), you’re guaranteed to have a good time. How to find it: Saturday 16th November, 1.30-3pm, Session #3 Track Ι: Morality and Ethics 1 – Willow East

Selfish or selfless? On the signal value of emotion in altruistic behavior This talk promises to be fascinating as it shows that the general populace holds a view of morality that widely differs from the view of morality most academics hold. Us ‘rational’ academics tend to think about morality like a math equation, where people sacrifice for others and don’t get any benefits – e.g. gifts or feeling a positive “helpers’ high” emotion. However, Barasch and colleagues show this is not the case. People actually think feeling a “helpers’ high” is a authentic signal of concern for others and are suspicious of the unemotional helper (e.g. the person many academics praise). The researchers do however show a boundary of this attribution which can help us understand where people in general see the line between selfish and selflessness in helping. How to find it: Saturday 16th November, 3:15-4:45 pm, Session #4 Track 3.

[N.B. Please check all session and presentation times in the official program before attending as typos may have slipped in!]

Final notes…

  • We’re covering the conference here (with a delay) as well as on Twitter both through @InDecision_Blog and our individual contributors: @RouxCaroline, @infomagpie, @leighblue and @troyhcampbell – conference hashtag is #sjdm2013
  • Don’t miss the Graduate Student Social Event on Saturday 16th from 6.45-8.45pm at the Willow Centre!
  • The InDecision dinner (featuring talks with three practitioners) on Saturday 16th still has 4 places left – please email asap if you want to join!
  • If you have any feedback on the blog or would like to get involved, please come speak to us – we’d love to hear from you!

In The Wild: Kelly Peters

kelly petersThis week in our practitioner series we’re featuring Kelly Peters, Chief Executive Officer and Managing Partner at BEworks, a behavioral economics firm based in Toronto. She has over twenty years’ experience leading strategy, technology and innovation in major companies, including RBC Royal Bank of Canada and BMO Bank of Montreal as well as an an MBA from Dalhousie University with a concentration in financial services.

Tell me about your work: how does decision-making psychology fit in it? I am the CEO of BEworks, a management consulting firm dedicated to the application of decision-making psychology to real-world challenges. The firm has been grounded in the interdisciplinary marriage of science and business since its inception in 2010 with two leading academics; Dan Ariely, and Nina Mazar, and two accomplished business strategists; Doug Steiner and Louis Ng. We also have two academic advisors: David Pizarro, a social psychologist from Cornell University and Supriya Syal, a neuroscientist working on her post-doctorate at University of Toronto. The hands-on engagement of academics in our projects is one critical thing that distinguishes us from many firms. This lets us do cutting-edge primary research in partnership with clients who want a competitive advantage.

Although our work is research intensive, we are hands-on practitioners designing experiments to change workflow and improve marketing strategies. I have an unusual analogy to explain how we bring three new techniques in the fight to improve the bottom line. The first technique is the right jab, which is the insight from behavioral science that explainswhy people make the decisions that they do; the second is a left hook which is about formulating hypotheses of what and how to influence people’s decisions; and the third is a drop-kick, which is empirical validation of the ideas through rigorous experiments.

We are finding that business leaders and policy-makers are hungry for scientifically-grounded innovation and experimention. They are starting to see how behavioral economics offers new solutions and new thinking. Our projects run the gamut of the four Ps of marketing, product, price, promotion, and place, but also process improvement work like fraud and collections. We have a diverse range of clients from around the world in financial services, retailers, news media, health care companies and even political campaigns. And we are seeing the same anomalies in rationality in every domain!

How did you first become interested in decision-making psychology? Growing up in the 1980s, I played text games on a TRS-80 and was the one who programmed my family’s early electronic devices. In university I studied philosophy, sociology, literary theory, political theory, and contemporary art. I became interested in technology and its impact on society, which is really about the behavior of adoption (remember Geoffrey Moore’s Crossing the Chasm) and attitudes towards technology (from denial to enthusiastic). Reading about Ted Nelson’s Project Xanadu led me to start my professional career in 1993 as a consultant focused on helping companies understand why and how to develop a web presence. I worked on the dotcom launch crew of the largest media properties in Canada. And though the media companies were the first to get online, I believe their business model depends on micropayments. Financial services were the first industry to have a real application for online capabilities. I took on a role as director of product development for a financial services dotcom where the goal was to fundamentally change the behavior of how people conduct their banking.

Most of my career was spent leading business strategy and innovation teams. Success depended on understanding what will drive adoption of new products and services, how to engineer a meaningful customer experience, and increase utilization of new channels like online banking. Few people realize how heavily banking relies on behavioral insights  – whether it’s understanding how to encourage customers to use new banking channels like ATMs or online banking, or from cheques to electronic transfers; to drive savings or borrowing; to engineering new products and driving their adoption; to assessing risk; and managing collections and preventing fraud.

In the 1990s, behavioral scoring data models were being developed to capture both the quantitative aspect of a person’s financial wherewithal such as their capacity for debt service and collateral, but also quantify “character.” This behavioral variable is what explains why a wealthy person could be a bad credit risk and a poor person could be a good one. On the other side of the balance sheet behavioral finance explains why a wealthy person can be a terrible saver and a poor person can be a diligent saver. Retail and commercial credit risk, behavioral finance, and enterprise risk management are theoretical constructs underpinned by models that derive explanatory power from behavioral attributes.

I gathered insights from thought leaders in economics and political theory (Hayek, Schumpeter) and risk theory and history (Against the Gods: The Remarkable Story of Risk by Peter Bernstein and Nassim Taleb’s book Fooled by Randomness). While these books provided incredible insight on how people are irrational, it was the work on “choice architecture” led by behavioral economists that provided the ah-ha, here’s how these insights can be applied to influence behavior. I devoured the research of Dan Ariely, Amos Tversky, Daniel Kahneman, Richard Thaler and Cass Sunstein along with the work of psychologists like Robert Cialdini. Businesses, and the academic programs they draw from, like MBAs and commerce degrees, ought to incorporate behavioral research and the scientific method if they want to understand their customers in non-intuitive or subjective experiential ways.

While at the RBC Royal Bank of Canada, I had the support of amazing executives and mentors to launch a series of behavioral economics projects starting in 2009. I had the joy of working with Piyush Tantia, John Balz and the ideas42 team. I also partnered with thought leaders like Nina Mazar and Dilip Soman at the University of Toronto & Rotman School of Business, which is in the process of becoming known as a global hub for applied behavioral economics research. With the support of the bank, I moved on to join Dan Ariely and our other partners to help build BEworks.

What type of research do you find most interesting, useful or exciting? This is a very difficult question! Every day is interesting and exciting and presumably useful! We continue to enhance our methodology. The incredible thing about behavioral science is it is endlessly refining what is understood about humans since there is a myriad of ways people are both rational and irrational! We launched our Diagnostics Toolkit in 2010, and after extensive research we recently launched a more comprehensive version. And, of course, seeing the results of our hypotheses validated through experiments is the most exciting part of what we do.

We also recently launched our Behavioral Economics Lab. We’ve started to conduct primary research in areas that we think are important or interesting. For example, we are in the midst of a series of experiments on retail investor risk appetite. Our hypothesis was that the conventional approach to measuring investor risk appetite is fraught with biases. We were able to demonstrate with simple decoys that investor risk appetite is quite malleable and prone to framing effects. This disutility is disconcerting because it gives investors and their advisors bad information about what financial strategies to pursue. We are excited that industry partners, investor education organizations, and regulators are very interested in our research. Our next step is to design and experiment with prescriptive solutions.

Do you see any challenges to the wider adoption of decision making psychology in your field? We have criteria for the kind of client we work with! We know that it’s hard for people to change and a number of things keep business leaders and policymakers doing things the same old way. But once leaders learn how to run their own experiments instead of relying on past experience, intuition, or outside experts who say they have all the answers, strategy formulation isn’t the same. Our clients have to be ready and committed to a scientific approach – both in the knowledge we bring to the table and the empirical approach to our work.

An interesting trend will I think work in our favor. The “quantified-self” movement is encouraging people to generate data and statistics in their everyday lives – how much time is spent in REM when they sleep, how many steps they take, and miles they drive. It is much easier now to be empirical in our everyday lives thanks to incredible technology innovation. Once people start looking at things with an empirical lens, relying on intuition becomes less satisfying. Most businesses struggle to make sense on the data they are gathering and giving it a purpose. The next natural step, which is where we can help, is grappling with how to employ this data to change behavior.

How do you see the relationship between academic researchers and practitionersThis relationship is the foundation of our company. Our team is a collaboration of academics and business consultants. Each partner brings a background of successful academic/business partnerships. In addition to our core team of experienced associates, we also have a strong team of interns currently pursuing degrees in psychology, economics, and public policy, so this adds to our bench strength. Our process is a virtuous circle of learning. The academics are committed to expanding the theoretical understanding of human nature. The practitioners like to see if and how these ideas hold in the real world which in turn provides further fodder for theoretical research. This integrated approach allows us to develop ideas that are both innovative in theory and in practice. We are growing the business by adding researchers who want to try and apply their academic pursuits with willing clients, and business people who aren’t afraid to set current practices aside. Plus the academics love playing with our large data sets.

What advice would you give to young researchers who might be interested in a career in your field? Like academia, the business world has its own language with arcane words like “solutioning” and “concretize” and concepts like “value-add” and “straw-dogs.” Just hang in there! You’re saying the same thing: modulations are “tactics” and findings are “results.” And there is similar methodological thinking to problem solving that was brought into business by a fair number of folks with engineering degrees. I believe that social scientists bring the same level of analytical thinking and rigor from their work with experiments and statistical analysis, plus they bring the evolving universe of cognitive and social psychology, and neuroscience.

We are teaching many businesses what to do with social science PhDs and helping social science PhDs who don’t know how they can use their skills in commercial terms. To academics, our platform presents the classic answer to their real world questions:  I wonder if I tried this with real data and real people, what the outcome would be, and whether it could change the way people act?  Few companies currently research or experiment in the way that a PhD has been trained to do. This is the essence of how BEworks is trying to change the nature of how business and policy leaders develop their strategies.

Kelly is also one of the speakers at a dinner organised by InDecision at the annual conference of the Society for Judgment and Decision Making in Toronto. The informal dinner will follow the Graduate Student Social Event (6.45pm to 8.45pm) Saturday 16th November at Joe Badali’s restaurant, a 5-minute walk from the conference venue. 

The informal dinner is an opportunity for graduate students to hear from practitioners on how they are applying JDM research in their work – other speakers include pricing consultant and writer Leigh Caldwell from The Irrational Agency and Paul Sas, principal research scientist at Intuit. 

Places are limited so please email to secure your place in advance. Some remaining spaces may still be available on Friday at registration desk on arrival at the conference. (For more details on either event please contact

Website | Twitter 

In The Wild: Tom Ewing

tom ewingNext up in our series of practitioners embracing the world of JDM research is Tom Ewing, Chief Culture Officer at market research agency BrainJuicer, where he works in the Labs team, helping translate the findings of decision science and psychology into methods that create business advantage for clients. His background is as an Internet analyst, social media researcher and journalist. His 2012 paper for BrainJuicer, “Research In A World Without Questions”, looked at the possibilities of observational and behavioural research in a commercial context, and it recently won the ESOMAR Excellence Award for the best market research paper of the year.

Tell me about your work: how does decision making psychology fit in it? BrainJuicer is a commercial market research and behaviour change company whose mission is to take advances in human understanding and to turn them into commercial advantage. And “human understanding” means behavioural economics, psychology, and decision science.

We want to create behavioural change for our clients. For commercial clients, this means applying the behavioural sciences to a brand owner’s problems and creating opportunities for them and their retail customers. For public service clients, this often means changing behaviour for healthier outcomes. For shoppers, customers, users of services, this means making decision-making faster and easier, and often making it more enjoyable too.

So our Behaviour Change Consultancy will take a client’s brief, understand the behaviour they wish to change and create behavioural activations that we test experimentally to demonstrate their effect.

Our research approaches support our goal to change behaviour for our clients, and are designed to “reflect and predict what people will actually do”, rather than what they think they do and say they will do – the standbys of traditional research. For instance, we put people under time pressure to recreate fast, System 1 decision-making in packaging and promotions research; we harness people’s social sense to understand the likely success of new product launches; we establish how people feel about advertising to predict its efficiency. And much as we like to test iteratively in our behavioural work, we like to re-test our recommendations to clients to demonstrate the value that we can bring.

How did you first become interested in decision making psychology? On a personal level it’s a natural fit with the curiosity that inspires most market researchers. First of all, you’re curious about what other people do, then you’re curious about why they do it. And then you realise that the stated reasons aren’t actually getting you very far and you want to dig further into how things really work.

As a company BrainJuicer has had an interest in consumer psychology long before I joined – we’ve been doing emotional ad testing since 2007, and tapping crowds for concept testing since 2004. Putting behavioural economics at the heart of our offer has been exhilarating for us as a company and fits with our conviction that market research has been getting consumers wrong for years – putting too much trust in claims and norms and not being curious enough about what people actually do.

What type of research do you find most interesting, useful or exciting? There’s often a gap between the interesting and the useful! Behavioural economics is made up of such a horde of studies, biases, heuristics, and findings that it feels initially like a game of Pokemon: you gotta catch ‘em all, and it seems almost impossible. In order to make it useful you have to make it accessible and tangible to non-specialists – which means you have to streamline it. We use a “Behavioural Model” which uses broad categories of environmental, personal and social influences on decisions that make sense to clients.

The idea is always to get from theory to action as quickly and easily as possible. So the work that leaps out at us tends to be the field experiments that help us to illuminate and bring the thinking to life – real-world test sites, ideally measuring real money changing hands at some point. That’s the arena we’re looking to play in, and frankly those are the findings which get us and clients most excited.

We are fans as well as practitioners. I still love a beautifully constructed experiment or unexpected finding. But it doesn’t really match the satisfaction of being able to change behaviour for our clients; to show how we might reduce hospital infections resulting from poor hand hygiene or to demonstrate how we might reduce binge-drinking.

Do you see any challenges to the wider adoption of decision making psychology in your field? Yes. The long term challenge is pretty similar to the one that faces economists trying to turn around textbook economics thinking. You end up with lots of acclaim and a few prizes but people still make the same mistakes based on the same bad theories. Changing behaviour is hard, and it doesn’t stop being hard just because you know about behaviour change. Industrialised market research has twenty years of norms which exert a powerful and reassuring pull on decision makers, even though they’re based on completely faulty models of how decisions work. We can’t talk about fast and easy decisions without facing up to the fact that choosing the existing option is the very definition of one!

The short term issue, I think, is that there’s an awful lot of excitement at the moment around technology – the power we now have to collect behavioural data. New technology is sexy, easy to adopt and an easy incremental step to take; changing your whole worldview is difficult, breaking habits is hard and systems are in place that make change difficult. So it’s understandable that technology often seems of greater interest to the industry than decision-making science. Who needs psychology when you have big data? Well we do, and more than ever. You absolutely need a thorough grounding in psychology to explain behaviour and tell you how to change it.

How do you see the relationship between academic researchers and practitioners? For BrainJuicer, it’s been mutually beneficial. Our Behavioural Model and the thinking that underpins our products has been developed in conjunction with academics. But you can’t change behaviour through pure argument and persuasion. If we are to change the behaviour of marketers, advertisers and other people in the research industry, we need to make the case for behavioural economics as engaging and as seductive as possible. I am firmly on the side of the popularisers over the purists.

Our behaviour change projects often involve extensive literature reviews by academics. We read a lot ourselves and have a database of studies with proven real-world effects. If it wasn’t for the academic research there would be no practitioners – we stand on their shoulders and we have to do right by them. And as practitioners it’s our job to apply the theory and make it matter.

What advice would you give to young researchers who might be interested in a career in your field? I think at the moment a background in decision science would be an incredible asset for a commercial research company – particularly if you’ve got experience in setting up experiments and how to properly control them. Market research has always been a melting pot of a profession – it’s drawn in psychologists, anthropologists, statisticians, technologists, arts graduates – and while it’s slightly more professionalised these days there’s still a thirst for relevant experience among the smarter companies. But we also need creatives, illustrators, designers, statisticians, writers and speakers to apply the theory, check it works and make it famous. So jump in, it’s an exciting time!

Twitter | Website | Blog

Guest post: Michael Blastland on Uncertainty

michael blastlandThis week we have a guest post from journalist, broadcaster and author Michael Blastland. In addition to creating the BBC 4 Radio programme ‘More or Less’, he has authored several books including The Tiger That Isn’t (published in the US as The Numbers Game: The Commonsense Guide to Understanding Numbers in the News, in Politics, and in Life) and The Only Boy in the World, about his son’s autism. He is a well-known campaigner for statistical literacy. His most recent book, The Norm Chronicles: Stories and numbers about danger, looks at the risks of everyday life and how to decode them. 

People tend not to like uncertainty. It’s confusing. It makes our choices riskier. What are we supposed to do when we’re not sure what’s going on?

No, if it can be nailed down, nail it. If it can be settled, sort it. And even if it can’t, maybe any answer is better than none. Faced with the stranger on the moor who says the true path is definitely this way, or the one who says ‘not sure, maybe over there somewhere,’ which do you choose?

For the stranger on the moor substitute the political leader, or the business leader. We like people who seem to know.

Then, a few weeks ago an old friend, Oli Hawkins, said he’d had an idea.  


What’s more, it was an idea about how to show the uncertainty in data.

Hazardous understatement.

More accurately, it was an idea about how to bring uncertainty to life so that we see its full extent and implications.

And I thought: this is brilliant; some people will hate it.

What I think Oli had done was to find a way of making statistical doubt more visible. This is no small trick. In doing so, he might have helped us see the world differently. But there’s also little doubt that it makes life less comfortable.

The nub of the problem he has been trying to overcome is, in a word, pictures.

I agree, that doesn’t sound like a problem. In fact, pictures are often the answer to the problem of how to interpret data. They can crystalize ideas and make vagueness vivid. Turned into pictures, numbers escape the fog of evidence for the blue sky of clarity. We take in so much more from a picture than from columns of data, we spot patterns, faster, we remember the picture, it can even be beautiful.

As with a character in film compared with a character in a novel, the wry smile and the twinkle in the eye is given settled form. For some of us, it’s hard to stop thinking that James Bond is Sean Connery.

‘So?’ you say. ‘What’s wrong with that? Isn’t this exactly what visualisation strives to do.’ Well, sometimes there’s nothing wrong at all. Sometimes it’s fab.

And sometimes it’s fantasy. Especially when the ideas themselves ooze doubt, when vagueness and uncertainty might be half the point, when the numbers are more mush than concrete.

I’m a huge fan of visualisation. Who isn’t? But uncertainty is visualisation’s portrait in the attic: a dodgy secret, an orthogonal truth, in keeping with the human tendency to avoid it.

How to say that the line is most likely here, doing this, but could be way over there doing that? This has never, in my view, been satisfactorily sorted. The understandable tendency of a lot of data-viz is to ignore it.

On those occasions uncertainty is acknowledged, a standard approach is the error bar. Here’s an example from Oli’s discussion of the problem:

blastland 1

‘The margin of error’ he says ‘reflects the 95% confidence interval for the estimate, which means there is a 95% chance that the actual value is within the range shown by the error bar and a 5% chance that it is outside this range. The size of the error bar is determined by the size of the sample on which the estimate is based.’

But as Oli points out, the error bars simply follow the trend.

They move up and down in a neat little dance either side of the central estimate, and our eyes follow, as if all estimates dance in the same direction. In fact, the true value might lie at any point along those error bars, or beyond, though with diminishing probability. That is, the true value could be at the top of one error bar and the bottom of the next. So this visualisation – improvement though it is on a plain bar chart – arguably obscures the potential movement.

Another example is the Bank of England’s fan charts for GDP, which apply both to future estimates and, more to the point here, to GDP in the past, about which we also remain uncertain. These fan charts show a range of estimates of the true value, in bands of probability.

They’re good. I like them. But they have exactly the same problem. All estimates echo the central line and visually reinforce our impression of the trend. Not the idea at all.

blastland 2

What we tend to ‘see’ in this chart, I think, is a rise and then a fall in the rate of growth in the past few years that might have happened higher or lower than the central estimate, but was basically in lockstep with it. And people draw all sorts of conclusions from that supposed trend about the conduct of economic policy.

But is it true? Because what could have happened is that the rate of GDP growth rose continually since 2009, as it swung from the bottom to the top of the Bank’s range of estimates. Rather than an economy that skirted double or even triple-dip recession, maybe we had an economy going from strength to strength for more than three years. Or maybe it was the other way round and we recovered spectacularly in late 2009 and then slammed into reverse and another shallow but protracted recession.

You’ll find little economic comment to this effect, and it’s not the Bank’s nor the ONS’s best guess, but it is perfectly within what the Bank thinks are reasonable bounds of uncertainty. Maybe one reason this discussion doesn’t happen, and the doubts tend to be smothered in the rush to an appalled/euphoric (delete as applicable) reaction, is because we don’t have the right way of showing their extent.

And fan charts like these are a relatively recent innovation. Before them, the lines were even more concrete.

There are other techniques for representing uncertainty. Howard Wainer’s ‘Picturing the Uncertain World’ is an interesting exploration of the subject. But we can, and should do more.

‘You know…’ I say, trying to inspire audiences of designers, ‘you have an opportunity here to work out how to use visual techniques to bring uncertainty properly to life. Do that, and you could help people see, maybe for the first time, the way that statistical evidence relates to real events. This could change the way we see the world.’

But if that sounds too much like hard work, well then, as I’ve put it elsewhere, we can always carry on with the same old statistical blah… only prettier. As Tim Harford has said, mis-information can be beautiful too.

My own attempt at the uncertainty problem was to make some fantasy league tables in which the position of each imagined school, or hospital, or whatever, bounced up and down randomly within the confidence intervals, moving up and down all over the shop. Who really ranked where? You couldn’t be sure. Which is irritating, but often as it should be.

But how to make this movement proportionate to the real probabilities? Cue Oli. He has found a way to animate the estimates within the confidence intervals so that they pop up just as often as probability suggests they should – given the data. He shows that this can be done with interval data so that we discover how different a trend might look over time, as well as with categorical data – like the school league-table example. He’s done it as a series of snapshots rather than a continually fluid movement, which helps pick out more clearly what the true trend might have been.

And…? Isn’t all this obvious? If that’s what you think, you’d be right in the sense that it is all implied by the existing maths of confidence intervals.

The answer may be that all that is new here is the articulation of an idea. And it may be true that the idea is already latent in the prior concept of confidence intervals. So what’s the big deal?

The big deal for me is that an idea that is latent – except in the minds of a few – isn’t an idea at all for the many. Articulating it is every bit as important as knowing it. I would say that, being in the communication business. But maybe the proof of how important it is to articulate these things, and also the proof of how well it’s been done to date, is how little there is in public argument about the extent of the uncertainty around numbers like these or what that uncertainty implies. If the idea is obvious, where’s it been?

Now you could just put that absence down to the ignorance of the commentariat and politicians, or you could add that maybe we could do it differently.

The acid test is what we see with the new method. Applied to the migration data, the effect is electric. Here are a few grabs from Oli’s visualisation as it runs through the variety of stories that could have been told.

Like this one…

blastland 3

Fairly flat, bit of a crest around 2010 maybe, maybe a hint of a rising trend – though this could be no more than a couple of weird years. Nothing to my eye leaps off the page over the long run.

Or like this.

blastland 4

Which looks pretty clearly like a step change in 2004. The numbers roughly double. A good one for those who want to say we ‘lost control of the borders’ and a sharply different reading of history.

Or what about this?

blastland 5

In which the key date moves back six years as we see a broadly rising trend all the way until about 2010, when ‘determined action by the Coalition finally brought it under control,’ presumably.

Or like this, when determined action by the Coalition since 2010 made hardly any difference.

blastland 6

Just click and play to see the variety of stories that could be true. The implications of the uncertainty are easier to grasp and harder to ignore. What also emerges is that some stories are more common and consistent than others. Very few iterations show 2012 higher than 2010 for example. So we see both what is most uncertain, and what is most likely. It’s not at all the case that the upshot of all this is to throw up our hands and say we’re clueless about what happened.

Not new? It’s revelatory. What if we did it to the GDP lines on the Bank of England’s fan chart, and animated them through a range of possible stories in all their top-to-bottom potentially volatile variety? What if we did the same to the monthly unemployment data?

Yes, it’s disturbing, destabilising, unsatisfactory in so many ways. It makes the world less nailable, less sorted. And I love it.

What’s especially thought provoking is that it makes you wonder how many more techniques there might be that could bring life to statistical insights, rather than bringing design or false clarity to dodgy data.

Don’t get me wrong. I think there’s some fantastic stuff out there. And anyway, uncertainty isn’t always a big factor. All the same, data visualisation is no more than a fancy distraction if it doesn’t help us see better. But when it does…  wow.

Norm Chronicles interactive site

Profile in the Guadian

Outside The Matrix: Florian Bauer

Following on from Kiki Koutmeridou, we’ll continue this week with another Outside the Matrix interview: Florian Bauer from Vocatus AG in Germany, who studied psychology and economics at the Technical University in Darmstadt, at MIT, and at Harvard University. He has devoted himself to research into behavioural economics and the psychology of pricing, which were also the subject of his doctorate (“the psychology of price structure”). Starting his career as a strategy consultant at Booz, Allen & Hamilton 1996, he joined with two colleagues in founding Vocatus AG (a full-service market research and consulting company) in Munich in 1999. He’s also a member of the board of the German Market Research Association (BVM), and regularly teaches as a visiting professor at several universities in Germany. In 2005 and 2010 he won the ‘German Market Research Award‘ for the ‘Study of the Year‘ and in 2010 the ‘Best Methodological Paper Award’ at the ESOMAR Congress (global market research conference), and has subsequently won the the ESOMAR “Research Effectiveness Award” both in 2012 and 2013. 

Tell us about your work: how does decision making psychology fit in it? Well, all I do is in fact decision making research. I see market research as nothing else than trying to understand the basic building block of an economy – the customers decision making process. And here, there is no better theoretical and methodological basis than behavioral economics even though this is often neglected in classic market research approaches.

Why you decide to go into industry instead of continuing in academia? Well, it was primarily “anticipation of regret”. I had a hard time deciding which path to follow. The reason why I picked business was for one part the idea that I could regret it later not having taken the chance to start my own company. For the other part it was the fact that I really wanted to apply the stuff I was doing and put it to test in the real world. Still today, this a thrill to me.

What do you enjoy the most in your current role?  Do you see any challenges to the wider adoption of decision making psychology in your field? I love that I can do what I like most: Focusing on applying behavioral economics in marketing in general and pricing in specific. I love that we were able to attract a team of more than 70 colleagues that share the same interest and want to rock the boat. The only challenge I can see is the reluctance to adopt new approaches when the old ones are still massively promoted by large international research agencies. But quite frankly, the solution to this is to seek for more innovative clients that are willing to switch gears and go beyond the classic market research approaches. And that works quite well.

How do you see the relationship between academic researchers and practitioners? I think the perspectives are extremely different although they could profit much more from each other. While academia is focusing on a specific effect, on a specific theory, and the analysis of different ways of looking at the issue, practitioners are focusing a broader array of different questions. Where in the end they have to make a recommendation fast and still good enough.

What advice would you give to young researchers who might be interested in a career in your field? Test and decide, maybe try to do academic and market research in parallel. In any case, find your own way and do not focus on traditional career paths.


Outside The Matrix: Kiki Koutmeridou

Kiki_20130929135455608Third in our series of those who moved into the private sector after completing their PhD in decision making psychology is Kiki Koutmeridou – a behavioural economics researcher within GfK NOP, a global market research agency based in London. She has a background in Psychology (BSc) and Neuroscience (MSc) and she completed her PhD in Cognitive Psychology at City University in 2013 focusing on memory and the strategic processing of retrieval cues. In her role as the head of the Centre for Applied Behavioural Economics, Kiki works in collaboration with City University, GfK and clients trying to explore how behavioural economics can be incorporated in the traditional market research.Since joining GfK NOP London in September 2012, Kiki has introduced behavioural economics theories to numerous research projects which focus on the application of academic findings to real-life situations.

Tell us about your work: how does decision making psychology fit in it? I’m currently the head of the Centre for Applied Behavioural Economics at GfK NoP, part of the GfK Group, an international market research organization. The Centre for Applied Behavioural Economics is a partnership between City University and GfK NoP in an effort to promote applied knowledge in the decision-making field. So, by definition, my work is all about decision-making psychology. I’ve just completed my PhD in cognitive psychology and more specifically in memory. When the opportunity presented itself to explore human decision-making behaviour in an applied setting, I didn’t think twice and have been working at GfK for two years now.

My role at GfK is two-fold. I contribute to the various client research proposals across the company by integrating the academic knowledge on decision-making into the suggested research design. I’m looking into ways in which the client’s research question can be answered via the various theories and findings from the behavioural economics field. For this purpose, I help at all stages of the project (experimental design, client meetings, field work, data analysis, presentations). In addition, I work in unison with several external (academic or not) collaborators to conduct fundamental research promoting applied knowledge of decision-making behaviour. As a consequence, we are in a position to subsequently approach suitable clients, to share our findings with them and to make a proposal that would be in their best interest.

Why you decide to go into industry instead of continuing in academia? Actually, I don’t think I’ve made such a decision. I haven’t excluded one for the other (yet!). Like I said, the Centre for Applied Behavioural Economics is in strong collaboration with City University. I spend a day per week at City University, where I finished my PhD, meeting with academics, discussing potential projects and visiting the library. Being still part of an academic institution gives you opportunities for collaborations, fruitful discussions and knowledge sharing. Being part of the industry gives you the chance to apply all this knowledge in the real world and observe the outcome. I consider I get the best of both worlds.

What do you enjoy the most in your current role? My role is not restricted to market research. On the contrary, I explore ways in which people can make better decisions in a variety of settings (consumer, health, financial etc…). What really thrills me is the opportunity to either apply the academic knowledge in the real world or derive new knowledge from the applied experiments towards this end. This is a two-way street that can change the status quo of how things function. The idea that I can be part of these changes gives meaning to what I do and great satisfaction.

Do you see any challenges to the wider adoption of decision making psychology in your field? While there is great conversational interest about the academic findings and some recognition of their benefits, it can at times be a challenge to encourage clients to move beyond tried and tested approaches. When I first joined the market research industry I was surprised that psychology wasn’t incorporated more in the everyday business. In every meeting about any project, the discussions were ringing bells about possible psychological theories that could be applied. But experimenting is often not on the table. However, Applied Decision-Making or Applied Behavioural Economics if you like, is still at its infancy. The challenge is to provide strong evidence of its benefits. It’s a matter of finding the right people, in the right places that can promote this line of research and highlight the benefits of decision-making psychology and its methods until they become part of the norm.

How do you see the relationship between academic researchers and practitioners? In a word: complementary. Academics and practitioners bring different but equally important elements into the equation. My current role is an example of just that: the academic environment provides new findings, old and new theories and innovative methodologies; businesses offer the opportunity to apply all this to the real world and they can provide large sample sizes (the nemesis of the academic world along with the funding). In addition, practitioners have hands on knowledge of the effects that academics describe. Collaboration between the two can only lead to better formulated, more accurate theories and predictions about human behaviour.

What advice would you give to young researchers who might be interested in a career in your field? The irony is that I’m in need of that advice too as a young researcher myself! However, based on my experience so far I have 3 suggestions

  1. Seize every opportunity as you never know where it might lead. I started working at GfK as a part-time data analyst. If you had asked me back then I wouldn’t be able to foresee my current role.
  2. Be open-minded. Nowadays, the boundaries are hazy and every field can be combined with just about any other. Do not limit your imagination about potential new applications or approaches.
  3. Be confident and proactive. There isn’t one right way of doing things so always voice your opinion. You are not supposed to know everything and quite frankly no one does. Remember that we learn more from our failures than from our successes. The important thing is to keep trying to find the answers and to keep reading around your field of interest. Brain is like a muscle – keep it fit!

Also from GfK NOP: interview with Colin Strong (In The Wild series)

Research Heroes: Colin F. Camerer

camererThis week we continue our Research Heroes series with Colin F. Camerer, who is the Robert Kirby Professor of Behavioral Finance and Economics at the California Institute of Technology. Before joining Caltech, he earned his PhD from the University of Chicago Graduate School of Business at the age of 22, and subsequently worked at the Kellogg, Wharton, and University of Chicago business schools. He’s published more than 50 articles and a book on behavioural game theory. He’s a past president of the Economic Science Association and in 1999 he also became the first behavioral economist elected as a Fellow of the Econometric Society. He’s also just become a recipient one of the 24 annual MacArthur Foundation Fellow grants.

I wish someone had told me at the beginning of my career… Learn more math! I was good at math but didn’t appreciate how important it is to learn it when you’re young. Math is central to applied economics and could be used more in JDM psychology. The same is true for statistics— knowing a lot of tricks helps you get the most from data, win arguments, and figure out what you do and don’t believe in other people’s papers. 

I most admire academically… (With apologies to many whom I’ve forgetfully excluded)  Dick Thaler, for setting a good example by writing a small number of papers on important questions, and making each a gem. Danny Kahneman for being so wise, getting wiser every year (how does he do it?) and writing so beautifully. Amos Tversky, for a steeltrap mind and tenacity in digging on a topic until he had it figured out and expressed in a simple formalism. George Loewenstein for his gift of synthesizing lots of ideas and examples into an insight in a way that is very fruitful for others to then pursue. Gary Becker for seeing the interesting economic elements in so many kinds of choices (like having children, and crime).  The economist Bob Shiller for being eclectic and for daring to write aggressively about the role of social forces in asset pricing (which everyone else thought was crazy and unmodellable but now is starting to gain traction). I also admire a lot of people JDMers may not have heard of in other fields. One is Joe Henrich, a cultural evolution anthropologist who did the first economics experiment in a small-scale society, which then led to an influential cross-society project. Three more are: Duncan Watts who knows a ton of things about social networks, Mike Kearns, a computer scientist who recently became interested in experiments on networks and problem solving, and Peter Dayan, a “dry” theoretical neuroscientist who is always coming up with remarkable bold ideas.

The best research project I have worked on during my career… If you’re doing it right, you almost always have the very genuine feeling that the paper you just finished is the best one (even though you had that same feeling N-1 times before). One of the best was our paper on taxicab driver labor supply (QJE 1997). It was a really simple insight and one of the earliest clear tests, outside of finance, between a behavioral alternative and a very standard economic idea—that labor supply curves slope upward (i.e., workers put in more hours when wages are higher). I was living in New York at the Russell Sage Foundation so off to the Taxi and Limo Commission I went. There sat a bored economist whose main job is to collect statistics so they can justify taxi fare increases every couple of years. It turned out they had done some studies asking drivers for information on the hours they drove on different days, so I left with a (free!) floppy disk full of data from them. We did not have any formal model in the paper, but others came along later, figured out the proper way to model it with reference-dependence, and replicated our basic finding.

With that paper, we also had a mixed editorial experience with a happy ending. We sent it to American Economic Review, where we ended up getting one silly short report basically saying “I don’t believe it” and mentioning measurement error, which we had addressed very squarely (with a good “instrumental variable”).  A lot of economists were (mindlessly) hostile back then. We withdrew it and submitted it to a special issue of the QJE honoring Amos Tversky and got incredible help from the editor there (Larry Katz) who is an outstanding labor economist and told us exactly what to do.

The worst research project I have worked on during my career… The worst was the first experiment we did in Charlie Plott’s class in winter 1980 at Chicago GSB. Charlie was an incredibly patient and generous teacher, so he required us to actually run an experiment. We were interested in finance at the time so we created an experiment to test whether specialists in stock markets would smooth prices as they are supposed to do in theory, by buying during price drops and selling during price increases.

We made every possible mistake. First, there was only one specialist per session and a lot of live traders, but only the specialist’s behavior was interesting. So the design had incredibly fragile internal validity—a distracted or confused specialist would just produce terrible uninteresting results. The instructions were a mess. And of course we did not plan well so 10 minutes before the experiment we were in the library– a 5 minute walk from the lab– Xeroxing the instructions. Now I tell students that their first experiment will be their worst—hopefully!, since there is a learning curve—so they should just pick something and get started, rather than fret and ponder endlessly trying to make it perfect.

The most amazing or memorable experience when I was doing research… Probably the most memorable was a paper exploring whether you could create herd behavior in a horse race betting market. At the time, people in economics were just beginning to formally model “cascades”, in which you observe decisions other people make— like a crowd outside a new restaurant—and decide how to combine your own belief with what you infer from the crowd.

By mistake I once put in a ticket for a race that had not been held yet, and the terminal screen came up “Do you want to cancel your bet?” So I realized you could make bets and cancel them before the race. Then I got the idea to make large bets on a horse, $500 or $1000, and see if those bets influenced others to bet on the same horse (herding) or to stick with their own hunches and bet against me. Either result would be interesting.

It was fun to actually make the bets and see what happened. It was a matched-pair design in which races with two similar horses were picked, and I literally flipped a coin to decide which of the two to bet on (the other one was a within-race control).  I had a little notebook and wrote down the betting totals every minute, it was fun being like a naturalist in the economic wild. It was also nerve-wracking because half the betting happens in the last three minutes, so there was always a chance I would get stuck in a slow line and not cancel the bet in time. Imagine having to explain to the university accountants why I needed to be reimbursed $1000 for a bet at the track!?

The one story I always wanted to tell but never had a chance… In graduate school and my first two assistant professor jobs, I had a small independent record label. I always wanted to write a short casual paper on behavioral decision making and valuation under ambiguity in businesses based on my experience. It was fun and actually made a bit of money, which was a miracle.

A research project I wish I had done… A few years ago Dave Perrett came to Caltech and showed some beautiful work using facial morphing. After that a PhD student (I think it was Meghana Bhatt) suggested that maybe you could make people think about the future differently by showing them an aged version of their own face.  We were lazy about actually doing it. Hershfeld et al. 2011 actually did this.  The general method of facial morphing could be used in lots of other JDM research, too.

If I wasn’t doing this, I would be… I would be a photojournalist or a documentary filmmaker. My first job after college was working for a beach newspaper in Ocean City, MD. I loved the idea of taking pictures and had an excellent semipro photographer coaching me. (This was in the old days where serious photographers would develop the film in a darkroom, in a chemical bath—it was tedious but cool!) Sadly, my pictures were terrible. At the very end of the job we discovered there was a light leak in the camera (sadface) so my pictures weren’t so awful after all. Anyway, pictures of dramatic events, especially political events and war, can be so riveting and important (like Nick Ut’s famous picture of the napalmed Vietnamese child running down the street). Documentaries can make the same impact in a longer form. And they are actually surprisingly profitable as a whole because they are cheap to make and because of the long tail from the possible huge box office gross.

The biggest challenge for our field in the next 10 years… In my view, probably the biggest challenge and opportunity is to make use of the amazing change in accessibility of new field data (so-called “big data”).  Economists have a head start on this because most of the data they work with are not experimental or survey data they produced, so they are well-equipped to find data and get answers out of it. Computer scientists are looking at these data too, and they have a huge edge in being able to get data (e.g. scraping websites etc.) If JDMers are stuck only in lab mode we will miss an opportunity to use both field and lab data to study robustness, whether interesting effects evident in short lab experiments persist over longer periods of time, and so on.

Keep your eyes open for where data are available. Lots of useful data are available from the web. In the US, the Freedom of Information Act (FOIA) requires governments to release any data they collected unless it’s classified. Many nonprofits and government agencies are interested in using behavioral science to make sense of what they do, and they are often eager to publish results (whereas companies may consider findings intellectual property and don’t want to publish it in order to keep it private). Tech companies like Google, Microsoft and Facebook have big research groups looking at their internal data and like having people spend time there as interns etc. A lot of people they hire are computer scientists who can be quite clueless about psychology and social science. JDM could add a lot.

Instead of thinking about what lab experiment to run, I hope some new researchers in JDM first think—what are the ideal field data to test my hypothesis?— then keep their eyes peeled for those data, including cold-calling companies asking for data. You can always run experiments as well if the field data are inconclusive about causality.

My advice for young researchers at the start of their career is… From a career point of view, it pays to specialize in a topic you find really interesting and explore it thoroughly using various tools. When you come up for tenure you want to be know as “Ms. Emotion and Risk” or “Mr. Overconfidence” or what have you. Don’t be shy about introducing yourself to senior researchers at conferences and sending them papers. Usually we won’t read the papers (or if so, not carefully enough to comment) but it gets your work into our memory.

Another important thing is to have a very clear understanding with your colleagues and department chair about what is expected of you to get tenure. Some places have very clear criteria, in terms of the number of papers and what journals count the most.

Another common mistake, in my opinion, is to invest too heavily in teaching pre-tenure. Teaching can be fun, you get positive feedback, and it’s deadline-driven. Research can be painful, frustrating, with negative feedback and no deadlines so that you can always procrastinate. To be very frank, as long as your teaching is adequate, research-oriented schools really do not care about teaching quality in making tenure decisions. If the colleagues who will be judging you say teaching does count a lot, get them to spell out what exactly that means and look carefully at the last 10 years or so of who actually did or didn’t get tenure. If star teachers with short vitas are getting fired that tells you what you need to do. When I was at Wharton business school there was a streak of people winning teaching awards then getting turned down for tenure just afterwards. It got so bad that people would start to worry if they won an award.

One more thing for women on the tenure track (and beyond): Many female colleagues complain that they get asked to do a disproportionate amount of service, such as serving on thesis committees, working on curriculum, recruiting, organizing speakers, and so on. Obviously these are activities that somebody has to do and you should feel obliged to do your share. The problem seems to be that women do too much. Maybe women feel more compelled to do it. Men seem to either not get asked as often or say No more often. It could also be that men do such a mediocre job that they get “punished” by not having to help out in the future.  While your tenure clock is ticking, you need to guard your research time fiercely (or enlist a senior colleague who can help you do that).

Departmental website

TED talk