Psychological Review, founded in 1894, is one of the most prominent journals in psychology today. Psychological Review focus on psychological theory and publishes papers that make important theoretical contributions to psychology. Associate Editor Prof. Susan Fiske helped us with more insight into the journal.

What makes you go “Wow!” or “Yuck!” when first read a submission? Clear statement of the argument in the title & abstract enable an immediate evaluation of an article’s contribution. It is amazing how often authors fail to be clear about their hypothesis and its significance.

What are the common mistakes people make when submitting/publishing? Failing to check whether the article is appropriate for that journal.

What are your best tips on how to successfully get published? Aha! Plus Evidence.  Good ideas, backed up by good science.

How can a young researcher become a reviewer? When is the best time during one’s PhD to start doing so? Participating in a grad-led journal club, reading and critiquing published articles. Telling one’s advisor that one would like some reviewing experience.  When asked, returning high quality, on time reviews.

What constitutes a good (i.e., well explained/written) review, from an editor’s standpoint? Or what makes one a good reviewer? Balanced, thoughtful, succinct.

How do you resolve conflicts when reviewers disagree? Reviewers often disagree because they are recruited for differing expertise. Editors must consider the inputs relative to the expertise and perspective of the reviewers.

What’s the best/worst way to react to a revise and resubmit, and worse, to a rejection? R&R needs to list, in a cover letter, each response to each suggestion, including to explain (rare) instances of decking to make certain changes.

Is there a paper you were sceptical about but turned out to be important one? Not that comes to mind.

As an editor, you get to read many papers and have an insight emerging trends, what are the emerging trends in research topics/methodologies? Interdisciplinary and global collaborations.

What are the biggest challenges for journals today? Maintaining humanity depute the volume.

Journal home page

More ‘Inside the Black Box’

The Journal of Behavioral Decision Making is a multidisciplinary journal publishing original empirical reports, critical review papers, theoretical analyses and methodological contributions. The Journal also features book, software and decision aiding technique reviews, abstracts of important articles published elsewhere and teaching suggestions. The objective of the Journal is to present and stimulate behavioral research on decision making and to provide a forum for the evaluation of complementary, contrasting and conflicting perspectives. These perspectives include psychology, management science, sociology, political science and economics. Studies of behavioral decision making in naturalistic and applied settings are encouraged. Associate Editor Frank Yates gives us his insights into the journal.

What makes you go “Wow!” or “Yuck!” when first read a submission? Papers that surprise me because of their conclusions or even their topics are the ones that most often make me go “Wow!”  So do papers that exhibit different ways of thinking about old topics, ones that force me to say, “I’ve never seen this idea before.  That is so cool.“  I must also say that I love manuscripts that clearly point toward ways that people can decide better than they normally do.  And if the authors can complement their messages with concrete and elegant illustrations, that’s even better.

I can honestly say that I have never said “Yuck!” in response to a submission, even to myself.  I realize that virtually every submission I have ever seen represents the culmination of a huge investment by the authors.  I would feel guilty if I didn’t acknowledge that investment.  On the other hand, certain aspects of some submissions do make me groan (“Oh, no!”).  Papers that are weakly motivated tend to do that, especially ones that report the results of studies that I infer began with nothing more than: “I wonder what would happen if we tried this manipulation.”  Groans are also evoked by papers that are unnecessarily hard to read.  This happens, for instance, when papers are overly abstract or are too tedious and too long relative to the significance of their messages.

What are the common mistakes people make when submitting/publishing? Well, whatever actions produce the kinds of groans I just mentioned are good examples.  An especially common error that authors make is assuming that the reader appreciates and knows as much about their focal research problem as they do.  This makes their papers dull and impenetrable.  One way to avoid this is to simulate the journal reading experience in advance of submission.  That is, recruit friends and colleagues who are similar to the journal’s audience but are naïve to the topic.  Then ask them to review and discuss with you in detail their interpretations of your work.  You are virtually guaranteed to uncover misconceptions that will amaze you: “Really?  That’s what you thought I meant?”

What are your best tips on how to successfully get published? My first suggestion is to somehow choose to work on problems that are easy to convince people are interesting and important to solve, and then solve them.  Our field is like baseball, where an outstanding hitter fails 2/3 of the time.  That being the case, successful authors necessarily must be unusually energetic and well organized.  That is, they must always be working on several projects simultaneously.  Therefore, despite the low “hit rate,” they maintain a steady flow of results that are ready to submit for publication.  My second tip is to learn to view and use reviewers as one’s collaborators.  Typically, reviewers are among the most knowledgeable people in the world concerning an author’s focal problem.  So why not exploit the expertise underneath their comments to sharpen your writing, your thinking, and your next studies?

How are reviewers selected? My goal is to have every submission read critically and constructively by 2-4 people who know more about an author’s research problem and related topics than just about anyone else in the field.  Some of these people are likely to be on our editorial board.  Many others will have been authors of articles cited in the manuscript.  Because it is a multidisciplinary journal, at JBDM, we make a special effort to have at least two different specialties represented on every team of reviewers.

How can a young researcher become a reviewer? When is the best time during one’s PhD training to start doing so? In my view, the best time for a PhD student to start reviewing is after he or she has developed expertise and credibility in a particular area of research and therefore would have something useful to offer and to gain as a reviewer.  Having successfully published in that area is usually a safe indicator of such expertise.  At that point, the student might be well advised to write to editors of journals that publish work in the student’s area of specialization, volunteering to review occasional submissions on particular topics.  The response is likely to be immediate and positive, since editors are always on the lookout for good reviewers.

Since reviewing is hard work and takes time, why would (or should) a PhD student want to serve as a reviewer?  What exactly is there to gain from doing so?  One reward is the sense of contributing to the advancement of the field at its cutting edge.  But the main advantage is the unparalleled potential for learning and inspiration.  The author, reviewers, and action editor for a journal submission are essentially an especially exciting (and consequential) expert seminar on a topic of great interest to everyone involved.  Moreover, all the members are highly motivated to get things right.  The reviewers and editor work really hard to make sure that they do justice to the author’s contributions.  In addition, no one wants his or her comments to appear foolish to the rest of the group.  Finally, the review process often serves to spark new insights and research problems in the minds of all the participants—authors, reviewers, and editors.

What constitutes a good (i.e., well explained/written) review, from an editor’s standpoint? Or what makes one a good reviewer? Good reviews offer valid analyses of the author’s ideas, reasoning, and methods, answers to this question: “Is this legitimate?”  In addition, though, for the benefit of the author as well as the action editor, a good review also clearly explains how the reviewer arrived at his or her conclusions.  The best reviews also provide helpful suggestions and guidance, including useful references and even design ideas that might help settle key questions that have been left unresolved by the author’s current efforts.  And good reviews are never mean-spirited.

How do you resolve conflicts when reviewers disagree? Although it is tempting to do so, I never rely on a simple “vote count” among the reviewers.  Instead, I try to understand why the reviewers disagree.  In my experience, more often than not, reviewers only appear to disagree because they are focusing on different aspects of the author’s work.  This frequently occurs because the reviewers’ own research programs have different foci.  So, relying on the specifics of the reviewers’ analyses as well as my own reading of the manuscript, I arrive at summary conclusions about the sensible disposition of the submission—acceptance, revision and resubmission, or rejection.

What’s the best/worst way to react to a revise and resubmit, and worse, to a rejection? This is an exceptionally important issue.  My sense is that such reactions are often critical determinants of many people’s career paths.  I have known numerous researchers who seem to have found rejections so demoralizing that they eventually abandoned their research careers.  But I have also known several highly productive investigators whose success seemed largely traceable to how they dealt with the emotions evoked by negative feedback in the review process.  Three elements of their strategies seemed to stand out and perhaps deserve emulation:

#1: Expect reviewers to identify weaknesses in your work, and accept that as a good thing; you have the opportunity to benefit from their expertise.

#2: Don’t allow yourself to brood over negative comments, even rejections; instead, immediately start working on your next move, be it a revision, a clarifying follow-up study, or the abandonment of a now-recognized dead end.

#3: If and when you revise and resubmit, in your cover/response letter, make sure to respond—respectfully—to every major reviewer comment; reviewers simply hate being ignored.

Is there a paper you were skeptical about but it turned out being an important one? That’s a funny question.  There definitely have been a few such papers, and you have probably read them.  It clearly would be unwise for me to identify them, though.

I assume that you asked the question because you wonder why my initial appraisals of such papers were “off” and what can be done to try to reduce such mistakes, or at least their impact. One basis for misappraisals, which seems uncommon, is that reviewers and editors sometimes misjudge the technical quality of authors’ reasoning and methods.  Given human fallibility, occasional misjudgments like that are inevitable.  I recommend that authors maintain vigilance for such mistakes and call attention to them in cover/response letters for submissions of revisions—again, respectfully.  Another reason that editors occasionally underestimate manuscript potential is that some authors prove to be unusually good at making marked improvements from one revision to the next.  They are especially adept at building on reviewer and editor comments.  Cultivating that skill seems wise.

As an editor, you get to read many papers and thus have insight about emerging trends.  What are the emerging trends in research topics/methodologies? Perhaps the most obvious trend is toward studies that focus on biological underpinnings, or at least correlates, of overt decision behaviors.  Our field has also seen a noticeable, and perhaps surprising, uptick in efforts to understand the role of time in people’s decision making.  There has been a good bit of excitement about the involvement of emotions in decision making, too.  Yet another noticeable trend has been toward assessing and explaining individual differences in decision making character and quality.

What are the biggest challenges for journals today? In my view, two related challenges are at the top of the list. The first is the increased volume of journal submissions. The second is the need for good reviewers to read and respond to those submissions, thereby accelerating the advancement of the field.  The problem seems to be exacerbated by increasing institutional pressures on potential reviewers to perform other duties.

More ‘Inside the Black Box’

Next in our Inside the Black Box series is Frontiers in Psychology, an open access journal that aims at publishing the best research across the entire field of psychology. Frontiers in Psychology publishes articles on the most outstanding discoveries across the entire research spectrum of psychology. The mission of Frontiers in Psychology is to bring all relevant specialties in psychology together on a single platform. Field Chief Editor Axel Cleeremans gives us his insights into this journal.

What makes you go “Wow!” or “Yuck!” when first read a submission? I go “Yuck!” instantly if the paper looks like it’s poorly written, if the figures don’t look good (see Tufte’s advice on that), if it contains typos, or if looks very verbose or boring. There is an important message there: If you don’t fine-tune the presentation of your findings, it’s as good as nothing.

“Wow!” can result from different factors. Sometimes it’s the finding itself — for instance, I find Geraint Rees’s recent demonstration that one’s experience of the Ebbinghaus illusion is inversely proportional the size of one V1 stunning. Other times it’s the sheer power of technique — Bonhoeffer’s applying two-photon microscopy to visualize synaptic growth in vivo is a good example of that. The cleverness of an experimental design is a further “Wow!” inducer; Jacoby’s process dissociation procedure, when I first read about it, definitely elicited a “Wow!” response from me. And then of course, I go “Wow!” when reading about impressive ideas. Rumelhart and McClelland’s PDP volumes made we go “Wow!” for years, as did Hofstadter’s “Gödel, Escher, Bach”.

What are the common mistakes people make when submitting/publishing? Submitting to the wrong journal. Making the story too complicated. Not having any story. Reporting uninteresting findings. Reporting uninteresting findings but trying to make them sound interesting. Failing to cite relevant work from many years ago that old editors know about.  Leaving typos in the manuscript. Ugly figures.

What are your best tips on how to successfully get published? Work on the most important issue in your domain. Build a good narrative. Papers that read like detective stories (and finish with an satisfying resolution!) are always good. Get the writing absolutely perfect. Of course, interesting and solid data. Simplify. Kill all the typos. Cite previous work. All referees first look for flaws because if any are found then the review is done and the referee can focus on something else. It is only when no surface flaws are found that the referee actually thinks about whether the paper is interesting…

How are reviewers selected? That very much depends on the journal. Some editorial systems are almost entirely automated, which has advantages (speed) but also disadvantages (relevance). Some editors hand-pick their referees based on different criteria (mostly, whether they think they know something about the topic and whether they think they’ll compose their review in time). Many systems offer referee suggestions based on keyword matches. Authors can also often propose referees themselves. This is a good idea as it speeds up the work of the editor, who will typically select referees both from the author’s suggestions and from his own pool of referees.

How can a young researcher become a reviewer? When is the best time during one’s PhD to start doing so? I wouldn’t do it too quickly — say, three years in your Ph.D. Reviewing an article is an important and difficult job. It gets much easier as your knowledge of the field grows and as your expertise at reviewing increases, but the first reviews you do are always very intensive jobs. You worry that you’ll be ridiculous in the eyes of the editor and the other referees. You worry that you missed a central point. You’ll spend days on your first review. On the other hand, knowledge of what’s going on in your field before it gets published can be invaluable — but for this, you can count on your advisor.

What constitutes a good (i.e., well explained/written) review, from an editor’s standpoint? Or what makes one a good reviewer? A good reviewer is a reviewer who turns in her review in time and who manages to discuss the paper from a neutral tone while clearly listing the issues that concern her, if any. And of course: Good reviews also contain a clear recommendation that is congruent with the listed points. Sometimes you get almost self-contradictory reviews. They begin by “This is a very interesting paper that uses clever methods” and finishes by “I recommend the paper be rejected”. This makes it almost impossible for an editor to use the review, as do reviews that contain too many subjective comments. Reviews should almost be written as though they were public comments, that is, with all the care one would use if one were talking in public about someone else’s work.

How do you resolve conflicts when reviewers disagree? That’s a tough one. I regularly receive conflicting reports, sometimes at either end of the spectrum (i.e. Referee #1 says “Reject”; Referee #2 says “Accept without revisions”). If both reports make sense (that is, it is clear both referees understood the paper), most typically, I will consult a third referee (which sometimes doesn’t help). When all else fails, you read the paper and make the decision yourself… (just kidding: editors read the papers, but then there is a difference between reading a paper and forming an expert opinion about it). It is worth mentioning here that some open access journals (i.e. Frontiers in Psychology) have adopted a completely different manner of resolving differences between referees, namely to ask referees and authors to interact until a consensus between referees is reached. Many conflicts between referees are solvable by iterated interaction — something that can be tough to achieve with the standard reviewing process.

What’s the best/worst way to react to a revise and resubmit, and worse, to a rejection? Revise and resubmit is pretty much the norm — it is exceptionally rare for a paper to be accepted right away. Dealing with rejection is understandably difficult. Your reaction to it very much depends on what you can attribute the rejection to. Being rejected from Science is not an indication that your research is not good; just that it’s not good enough, or not novel enough, or not interesting enough in the eyes of Science’s editors. You may think otherwise and feel wronged somehow, but it’s not your decision to make in either case, so it’s best to move on and submit to another journal. The worst case scenario is when you submit to a mediocre journal, wait for months, and find that your paper is rejected. If you really feel a “reject” decision was incorrect, it’s always a good idea to interact with the editor. As an editor, I only use “reject” when all referees agree that the paper is not publishable. Dealing with a revise and resubmit is easy: Just address all the points raised by the referees one by one and thoroughly. In the vast majority of cases, papers in that category will end up published;  it’s just a matter of taking all the points seriously and in detail.

Is there a paper you were skeptical about but turned out to be important one? Not that I can remember as an editor. A couple of my own papers as an author, though, had very difficult beginnings and turned out to be considered as quite important. Science is about data, but also involves rhetoric: Not only do the data have to be important, but you also have to present the results and their implications in a persuasive manner.

As an Editor, you get to read many papers and have an insight emerging trends, what are the emerging trends in research topics/methodologies? There is an important ongoing discussion on twitter, blogs, facebook, email and the press about the importance of replication in psychology. Developing methods that make it possible to analyze replication efforts properly, as well as promoting the publication of replication findings, are important issues. One of the most interesting methodological developments in this respect is the emergence of novel statistics based on Bayes’ ideas. I also continue to be impressed with the increased sophistication of neuroimaging methods — think MPVA for instance. Increased meta-data in all fields will also make all sorts of meta-analyses possible.

What are the biggest challenges for journals today? The challenges are not the same for traditional journals and for new, online, typically open-access journals. Some journals are more or less immune from challenges because of their extraordinary status in the field. The challenge for traditional journals is to stay relevant in an increasingly open-access, rapid-fire world: Interesting results are tweeted or otherwise shared almost instantly, and people want to download the relevant material freely and right away. The challenge for open-access journals is to accrue enough credibility. A challenge that faces every actor today, individuals and journals alike, is to find interesting ways of attracting attention. So much is published today (considerably more than even a few years ago) that it becomes a challenge to even find relevant material.

Journal home page

‘Inside the Black Box’ series home page