Simple feedback does as much as a 360 review

The largest meta-analysis of leadership training found no advantage for 360-degree feedback over feedback from a single source, on any outcome it could test.

Shelves filled with rows of ring binders in different colours.
A multi-source report returns dozens of scored items. The evidence suggests the volume is part of the problem. Photo: Richard Heinen on Unsplash

Across 335 samples, leadership programmes that used 360-degree feedback did no better than programmes that used feedback from a single source. Not on learning, not on transfer, not on results.

The multi-source version is also the expensive one. A 360 exercise for one cohort of managers means designing or licensing an instrument, nominating and chasing raters, collecting responses, generating reports, and debriefing each participant individually. For a group of forty people that is a substantial piece of work before any development has happened. The premise is that peers, direct reports and a manager together give a fuller and less flattering picture than one source, and that a fuller picture produces more change. The first half is intuitive. The evidence for the second half is not there.

The largest training dataset tested the 360 directly, and found a null

The 2017 meta-analysis of leadership training by Lacerenza and colleagues pooled 335 independent samples and tested fifteen design features against four outcomes. Two of those features concerned feedback, and they were tested separately.

The first was whether the programme included feedback at all. The second, stated as a formal hypothesis, was whether 360-degree feedback outperformed feedback from a single source.

The result on the second was a null. Programmes using 360-degree feedback showed no significant difference from programmes using single-source feedback on learning, on transfer or on results. The authors record the hypothesis as unsupported.

Nor is this an incidental finding tucked into an appendix. It was one of the analysis's stated hypotheses, tested on the largest dataset the field has, and it failed on all three outcomes where it could be tested.

Feedback of any kind helps one outcome, behaviour at work

The first test produced a more interesting pattern than a simple endorsement.

Programmes that included feedback produced significantly stronger transfer, meaning behaviour change at work, than programmes without it. On reactions, on learning and on results there was no significant difference.

So feedback earns its place for exactly one thing, and it happens to be the thing that matters most. Whether people know more at the end of a programme is unaffected by whether anyone told them how they were doing. Whether they behave differently afterwards is not.

The asymmetry says what the feedback is for. It is not an assessment instrument. It is the mechanism by which a participant finds out whether the new behaviour is landing, which is a different job with different requirements.

No support was found for Hypothesis 10, as there were no significant differences between programs using 360-degree feedback compared to programs using single source feedback.

In coaching, multi-source feedback came with smaller effects

The leadership training evidence is a null. The coaching evidence goes further, and runs the other way.

Jones, Woods and Guillaume's 2016 meta-analysis of workplace coaching tested multi-source feedback as one of four practice moderators across seventeen studies. Coaching engagements using multi-source feedback produced smaller positive effects than engagements without it.

The authors had hypothesised the opposite, on the reasonable grounds that a coach is a better vehicle for multi-source feedback than a report handed over with no conversation. The data disagreed.

Seventeen studies is a small base and the finding has not been replicated. It is a signal, not a settled result. But it points the same way as the much larger training meta-analysis, and it came from a different team, on a different intervention, with a different method.

Needs analysis, practice and spacing showed the advantages the 360 did not

The findings do not say 360s are worthless. They say the multi-source instrument has not been shown to beat the simpler alternative, and in one dataset performed worse.

So the question becomes what else the same money could buy. In the same meta-analysis, the design features that did show significant advantages were a formal needs analysis, practice combined with demonstration and information, and sessions spaced across weeks instead of compressed into one block. Those are covered in what the evidence says leadership training changes.

There are good reasons to run a 360 that have nothing to do with the effect size. It can give an organisation a shared language about behaviour, a defensible input to promotion decisions, and a baseline for measuring change over time. Those are legitimate, and they are different purposes from development, to be funded and evaluated as what they are. The measurement question more broadly is in how to measure whether a leadership programme worked.

For behaviour change, the evidence supports feedback. It does not support paying more for multi-source feedback. It strongly supports the practice time that the 360 budget usually comes out of.

Why the 360 fails to outperform is still untested

Neither paper establishes a mechanism. Nobody knows why the multi-source version fails to outperform, and two accounts are consistent with the data.

The volume problem. A 360 report typically returns dozens of scored items across several rater groups. A participant who receives that has to work out what it means before they can act on it, and much of the debrief goes on interpretation rather than on what to do differently. Feedback from one credible source about one specific thing arrives already actionable.

The defensiveness problem. Aggregated ratings from anonymous colleagues invite a search for who said what, and a rating that conflicts with someone's self-image is easier to dismiss when nobody has to own it. A long-standing literature documents feedback interventions that sometimes worsen performance, particularly when the feedback is directed at the person and not the task. Anonymous multi-source ratings are close to the worst case on that dimension.

Both accounts point to spending less on collecting feedback and more on what happens afterwards. Which of them is right, or whether both are, neither meta-analysis could test.

Common questions

Does 360-degree feedback improve leadership development outcomes? No significant advantage over single-source feedback was found in the largest meta-analysis of leadership training, across learning, behaviour change and organisational results.

Is feedback useful in leadership training at all? Yes, for behaviour change specifically. Programmes including feedback produced significantly stronger transfer to the job than those without it. For participant reactions, learning and organisational results there was no significant difference.

What did the coaching research find about multi-source feedback? A 2016 meta-analysis of workplace coaching found that engagements using multi-source feedback produced smaller positive effects than those without it, the opposite of what the authors expected.

Should we stop running 360-degree reviews? Not necessarily. The evidence says they do not outperform cheaper feedback for development purposes. If they serve another purpose, such as a shared behavioural language or an input to promotion decisions, that purpose should be stated and evaluated on its own terms.

Why might 360-degree feedback underperform? Two explanations fit the data. A large scored report takes interpretation before it can be acted on, and anonymous ratings that conflict with a person's self-image are easy to dismiss. Neither has been tested directly in these studies.

How many studies are these findings based on? The leadership training finding comes from 335 independent samples. The coaching finding comes from seventeen studies, which is small enough that it should be treated as a signal rather than a settled result.

What has better evidence than 360-degree feedback? In the same meta-analysis: conducting a formal needs analysis before designing the programme, combining information, demonstration and practice rather than using any one alone, and spacing sessions across weeks instead of a single block.

Sources

  1. Lacerenza, Reyes, Marlow, Joseph & Salas (2017), Leadership training design, delivery, and implementation: A meta-analysis, Journal of Applied Psychology 102(12)
  2. Full text of the leadership training meta-analysis, Doerr Institute, Rice University
  3. Jones, Woods & Guillaume (2016), The effectiveness of workplace coaching: A meta-analysis of learning and performance outcomes from coaching

More from the Journal