Without a problem, without a failure, without a flaw, without a frustration, innovation has nothing to latch on to. It loses its pivot. As [James] Dyson puts it: ‘Creativity should be thought of as a dialogue. You have to have a problem before you can have the game-changing riposte.’

Perhaps the most graphic way to glimpse the responsive nature of creativity is to consider an experiment by Charlan Nemeth, a psychologist at the University of California, Berkeley, and her colleagues. She took female undergraduates and randomly divided them into five-person teams. Each team was given the same task: to come up with ideas about how to reduce traffic congestion in the San Francisco Bay Area. These five-person teams were then assigned to one of three ways of working.

The first group were given the instruction to brainstorm. This is one of the most influential creativity techniques in history, and it is based on the mystical conception of how creativity happens: through contemplation and the free flow of ideas. In brainstorming the entire approach is to remove obstacles. It is to minimise challenges. People are warned not to criticise each other, or point out the difficulties in each other’s suggestions. Blockages are bad. Negative feedback is a sin. As Alex Faickney Osborn, an advertising executive who wrote a series of bestselling books on brainstorming in the 1940s and 1950s, put it: ‘Creativity is so delicate a flower that praise tends to make it bloom, while discouragement often nips it in the bud.’

The second group were given no guidelines at all: they were allowed to come up with ideas in any way they thought best.

But the third group were actively encouraged to point out the flaws in each other’s ideas. Their instructions read: ‘Most research and advice suggests that the best way to come up with good solutions is to come up with many solutions. Free-wheeling is welcome; don’t be afraid to say anything that comes to mind. However, in addition, most studies suggest that you should debate and even criticise each other’s ideas.’

The results were remarkable. The groups with the dissent and criticise guidelines generated 25 per cent more ideas than those who were brainstorming (or who had no instructions). Just as striking, when individuals were later asked to come up with more solutions for the traffic problem, those with the dissent guidelines generated twice as many new ideas as the brainstormers. Further studies have shown that those who dissent rather than brainstorm produce not just more ideas, but more productive and imaginative ideas. As Nemeth put it: ‘The basic finding is that the encouragement of debate – and even criticism if warranted – appears to stimulate more creative ideas. And cultures that permit and even encourage such expression of differing viewpoints may stimulate the most innovation.’

The reason is not difficult to identify. The problem with brainstorming is not its insistence on free-wheeling or quick association. Rather, it is that when these ideas are not checked by the feedback of criticism, they have nothing to respond to. Criticism surfaces problems. It brings difficulties to light. This forces us to think afresh. When our assumptions are violated we are nudged into a new relationship with reality. Removing failure from innovation is like removing oxygen from a fire ...

Imagination is not fragile. It feeds off flaws, difficulties and problems. Insulating ourselves from failures – whether via brainstorming guidelines, the familiar cultural taboo on criticism or the influence of cognitive dissonance – is to rob one of our most valuable mental faculties of fuel …

Failure and epiphany are inextricably linked.