Jill Lepore has produced an excellent, well-written and passionate refutation of Clayton Christensen’s theory of disruptive innovation in the New Yorker; but I can’t help thinking the target of her piece is not just Christensen’s work, but also the New York Times’ Innovation Report, described by the Nieman Journalism Lab as “one of the key documents of this media age.” By questioning not just Christensen’s ideas, but the data and case studies on which he based his conclusions, Lepore is going right to the source of the ‘disruption’ that threatens to replace the quality journalism of the New York Times and the New Yorker with the cheap sugar hit of traffic and infotainment represented by Upworthy and Buzzfeed, and disturb the comfortable, intellectual purity of the writer by noisily demolishing the wall between ‘content’ and ‘the business’.

Lepore concludes that Christensen’s work has more retrospective than predictive value, which I think is entirely fair.

“Disruptive innovation is a theory about why businesses fail. It’s not more than that. It doesn’t explain change. It’s not a law of nature. It’s an artifact of history, an idea, forged in time; it’s the manufacture of a moment of upsetting and edgy uncertainty. Transfixed by change, it’s blind to continuity. It makes a very poor prophet.”

But in building up to this conclusion, I think she is unfair to imply this idea is a form of secular historical determinism:

“The idea of innovation is the idea of progress stripped of the aspirations of the Enlightenment, scrubbed clean of the horrors of the twentieth century, and relieved of its critics. Disruptive innovation goes further, holding out the hope of salvation against the very damnation it describes: disrupt, and you will be saved.”

Of course disruptive innovation has been massively overplayed as an idea, and the challenges Lepore poses to Christensen’s work deserve a thoughtful and honest response from the author. But whilst ‘disruption’ has undoubtedly become something of an unthinking ideological trope for the startup world, that does not mean that some of the theory’s basic ideas are wrong. For me, it is less about the inexorable march of progress in business than it is about understanding a basic feature of evolution: if you over-optimise for the the past and lack decent predators, then you become lazy and myopic and ripe for disruption, often by a smaller creature that can solve the problems of your environment in a totally different way. In business, I don’t think many people would doubt that this helps explain both the rise of the pocket calculator, the LCD screen, online music, etc; and, at the same time, the failure of previously successful firms such as Kodak and Nokia.

I am uncomfortable with the fetishisation of ‘disruption’, partly because disruption is often negative and partly because it is often conflated with destruction rather than improvement. But I do believe it is important to be open to change, and that is something that the intellectual elite of quality journalism has failed to do – they have taken their privileged societal role for granted, which is partly why less socially-valuable challengers have found such easy pickings. And let’s be honest, whether we are talking about the massively loss-making Guardian in the UK or the reliance of newspapers in the United States on rich proprietors who saw newspaper ownership as a status symbol, there is work to be done if quality reporting is to find a sustainable role in the future.

Rather than focus only on sites like Upworthy and Buzzfeed, it is worth looking at what Vice, Matter and a host of smaller niche online publications are doing in an attempt to solve this problem. The future may yet provide new opportunities for quality work. Ironically, quality journalism is a great example of the kind of business that internet guru Tim O’Reilly used to urge us to create – one that ‘creates more value than it captures.’ This is, incidentally, at the heart of the fairness argument made by Axel Springer’s Mathias Döpfner in his fascinating tussle with Google in Europe. Without the mainstream media, of course, a great deal of social media chatter would not exist. But more importantly, nor would modern society or democracy. The problem is that MSM consumes cash and produces social value, but as we are seeing with the generosity of people on Kickstarter and crowdfunding platforms, even this might not be an impossible problem to solve.

In attacking the messenger – somewhat mischievously at times, such as when citing the collateralisation of debt that led to the financial crash as an example of disruptive innovation (it was more like wilful deception) – Lepore misses some of the value Christensen’s thinking beyond the ‘Innovator’s Dilemma’. For too long, businesses (mainstream media included) have responded to changing circumstances with a combination of financial belt-tightening and minor efficiency innovations, but as Christensen argued when I watched him speak in London last year, firms should ideally look beyond the short-termism of financial management models and create the kinds of empowering innovations that can bring about new products or take existing ideas into new markets. Steve Denning recently expanded on this idea in Forbes. Ironically, given Christensen’s theory, this longer-term approach to business is often characteristic of some of the oldest and most succesful companies in the world today.

In our work, which is about bringing together the energy for change and innovation potential of startups with the experience, skill and scale of established firms, we would never say “CHANGE OR DIE!” Instead, we provide rational arguments for faster, more imaginative change to organisations who for quite understandable reasons are motivated to continue doing what has made them successful in the past. Ideology is a distortion and should play no part in this – if we cannot make a reasoned case for change, then we will not succeed.

If you haven’t read Jill Lepore’s piece yet, it is very good and we need more critical thinking like this in both business and technology.