When technologies can’t evolve

In what way, and on what basis, should we attempt to steer the development of technology? This is the fundamental question that underlies at least two discussions that I keep coming back to here – how to do industrial policy and how to democratise science. But some would simply deny the premise of these discussions, and argue that technology can’t be steered, and that the market is the only effective way of incorporating public preferences into decisions about technology development. This is a hugely influential point of view which goes with the grain of the currently hegemonic neo-liberal, free market dominated world-view. It originates in the arguments of Friedrich Hayek against the 1940′s vogue for scientific planning, it incorporates Michael Polanyi’s vision of an “independent republic of science”, and it fits the view of technology as an autonomous agent which unfolds with a logic akin to that of Darwinian evolution – what one might called the “Wired” view of the world, eloquently expressed in Kevin Kelly’s recent book “What Technology Wants”. It’s a coherent, even seductive, package of beliefs; although I think it’s fatally flawed, it deserves serious examination.

Hayek’s argument against planning (his 1945 article The Use of Knowledge in Society makes this very clearly) rests on two insights. Firstly, he insists that the relevant knowledge that would underpin the rational planning of an economy or a society isn’t limited to scientific knowledge, and must include the tacit, unorganised knowledge of people who aren’t experts in the conventional sense of the word. This kind of knowledge, then, can’t rest solely with experts, but must be dispersed throughout society. Secondly, he claims that the most effective – perhaps the only – way in which this distributed knowledge can be aggregated and used is through the mechanism of the market. If we apply this kind of thinking to the development of technology, we’re led to the idea that technological development would happen in the most effective way if we simply allow many creative entrepreneurs to try different ways of combining different technologies and to develop new ones on the basis of existing scientific knowledge and what developments of that knowledge they are able to make. When the resulting innovations are presented to the market, the ones that survive will, by definition, the ones that best meet human needs. Stated this way, the connection with Darwinian evolution is obvious.

One objection to this viewpoint is essentially moral in character. The market certainly aggregates the preferences and knowledge of many people, but it necessarily gives more weight to the views of people with more money, and the distribution of money doesn’t necessarily coincide with the distribution of wisdom or virtue. Some free market enthusiasts simply assert the contrary, following Ayn Rand. There are, though, some much less risible moral arguments in favour of free markets which emphasise the positive virtues of pluralism, and even those opponents of libertarianism who point to the naivety of believing that this pluralism can be maintained in the face of highly concentrated economic and political power need to answer important questions about how pluralism can be maintained in any alternative system.

What should be less contentious than these moral arguments is an examination of the recent history of technological innovation. This shows that the technologies that made the modem world – in all their positive and negative aspects – are largely the result of the exercise of state power, rather than of the free enterprise of technological entrepreneurs. New technologies were largely driven by large scale interventions by the Warfare States that dominated the twentieth century. The military-industrial complexes of these states began long before Eisenhower popularised this name, and existed not just in the USA, but in Wilhelmine and Nazi Germany, in the USSR, and in the UK (David Edgerton’s “Warfare State: Britain 1920- 1970″ gives a compelling reinterpretation of modern British history in these terms). At the beginning of the century, for example, the Haber-Bosch process for fixing nitrogen was rapidly industrialised by the German chemical company BASF. It’s difficult to think of a more world-changing innovation – more than half the world’s population wouldn’t now be here if it hadn’t been for the huge growth in agricultural productivity that artificial fertilisers made possible. However, the importance of this process for producing the raw materials for explosives ensured that the German state took much more than a spectator’s role. Vaclav Smil, in his book Enriching the Earth, quotes an estimate for the development cost of the Haber-Bosch process of US$100 million at 1919 prices (roughly US$1 billion in current money, equating to about $19 billion in terms of its share of the economy at the time), of which about half came from the government. Many more recent examples of state involvement in innovation are cited in Mariana Mazzucato’s pamphlet The Entrepreneurial State. Perhaps one of the most important stories is the role of state spending in creating the modern IT industry; computing, the semiconductor industry and the internet are all largely the outcome of US military spending.

Of course, the historical fact that the transformative, general purpose technologies that were so important in driving economic growth in the twentieth century emerged as a result of state sponsorship doesn’t by itself invalidate the Hayekian thesis that innovation is best left to the free market. To understand the limitations of this picture, we need to return to Hayek’s basic arguments. Under what circumstances does the free market fail to aggregate information in an optimal way? People are not always rational economic actors – they know what they want and need now, but they aren’t always good at anticipating what they might want if things they can’t imagine become available, or what they might need if conditions change rapidly. There’s a natural cognitive bias to give more weight to the present, and less to an unknowable future. Just like natural selection, the optimisation process that the market carries out is necessarily local, not global.

So when does the Hayekian argument for leaving innovation to the market not apply? The free market works well for evolutionary innovation – local optimisation is good at solving present problems with the tools at hand now. But it fails to be able to mobilise resources on a large scale for big problems whose solution will take more than a few years. So, we’d expect market-driven innovation to fail to deliver whenever timescales for development are too long, or the expense of development too great. Because capital markets are now short-term to the point of irrationality (as demonstrated by this study (PDF) from the Bank of England by Andrew Haldane), the private sector rejects long term investments in infrastructure and R&D, even if the net present value of those investments would be significantly positive. In the energy sector, for example, we saw widespread liberalisation of markets across the world in the 1990s. One predictable consequence of this has been a collapse of private sector R&D in the energy sector (illustrated for the case of the USA by Dan Kammen here – The Incredible Shrinking Energy R&D Budget (PDF)).

The contrast is clear if we compare two different cases of innovation – the development of new apps for the iPhone, and the development of innovative new passenger aircraft, like the composite-based Boeing Dreamliner and Airbus A350. The world of app development is one in which tens or hundreds of thousands of people can and do try out all sorts of ideas, a few of which have turned out to fulfil an important and widely appreciated need and have made their developers rich. This is a world that’s well described by the Hayekian picture of experimentation and evolution – the low barriers to entry and the ease of widespread distribution of the products rewards experimentation. Making a new airliner, in contrast, involves years of development and outlays of tens of billions of dollars in development cost before any products are sold. Unsurprisingly, the only players are two huge companies – essentially a world duopoly – each of whom is in receipt of substantial state aid of one form or another. The lesson is that technological innovation doesn’t just come in one form. Some innovation – with low barriers to entry, often building on existing technological platforms – can be done by individuals or small companies, and can be understood well in terms of the Hayekian picture. But innovation on a larger scale, the more radical innovation that leads to new general purpose technologies, needs either a large company with a protected income stream or outright state action. In the past the companies able to carry out innovation on this scale would typically have been a state sponsored “national champion”, supported perhaps by guaranteed defense contracts, or the beneficiary of a monopoly or cartel, such as the postwar Bell Labs.

If the prevalence of this Hayekian thinking about technological innovation really does mean that we’re less able now to introduce major, world-changing innovations than we were 50 years ago, this would matter a great deal. One way of thinking about this is in evolutionary terms – if technological innovation is only able to proceed incrementally, there’s a risk that we’re less able to adapt to sudden shocks, we’re less able to anticipate the future and we’re at risk of being locked into technological trajectories that we can’t alter later in response to unexpected changes in our environment or unanticipated consequences. I’ve written earlier about the suggestion that, far from seeing universal accelerating change, we’re currently seeing innovation stagnation. The risk is that we’re seeing less in the way of really radical innovation now, at a time when pressing issues like climate change, peak cheap oil and demographic transitions make innovation more necessary than ever. We are seeing a great deal of very rapid innovation in the world of information, but this rapid pace of change in one particular realm has obscured much less rapid growth in the material realm and the biological realm. It’s in these realms that slow timescales and the large scale of the effort needed mean that the market seems unable to deliver the innovation we need.

It’s not going to be possible, nor would it be desirable, for us to return to the political economies of the mid-twentieth century warfare states that delivered the new technologies that underlie our current economies. Whatever other benefits the turn to free markets may have delivered, it seems to have been less effective at providing radical innovation, and with the need for those radical innovations becoming more urgent, some rethinking is now urgently required.

8 Responses to “When technologies can’t evolve”

  1. Brilliant post Richard, thanks.

    Do you have any views on how these radical innovations should be catalysed?

    Perhaps that’s one for your next post!?

  2. Richard Jones says:

    Thanks, Hilary. My views on all this stuff are still forming, and I’m sure I’ll be writing more about it. But it’s worth highlighting something in your comment – you talk about what might “catalyse” radical innovations. Maybe I’m overinterpreting, but the metaphors we use tell us something about our assumptions. The idea that all it takes to get these radical innovations is a catalyst – a pinch of magic fairy dust, to bring the right people together, to fix a minor market failure – is perhaps one of those notions I’m questioning here. Maybe it doesn’t need just a catalyst, but instead it needs a lot of sustained effort and serious resources. Then the question is a different one, about how you create the will to make that happen and the way of mobilising it.

  3. Ah yes. I thought about what word I was using and chose that to denote a certain amount of energy, purpose & agency which I felt was required to stimulate and deliver such a radical rethink. As a non-scientist I was using catalyst in a very flabby way, I should have known better!

    But agree there is no quick fix, or obvious easy option – such systems change will require sustained effort, resources, connections and focus, and probably a bit of luck and fairy dust to boot.

    However, I am not sure that there is a clear enough view from those who have the capacity to deliver the rethink that innovation is ‘broken’ enough to require it and to deal with the potentially unpopular trials and errors that may entail.

    I am also interested in the view which questions whether we really need much more growth and innovation, radical or otherwise, and that a rethink of the way we view our place in the world, our attitudes to consumption etc is still more overdue.

  4. Richard Jones says:

    Hilary, you make two very important points. Firstly, I agree that there are many people who are very heavily invested in the idea that things are basically ok as they are. We’ll see how long that’s sustainable for, as our economic slump extends beyond the experience of living memory and a period of sustained erosion of living standards continues. You second point, about whether we actually need innovation, is central too. Again, I’d argue that there is innovation and innovation, and rethinking what we really need is overdue. But I would insist that there is some innovation we very much do need – we depend for our existence on technologies that we know aren’t sustainable, so we have to find better ones.

  5. Yes, finding better solutions to unsustainable technologies one essential job, but incentivising/persuading/forcing those with the vested interest in the status quo to see that they are indeed superior, a whole other ballgame.

  6. Jim Fehlinger says:

    > The market certainly aggregates the preferences and knowledge
    > of many people, but it necessarily gives more weight to the
    > views of people with more money, and the distribution of money
    > doesn’t necessarily coincide with the distribution of wisdom or
    > virtue. Some free market enthusiasts simply assert the contrary,
    > following Ayn Rand. There are, though, some much less risible moral
    > arguments in favour of free markets which emphasise the positive
    > virtues of pluralism, and even those opponents of libertarianism who
    > point to the naivety of believing that this pluralism can be maintained
    > in the face of highly concentrated economic and political power need
    > to answer important questions about how pluralism can be maintained
    > in any alternative system.

    It can only be maintained sporadically and imperfectly, of course.
    The Ayn Randians have their way most of the time — concentrations of
    money (and hence power) always have the preponderant say — mostly
    in modern Western democracies via the “manufacture of consent” (as Chomsky
    puts it) rather than outright force (at least in the domestic-policy sphere).

    Occasionally those harmed by the manufactured consensus manage to
    break it — via political alliances with people of conscience or
    outright revolt. Or via the institutionally-guaranteed freedom to act
    of independent-minded jurists or (occasionally) independent-minded

    That’s the best the human race has managed to achieve.

  7. Zelah says:

    Hi everyone,

    To be fair to Hayek, His reply to the technological stagnation issue would be to allow Monopolies to run riot!

    This is for me where the Efficient Market Hypothesis (Updated Hayek!) inconsistencies become manifest! In the real world the Government if it likes it or not has a role either as Referee, Investing for the Future, Skyblue Research or all of the above.


  8. The Keystone Garter says:

    I spoke with the head of Bruker once. What a great presentation. Open to the public for free and an open bar that doesn’t cut you off until 7 beers (ales). He used the incremental model (evolutionary model) in his company. Researchers would just play around with different SPM tool blueprints or whatever, until they hit upon something that filled a market.
    I know now the taxed government intervention is more efficient. One reason is people in the USA tend to get rich by being Libertarian parasites; aren’t very innovative. That health biosensor person-to-person proximity sensor I proposed (I’m poor I can’t actually build it let alone find a min wage job) was pure T.Edison/Da Vinci multidisciplinary brilliance. I wouldn’t have been able to dream it up two years ago. For public policy, instead of looking at the most efficient actors and working from there, I’m able now to pick abstract thesis right out of mid-air. This is something that the annual P/E and short-term stock-optioned markets simply can’t do. If R+D markets were decade to decade investments like housing or infrastructure is, maybe. A gov is the only institution that might look that far ahead, or go off on that big of a tangent.