The West Yorkshire snack vortex, his personal shopper, NHS computing, and FRES

I’m currently reading Oliver Campion-Awwad, Alexander Hayton, Leila
Smith, and Mark Vuaran of Cambridge Computer Lab’s case study
on the NHS National Programme for IT, an old topic of this fine blog’s. There is so much in common here with Think Defence‘s epic blog series on the disastrous FRES project it’s not even funny.

In this post we’ll look into how, why, and what this has got to do with Eric Pickles.

The Labsters conceptualise the NPfIT disaster as being another case for the growing academic study of IT project failure. Obviously, they would, they’re the Cambridge Computer Lab’s MPhil in Public Policy – wait, that’s an actual degree? wow! They identify various common points from past failures and note that NPfIT had them in spades, despite the fact that a lot of what was known about project failure in 2002 was known from studying NHS IT projects. These were as follows:

Haste – an unrealistic timetable, no time to engage with users, inadequate preliminary work, failure to check progress, failure to test systems

Design – failure to recognise the risks of large IT projects, failure to recognise that the longer the project takes, the more likely it is to be overtaken, sheer ambition, project is too large to manage, confidentiality issues

Culture and skills – a lack of clear leadership, not knowing or constantly changing the aim of the project, not committing necessary funding from the outset, a lack of concern for privacy, no exit plan or alternatives, lack of project management skills, emphasis on price, suppliers depend on lowballing and charge heavily for variations to poorly written specifications

They provide a detailed history of NPfIT, starting off with the original Information for Health strategy in 1997-1998, which actually sounds quite awesome. This strategy foresaw various aims, which boil down to three functional areas – a system for keeping records and running workflows, a management information system for both managers to monitor how well it worked and clinicians to monitor how effective medical practice was, and a Web site to disseminate information about health and to help the public influence NHS policy. The nice thing about this is that the three areas have minimal interfaces, and even those are read-only (the MIS sucks up logs from the first; the Web site uses stats from the MIS).

The even nicer thing about it was that it didn’t require centralisation, indeed, its five principles explicitly required the maintenance of patient confidentiality and local ownership of the system. These were maintained as late as 2000. The even nicer thing still was that it had a sort of Stafford Beer recursive quality; medics would use evidence from it to practice evidence-based medicine, managers would use evidence from it to improve the organisation, and we, the political nation, would use evidence from it to supervise the managers.

How this went wrong, well, it’s basically got Tony Blair’s name on it. Replacing Information for Health with NPfIT was Blair’s idea. It came from Tony Zoffis, as they used to say. Microsoft (and Cisco) pitched him silly and he then imposed it on everyone else. The key meetings were in February and March 2002, exactly the apex of Blair’s self-confidence. The Wanless review, better-known for pointing out that the NHS needed money more than it needed “reforms”, played a role; Wanless thought it needed better IT, and that the central government should hold it to doing it, but he didn’t actually recommend anything like NPfIT. However, the report did serve to sell NPfIT to the Treasury, the only force in British politics that could veto a project backed by the prime minister.

I’ve got more to say about this, but let’s pause for a quote.

Management consultant Thomas Brooks, who was involved in NPfIT under contract for a number of trusts, commented that in the procurement process “the iSoft Lorenzo offering was selected from paper descriptions with minimal demonstrations of prototype software elements”

Compare:

One of the Boxer prototypes in APC configuration would participate in the Trials of Truth, joined by Véhicule Blindé de Combat Infanterie (VBCI) from Nexter (previously Giat Industries) and the General Dynamics Piranha V. It was rumoured that the head of DE&S, Lord Drayson, wanted the VBCI because it would be quickest into service.

Both Boxer and VBCI were rejected by the Army, despite what Lord Drayson wanted. We had a choice of two vehicles that would need minimal development and were already (or about to be very soon) in production, and a PowerPoint design, the PowerPoint of course won the trials. The actual vehicle trialled was not Piranha V but Piranha Evolution, a surrogate for the final design.

It’s uncanny, isn’t it? FRES was a huge unwieldy mess with too many stakeholders, aims that constantly changed, and a deliberate determination to avoid meaningful test or development on the government’s time. There was even an insult for civil servants or officers who dared suggest that the end product might be an armoured vehicle of some sort, rather than, say, a new shade of the colour blue. They were said to be “solutionising” and usually moved on. Concrete thinking was reserved to private contractors only.

NPfIT had a parallel phenomenon, the “outcomes-based specification”. Presumably this originated in some sort of vague awareness of object-oriented programming, but the silliness can be summed up by the fact they set out to draft an outcomes-based specification for a standard data interchange format, which sounds like either a sheet of A4 with “IT SHOULD WORK” on it, or else an elaborate exercise in doing the actual work without admitting to it. It didn’t go anywhere; when things eventually deployed, huge amounts of time were taken up mapping nonstandard field names.

As a result, both projects tended to drift in a cycle between ambition growing without limit, untethered from the ground truth, and stodginess, lacking inspiration because out of touch with the possibilities of the technology.

It seems hard to fault FRES for haste of all things, but I think I would. Reading through TD’s series, one thing which stands out is the combination of haste – there was never time to do it properly – with timelessness – deadlines were never allowed to bite. This is precisely what happened with NPfIT, and come to think of it, every project I’ve been involved with that failed. It’s a special kind of time when frenzy and stasis combine. NPfIT’s schedule was always wildly unrealistic – Department of Health R&D Director Sir John Pattison promised Blair delivery in 2 years, 9 months in a meeting where Pattison recalled nobody seemed to be able to say “no” to Blair – but every time the deadline came up, it was just rolled over.

In both cases, the government tried to outsource its own outsourcing. The only element of NPfIT that worked properly and that was part of The Vision was the network. BT built that and it did the job itself. Everything else was contracted out to a contractor, who then subcontracted, creating a minimum of three layers of abstraction between the customer and the supplier.

Under FRES, the government hired WS Atkins as a “systems house”, whatever one of those is, shoved in between the MOD and the various contractors. The idea was apparently that they would manage the process of managing the development of the system (no solutionising!) and also its procurement.

This is the logical end point of our friend Pickles’ worldview. Pickles claimed back in the 1980s that there was a US town council that met once a year, just to issue contracts to run all its services for the next 12 months. This turns out to have been a fairy tale, but that’s by the by. He tried to implement this in Bradford and failed, but its spirit infused the procurement reforms of the 1990s, all of which were designed on the basis that letting the government get involved in the stuff it bought would be stupid. Instead, the point would be to pick as between brands of biscuits at the supermarket. The really weird thing here, though, is that Pickles procurement differs quite dramatically from the sort of thing neoliberals like to say about stuff you buy in the supermarket. Rather than being an active and informed consumer, the government is expected to use a personal shopper. How well this works…well he did spend £10,000 on snacks.

With regard to NPfIT, this intersected with other political imperatives. The famous LSPs, the super-contracts that didn’t actually match any NHS structure and only existed to make a better size of contract for Accenture or whoever, were also intended to channel a more general management influence from Blair’s office into the NHS. The Gate Zero review of NPfIT said:

There is widespread appreciation that the programme is a change programme first and foremost albeit with significant IT elements

Sir John Pattison said in mid-2002:

there was a need to create a new tier in the procurement process “to ensure not only that technology solutions are available and accredited, but to underpin those implementations with comprehensive change management

But what was the change that needed to be managed? It is surely very telling that the Health Secretary involved was Alan Milburn, memorably described at the time as leading the “Special Republican Guard” of ultra-Blairites, inventor of foundation hospitals and independent sector treatment centres. This was the peak of Blairite confidence; on the privacy front, Milburn had just legislated himself the right to dispose of NHS information as he pleased. (Is it significant that as we now know, GCHQ was growing at a rate of knots?)

Arguably, between him and Tony Zoffis, what was wanted wasn’t a management information system but rather a management imposition system. Rather than a system that would aid in the practice of evidence-based medicine and in public scrutiny, they wanted one that would help generate policy-based evidence to defend the changes it imposed after the fact, and to protect it from public scrutiny. Part of the take-home message here is that even had NPfIT worked, there is an argument that it shouldn’t have.

Meanwhile, FRES was certainly intended to support an army with global capability. However, its eventual consequences are a reconnaissance vehicle that weighs 32 tonnes and can’t cross most of the bridges in the Home Counties. (Hey, it’s no funnier than software you decided to stop buying but that kept coming anyway.) You could make a similar case, but then, if you decided to build a super-heavy armoured behemoth of an army that could crush anything as long as it was within a few miles of the border, you’d get today’s Israeli Defence Forces.

It is possible that there is so much software in anything important today that all big projects exhibit some of the characteristics of big software projects. Technology changes aims, though, but not as much as aims change technology. I fear that the problem is different. The upshot of both these stories is that the aims of the political settlement under which we live may make these procurements impossible.

14 comments

  1. Metatone

    Thanks for this. Very interesting.
    My own experiences from working in the NHS around the fringes of the NPfIT really point to the contracting as a huge issue. Reminds me of your piece about the aeroplane Milton Friedman built. Relatively simple and obvious things (like data exchange standards) fell into a grey zone between contractors…

  2. gastro george

    Heh, Atkins home page has the temerity to claim “We have the breadth and depth of expertise to respond to the most technically challenging and time critical infrastructure projects”.

  3. Metatone

    One of the oddities of the modern contracting landscape is that contracts go to “firms” like Atkins, or Accenture, but the staff inside the firm is usually churning at a high enough rate (given that these are multi-year projects) that any past success isn’t much help because the tacit knowledge has already been shifted out the door in favour of someone younger and cheaper…

    • John Styles

      When I worked for this sort of consultancy (not as a consultant), the rule seemed to be ‘every potential client gets the best CV then whoever makes the most trouble gets the actual consultant’. I have used this knowledge to my advantage when we had an outside training course, got a good CV then surprise surprise at the last moment someone was substituted. I knew I just had to make myself more objectionable than the next guy…

  4. Phil

    widespread appreciation that the programme is a change programme first and foremost albeit with significant IT elements

    Dear God. I think you’ve just answered a question which – though it may sound like an old joke – has genuinely puzzled me for several years: what do these people actually do? The IT dept where my OH works had an AA infestation once; there seemed to be far too many people, working far too long and hard, for any identifiable results. But perhaps that’s the point. If you want a system developed, after all, you’re primarily going to want people who can develop systems, and what you’ll want them to do is develop a system for you – and, with the worst will in the world, there are only so many layers (of abstraction or of management) that you can put into that process. If you want change, though, the sky’s the limit – you’ll need new procedures, you’ll need a bureaucracy to manage them, then you’ll need procedures for the bureaucracy to follow… As for getting it finished, you’re covered both ways – finish the IT component and there’s still more work to do, leave the IT in a shambles and, well, never mind that – just look at all the change you’ve brought about!

  5. Chris Williams

    Ah, this “huge amounts of time were taken up mapping nonstandard field names”. The Home Office’s ADP unit — who were an offshoot of Treasury O&M and reporting back to it — discovered this in _1959_. Pretty much all of that sorry saga of NHS IT provision involves failing to notice lessons which the Police National Computer team and its precursors learned in the 1960s. More https://www.academia.edu/1723022/The_Arrival_of_the_Computer_in_the_UK_Criminal_Justice_System_1955-1975

  6. gastro george

    Speaking as a software engineer in a company that successfully developed and installed a national electronic prescription service in a European country in 1994(!), that case study is one hell of a car crash.

  7. Metatone

    @Chris Williams, @gastro george

    One of the most curious aspects of UK government IT failures is how it’s always turned into propaganda for the latest fashion in programming (Agile would appear to be the beneficiary in this case). Now I’m not knocking Agile, but I really think it lets the players involved off the hook to claim that “Agile would have fixed it” since really much more basic issues of competence seem to be the root problem…

    • gastro george

      Methodolodgies have their place, but methodologies are rarely a solution in themselves. IMHO. As Alex’s post and the case study show – the reasons for the success or failure of IT projects are pretty well known and are agnostic of methodology. It’s just that there are enough idiots-with-power who don’t know them, or are prepared to ignore them. Roll on the next disaster AKA Universal Credit.

  8. Chris Williams

    Yeah, one of the features of the PNC project was that the client (HO) employed a lot of expertise – a noticeable fraction of all the systems analysts in the UK. So they were in a position to cut through all the bullshit which the potential contractors (esp ICL and IBM) tried to foist on them. And when they discovered that some elements of their wishlist were unworkable, they dropped them.

    • Guano

      I read Transport for London’s recent “lessons learnt” report about the contract for signalling on the District/Circle/Metropolitan lines that was terminated after about a year. What struck me was the amount of expertise employed within the client organisation (London Underground/TfL), how active this technical management by TfL was and that, if there was a lesson, it should have been more active or pro-active.

      This is a very different picture from the public image of a contract, which implies signing an agreement and sitting back to wait for the results. It is also very different from the ideology of public-private partnerships, which implies that the public sector does not have expertise and the private sector will solve the problems. In reality there is no alternative to having public sector expertise tightly managing a project even if other expertise is bought in.

  9. LImericky Dicky (off duty)

    Well as I keep going to Labour Party events and hypnotically repeating to whoever seems most likely to have some networky influence, it really is about time that (ahem) the pendulum moved back some way towards a mixed economy (add disingenuous questions about how rail nationalisation polls).

    So given that these guerilla tactics can hardly fail and the 3 1/4 nations of the UK will rally behind a man with some real convictions and ideas who would only be seen dead on an oligarch’s yacht, it’s time to start some Planning.

    And the obvious suggestion here is a National IT Service. Srsly.

Post a comment

You may use the following HTML:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>