NHS IT systems in NWWM slow to recover

Dozens of NHS trusts across the North West and West Midlands remained without access to their main patient management software today, after a Sunday morning failure at a data centre run by Computer Sciences Corporation (CSC) hosting NHS systems. Some trusts face being without systems until Thursday.

The IT disaster, believed to be the most widespread ever to occur in the NHS, has left 88 trusts – eight acute trusts and 72 primary care trusts – without access to centrally hosted patient administration systems (PAS). Although PAS systems do not usually contain clinical data they directly support clinical care and are essential to safely identifying patients.

One NHS IT director, speaking anonymously, told EHI: “It’s a very serious problem.” The director added that trusts had been told not to answer press enquiries on the problem.

Ewan Davis, chairman of the British Computer Society Primary Health Care Specialist Group, told EHI that the systems affected directly supported patient care: “These are systems responsible for the administration of day-to-day clinical care. Although that clinical activity may not fail they are certainly stressed [by] having to fall back to manual systems.”

Sources close to the National Programme for IT told EHI that they hoped to have 40% of the 88 affected trusts back up and running by Wednesday, while a spokesperson for NHS Connecting for Health told EHI “We are hoping to have all trusts restored by Thursday.”

The eight acute trusts affected, have included Birmingham Children’s Hospital and University Hospital Birmingham, representing two thirds of the acute systems implemented under the NHS Connecting for Health programme.

Some trusts are beginning to be restored. By lunchtime today these included Morecambe Bay in Lancashire, and Birmingham Children’s hospital, and a number of PCTs. Reports reaching EHI, however, indicated that some restored trusts, including two PCTs, were experiencing extremely slow access, to the point that staff could not log-on or use systems.

NHS Connecting for Health the agency responsible for the £12.4bn NHS IT upgrade, said the failure stemmed from interruption to the power supply at the CSC “data centre”. The ‘hot’ standby disaster recovery data centre that was then meant to automatically take over, ensuring an uninterrupted service, failed to work.

The NHS IT director told EHI: “It’s just frightening, the fact that the disaster recovery site wasn’t ready and the recovery didn’t happen.” 

The director added: "If this happened further down the line and took down clinical systems embedded into patient care it would have been a disaster for the NHS." 

Under the terms of its £973m IT contract with the NHS CSC is meant to provide a ‘non-stop’ uninteruptable service available 24/7. The failure over the weekend will raise questions over CSC’s contractual performance; whether NHS CfH has done adequate due diligence of its contractor’s disaster recovery arrangements.

Davis told EHI: “I am disappointed that we have had this demonstration of what was meant to be a highly specified data centre.” He added that it was likely to put back progress on persuading GPs they are best served on moving to a data centre environment.