Evidence of e-health benefits needed, says IPPR report

The report, entitled ‘Public Value and e-Health’, says that effective evaluation of current investments will be crucial for benefits to be proven and professional and public support secured. It also warns that threats to the success National Programme for IT (NPfIT) include a failure to consult properly with medical professionals, the lack of IT skills within the healthcare service and poor understanding of what benefits the programme is meant to deliver.

While the potential benefits of better use of ICT in the NHS are clear, there is as yet very little evidence to show that it is making a difference to the delivery of care or responsiveness of services.

“We have looked for evidence that ICT is helping to deliver sources of public value such as better value for money, greater satisfaction with services, improved health outcomes or greater trust in the NHS," says the report.

“Despite the clear potential benefits of improving use of ICT, really solid evidence of a positive impact is still quite scarce,” it concludes. “We were surprised by the disparity between the obvious potential benefits of using ICT more effectively in health and the paucity of evidence that these benefits were being delivered in practice."

Jamie Bend, IPPR research fellow and author of the report, told E-Health Insider that the absence of strong evidence-based research on the benefits of e-health made it more difficult to implement the National Programme for IT and convince clinicians of its benefits.

“The roll-out of NPfIT would be much easier if e-health had a solid research base.  The fact that evidence doesn’t exist in relation to IT systems makes it more difficult to convince clinicians about NPfIT."

He added that the investment decisions behind NPfIT had been based not on hard evidence but on the 2002 Wanless review, which identified increased ICT investment in the NHS as critical to NHS modernisation, and the fact the potential benefits being argued for are something policy makers “find very easy to conceptualise".

Bend said: “Eventually that sort of good faith will run out and we will need evidence to back up the claims for e-health." Without such evidence he warned there was a real risk that politicians, clinicians and the public will become disillusioned and sceptical about the merits of e-health.   

“A lot of what is going on around the national programme is relatively obscure, I’m hoping that there will be evaluation carried out locally and by the national programme nationally."  He said initial indications from the national programme that it would carry out such research as part of its work on benefits realisation were encouraging.

On cost savings Bend noted: “We found a lot of anticipated and projected cost savings, but little evidence of more detailed evaluations of whether savings are delivered in reality."

The IPPR examined over 40 evaluation reports related to projects for the six key NHS IT services, including electronic patient records, electronic booking and electronic transfer of prescriptions, looking to identify where projects had delivered ‘public value’.

The report states that evaluations of past projects have suffered from common problems: too little time and too few resources had been allocated to evaluation; the rationale for evaluation was often unclear; sufficient data for evaluation was often not available; and the evidence presented was not always robust enough.

One of the key issues highlighted is the need to manage the change in business processes and working practices that will accompany the new IT systems to be delivered by the IPPR.  The report expresses concern about the whether the NHS has the capacity to deal with ICT-enabled change on the scale required to realise the full benefits of the national programme.

“There may simply be insufficient capacity within the NHS to cope with the magnitude of change that will be required. Inadequate funding, insufficient skilled staff and the competition of other priorities may mean that although ICT systems have been procured, the benefits delivered will not be as great as they might have been."

Another warning is sounded on the potential consequences of the degree of centralisation represented by the national programme. “This could potentially make systems insufficiently flexible to take account of useful variations in local working practices and might also lead to trailblazing NHS organisations being held back," says the report.

Bend added that further work was needed from the national programme to engage with clinicians and the public to build awareness and confidence. “The public just wants to be dealt with by their doctor and don’t have a good understanding of how the NHS works of their information is used."

“Without building the public’s awareness about what is being done there is a risk that it only takes a couple of stories on misuse of records to really trigger a collapse of public confidence in the system, so we need much more public engagement."