Joe’s view: of ratings services

I have always been struck by the need for feedback on quality of care in a regular and meaningful way; instead of just relying on complaints or the results of enquiries into homicides and suicides.

I wrote my first paper on using computers to measure and improve psychiatric services 17 years ago, and patient feedback was always going to be part of that process.

Later I became medical director of a large mental health trust, which felt, at times, like “flying blind.” Self-reported quality measures used by trusts have proven unreliable – just think of those at Mid Staffs.

I dream of a health-based ‘End User Knowledge Network’ to rival Trip Advisor to inform patients and medical directors alike where pockets of excellence and failure lurk.

Last summer, I had a television aerial fitted and 20 minutes after the job was finished I got an email asking me to rate the service. Surely healthcare should be able to do something similar. I decided to experiment with available internet-based feedback systems in my own adolescent psychiatry clinic.

NHS Choices

In July, I recommended to patients that they should visit the NHS Choices website to feedback on the quality of care I provide. This appeared to produce no responses.

When I attempted to feedback myself, it wasn’t immediately obvious where to go on the Choices website and when I found the correct page I could not feedback on the specific service I provide. Indeed, the hospital I work at was not listed; it was only possible to feedback on the trust as a whole.

I imagine most patients just give up at this point, and it seems to be the angry who persist long enough to post something, judging from the content of the postings.

When I discussed the issue with the NHS Choices team, it became clear that it is the responsibility of the trust to make sure information about services is accurate and sufficiently granular to make feedback meaningful. I intended to have the information about my service updated – but, as a busy clinician, I never got round to it.

Patient Opinion

In August, I recommended that patients visit the Patient Opinion website to feedback on the quality of care I provide. This appeared to produce no responses.

All trust letters carry the Patient Opinion logo, but have generated only 15 responses in two years. Although it is obvious where to provide feedback on the Patient Opinion website, it again lacked the granularity to feedback on my service – just finding my trust took some time. Also the website didn’t work on my iPhone.

Patients don’t seem to think in units of “whole trusts” but are more likely to think in terms of individuals or departments or hospitals. The aim of the website, to be fair, isn’t simply to rate care but to start a conversation. It seems to fulfil that function well, but it produces little volume of feedback.

I Want Great Care

In September, I recommended patients visit the I Want Great Care website, after I registered with the site and downloaded the business cards it provides to be handed out to patients. I handed out 20 cards and got five responses.

Interestingly, it was easy to tell who was feeding back from the content of the posts and the fact that numbers were very low. All the feedback was very positive, in contrast to the organisation-level feedback, and was always posted within 24 hours of a clinic visit. I Want Great Care offer an iPhone application, but it does not work. If it had worked, I would have given it five stars.

Wellnote

This is an iPhone application that was produced last year. It allows patients to rate general practices, dental practices and hospitals, but not individual practitioners.

The vast majority of practices and hospitals have no ratings a year after the app was launched. I could not find my hospital on the app, so I was unable to use the app for patients to rate my service. Very nearly excellent; but not quite good enough.

Discussion

This was far from a scientific study but a few tentative conclusions can be drawn. Firstly, feedback mechanisms need to be easy to use and currently paper beats electronic hands down for generating volume of feedback. I still hand out paper feedback questionnaires, and these have brought in 2,000 responses in a year.

Secondly, feedback not collected in the immediate aftermath of contact with the service will not be collected. Thirdly, patients appear to want to feed back on individuals more than they want to feedback on organisations. Fourthly, a business card with details of how to feedback seems a useful way of increasing feedback.

The psychology of using I Want Great Care was also of interest. Because the feedback was about me, not about my organisation, I cared very much what people said.

While I think my personally handing a card to patients and asking for feedback helped to encourage feedback, I did notice that I was less inclined to hand a card to a patient who was unhappy at my unwillingness to, say, prescribe Valium.

Once people began feeding back, I noticed that I was very much on my best behaviour and availed myself of the feature that sent me a text message to alert me to new feedback.

As my own feedback was very positive, I decided to check on the feedback received about doctors I knew, to see if everyone got positive feedback (an exam in which everyone scores 100% is worthless). In fact not all doctors get good feedback and some doctors whom I had perceived negatively had significant negative feedback – indicating some level of validity.

Interestingly, TripAdvisor has rapidly become essential to travellers, but sites providing patients the opportunity to feedback on their healthcare still receive very small numbers of visitors.

Surely people care more about their healthcare than their holidays? Discussing this with some of my patients, the reasons for this put forward were:

• You don’t get to choose your psychiatrist like you would choose a hotel, so there’s no point feeding back
• You might upset the doctor and make matters worse
• It’s not like booking a holiday; it’s not your own money.

It may be that until real choice is offered to patients, an End User Knowledge Network like TripAdvisor will not gain the traction that would be required to drive up quality and to give medical directors the granular view of their organisations that they require to prevent another Mid-Staffs.

Conclusions

Electronic patient feedback could help improve quality of healthcare, but the means of feedback needs to be: easy to use; timed to immediately follow interaction with the service provided; and granular – preferably down to named clinician level.

With this in mind, my recommendations would be:

• Further development of patient feedback mechanisms (for example text messaging and/or smartphone apps) should take on the points above and, in particular, work to ease patients’ ability to feedback in a timely manner.
• Patient Opinion and NHS Choices need to offer a greater level of granularity and support to individual trusts. As currently configured, the amount of feedback generated does not appear to offer value for money.
• Until the NHS offers genuine choice, a way needs to be found to motivate patients to provide feedback (Caremiles???) on their care.
• An incentive to produce patient feedback should be included in the tariff system or patients could be incentivised to provide feedback (free prescriptions???)

About the author: Joe McDonald is a practising NHS consultant psychiatrist. Over the past five years he has been an NHS trust medical director and national clinical lead for IT at NHS Connecting for Health – a stint that included 18 months as medical director of the Lorenzo delivery team!

His experiences in the National Programme for IT in the NHS have left him with a passion for usability and "end user knowledge networks.” He is the founding chairman of the National Mental Health Informatics Network. Motto: we don’t get fooled again. Follow him on twitter @CompareSoftware