DUNCAN=20
BRIAN DOUBLE | View/Change User Information |
Published 2 December 2009, =
doi:10.1136/bmj.b5242
Cite=20
this as: BMJ 2009;339:b5242
News
Safety Data
The mysterious Dr Foster
Nigel Hawkes, freelance journalist =
nigel.hawkes1{at}btinternet.com
A consultancy=92s dire warnings about safety in = many of=20 the UK=92s hospitals dominated the weekend headlines. But = were=20 their warnings right? Nigel Hawkes reports
Who is this Dr Foster, the source of the weekend=92s headlines = about=20 the NHS? He seems to be saying that the elaborate system for=20 inspecting hospitals set up=97and recently reformed=97by the = government=20 is missing some pretty obvious examples of bad medicine. =
Thousands of patients dying, hospitals failing to respond to=20 safety alerts or unexpected deaths, swabs and drill bits left = inside patients after operations ... isn=92t this the kind = of=20 thing any respectable government inspector might be expected = to spot?=20 What on earth is going on?
Dr Foster is not, of course, a doctor at all, but a witty name=20 chosen by two journalists when they set up a healthcare = analysis=20 company eight years ago to exploit the mass of data churned = out=20 by the NHS ("Dr Foster went to Gloucester, in a shower of=20 rain...").
Tim Kelsey from the Sunday Times and Roger Taylor from the=20 Financial Times saw an unexploited commercial = opportunity in=20 the work of Professor Sir Brian Jarman of Imperial College.=20 Measuring how well hospitals perform is not an easy task, but = Professor Jarman had devised a way of doing it by using = Hospital=20 Episode Statistics (HES), data produced in huge volume by the = NHS=20 for accounting purposes. His key paper on the subject = appeared=20 in the BMJ in 1999.
Kelsey and Taylor=92s timing was good. The Department of = Health=20 under Alan Milburn needed evidence to support and monitor = progress in=20 implementing the NHS Plan and, after some initial hesitation, = embraced Dr Foster warmly. The first Good Hospital Guide = under the Dr=20 Foster imprint appeared in 2001. Hospitals were under notice = to=20 collaborate with Dr Foster, however little they liked doing = it.=20
So keen was Milburn=92s successor Patricia Hewitt on the = product=20 that she bought the company. Under a deal negotiated in 2006, = =A312=20 million of public money was invested in a joint venture = between Dr=20 Foster and the NHS body, the Information Centre for Health = and Social=20 Care, to create Dr Foster Intelligence. The fact that this = deal went=20 through without a competitive tender raised some eyebrows, = and was=20 heavily criticised by the National Audit Office. The = department had=20 paid too much, and failed to give others a chance to bid, the = NAO=20 ruled.
Professor Denise Lievesley, a statistician who had become chief=20 executive of the Information Centre when the deal was already = sealed, was also unhappy about it. She protested to her = bosses=20 in the NHS. She protested again in 2007 when a contract to = provide=20 material to the NHS Choices website was awarded to Dr Foster, = again (in her view) without proper procurement procedures. = Shortly=20 after this she left her job, with a gagging clause preventing = her from telling her side of the story.
So the irony of the weekend=92s stories is that a company = granted at=20 least one sweetheart deal=97and possibly two=97by the = Department of=20 Health has been the bringer of exactly the sort of bad news = the=20 department would rather not have heard. There have been hints = that=20 the department may have fallen out of love with Dr Foster. It = lost=20 the NHS Direct contract in July 2008, and set up a rival = website for=20 its Good Hospital Guide.
That said, is what Dr Foster says about the NHS to be relied = upon?=20 Professor Jarman=92s method of comparing hospitals is called = the=20 Hospital Standardised Mortality Ratio, or HSMR. From the HES = data the=20 number of patients dying after a range of different = procedures in=20 hospitals is available. All things being equal, good = hospitals will=20 kill fewer patients than bad ones.
The difficulty is that all things are not equal. Hospitals vary = in=20 many ways=97the area they serve, the age of their patients, = the=20 efficiency with which they record the data. HSMRs are corrected, = so=20 far as is possible, for these variables so that at the end of = the=20 process one hospital can be compared directly with another. = Professor=20 Jarman is a leading world authority on this subject, so = nobody doubts=20 the probity of the process.
But many do question the results. One striking feature of this=20 year=92s results is a sharp fall of 7% in HSMRs across = England.=20 This is good news, if slightly too good to believe without = careful=20 scrutiny, because it implies that many fewer patients are = dying=20 in England=92s hospitals than in previous years.
But the fact that HSMRs across the board have fallen so fast = means=20 that the target has moved. The results are rebased so that a = hospital=20 that exactly matched the national average would score = 100=97those with=20 more deaths above 100, those with fewer below 100. The = rebasing means=20 that some hospitals who know they have improved appear to = have done=20 worse, because they haven=92t improved quite as much as the=20 average.
Others say the Dr Foster method misrepresents how good they = are.=20 Alternative methods for calculating HSMRs exist, and in some = cases=20 paint a much more flattering picture. One reason for this may = be that=20 hospitals do not invariably "code" their patients fully. If a = patient=20 has four or five co-morbidities (conditions that make = survival less=20 likely) then this should be taken into account and corrected = for. If=20 they are not recorded properly, then a hospital will have a = higher=20 HSMR than it should. There may even=97perish the thought=97be = the=20 suggestion that clever hospitals are exaggerating = co-morbidities to=20 improve their HSMRs. It wouldn=92t be the first time a target = has=20 been "gamed" in the NHS.
Another possible source of error is that Dr Foster counts only=20 deaths that occur in hospital. If patients are discharged and = die the next day, at home or in a hospice, their deaths go=20 unrecorded. Professor Jarman has shown that in-hospital = mortality is=20 a good proxy for all deaths within 30 days, but others are = not so=20 sure.
So the Good Hospital Guide should be seen for what it is: a=20 sincere attempt to measure something very elusive, but very=20 important. It is not the last word, and nor should the = mortality=20 figures be taken in isolation, without the other measures = that=20 are included in the guide.
But is it more reliable than the Care Quality Commission = verdicts?=20 Given the frequency with which good ratings from the CQC and=20 its predecessor the Healthcare Commission have been = overturned=20 with a few months, Dr Foster provides an important = corrective.=20 Both approaches have their virtues, but careful scrutiny and = a=20 degree of scepticism should accompany the reading of either.
Cite this as: BMJ 2009;339:b5242
A version of this article was first = published in the=20 Independent.
![3D"](3D"http://www.bmj.com/icons/spacer.gif"){kind=spacer}
Related external webpages:
This Article
![3D"Freely](3D"http://www.bmj.com/icons/shared/toc/free.gif")
=20
- Respond = to this article=20
- Respond = to this article=20
Services
- Downlo= ad=20 to citation manager=20
Google Scholar
PubMed
Related Content
Bookmark with
What's = this?
What's new
Latest blogs
Find BMJ on:
- Follow = BMJ on=20 twitter
Services
Tools
Online poll
Resources
- Readers=20
- Authors=20
- Reviewers=20
- Media=20
- BMA = members=20
- Advertising and = sponsors=20
- Subscribers =
Rapid responses for this article
There are no rapid responses = for this=20 article.Print issues
![3D"Access](3D"http://www.bmj.com/columns/icons/Non-EEAsmallBMJbutton.gif"){kind=icon}
=
![3D"Whats](3D"http://www.bmj.com/columns/icons/bmj-studentbmj-sml.gif"){kind=icon}