Jump to: Page
Content, Site
Navigation, Site
Search,

You are seeing this message because your web browser does not support basic
web standards. Find out more
about why this message is appearing and what you can do to make your experience
on this site better.

Search the BMJ

BMJ

BMJ Journals

BMJ Careers

BMJ Learning

Evidence Centre

BMJ Group

Search this site:

Advanced search

DUNCAN
BRIAN DOUBLE | View/Change User Information | Email alerts |
Sign
Out | Click here if you are not DUNCAN BRIAN DOUBLE

Home

Research

Methods
and Reporting

Education

Clinical
review

Practice

Shortcuts

Endgames

News

Comment

Editor's
choice

Editorials

Letters

Rapid
responses

Features

Observations

Head
to head

Analysis

Views
& reviews

Obituaries

Minerva

Fillers

Blogs

Podcast

Topics

Clinical
topics

Non-clinical
topics

Series

Theme issues

Video

Print
Issues

Last
seven days

Past weeks
(Monday-Sunday)

Print issue

Rapid
responses

Polls

Debates

Blogs

Podcasts

Video

Academic medicine

US highlights

BMJ USA 2001-2005

About
BMJ

Home > News > BMJ 2009;339:b5242,
doi: 10.1136/bmj.b5242 (Published 2 December 2009)

Published 2 December 2009, doi:10.1136/bmj.b5242
Cite
this as: BMJ 2009;339:b5242

News

Safety Data

The mysterious Dr Foster

Nigel Hawkes, freelance journalist

nigel.hawkes1{at}btinternet.com

A consultancy’s dire warnings about safety in many of
the UK’s hospitals dominated the weekend headlines. But were
their warnings right? Nigel Hawkes reports

Who is this Dr Foster, the source of the weekend’s headlines about
the NHS? He seems to be saying that the elaborate system for
inspecting hospitals set up—and recently reformed—by the government
is missing some pretty obvious examples of bad medicine.

Thousands of patients dying, hospitals failing to respond to
safety alerts or unexpected deaths, swabs and drill bits left
inside patients after operations ... isn’t this the kind of
thing any respectable government inspector might be expected to spot?
What on earth is going on?

Dr Foster is not, of course, a doctor at all, but a witty name
chosen by two journalists when they set up a healthcare analysis
company eight years ago to exploit the mass of data churned out
by the NHS ("Dr Foster went to Gloucester, in a shower of
rain...").

Tim Kelsey from the Sunday Times and Roger Taylor from the
Financial Times saw an unexploited commercial opportunity in
the work of Professor Sir Brian Jarman of Imperial College.
Measuring how well hospitals perform is not an easy task, but
Professor Jarman had devised a way of doing it by using Hospital
Episode Statistics (HES), data produced in huge volume by the NHS
for accounting purposes. His key paper on the subject appeared
in the BMJ in 1999.

Kelsey and Taylor’s timing was good. The Department of Health
under Alan Milburn needed evidence to support and monitor progress in
implementing the NHS Plan and, after some initial hesitation,
embraced Dr Foster warmly. The first Good Hospital Guide under the Dr
Foster imprint appeared in 2001. Hospitals were under notice to
collaborate with Dr Foster, however little they liked doing it.

So keen was Milburn’s successor Patricia Hewitt on the product
that she bought the company. Under a deal negotiated in 2006, £12
million of public money was invested in a joint venture between Dr
Foster and the NHS body, the Information Centre for Health and Social
Care, to create Dr Foster Intelligence. The fact that this deal went
through without a competitive tender raised some eyebrows, and was
heavily criticised by the National Audit Office. The department had
paid too much, and failed to give others a chance to bid, the NAO
ruled.

Professor Denise Lievesley, a statistician who had become chief
executive of the Information Centre when the deal was already
sealed, was also unhappy about it. She protested to her bosses
in the NHS. She protested again in 2007 when a contract to provide
material to the NHS Choices website was awarded to Dr Foster,
again (in her view) without proper procurement procedures. Shortly
after this she left her job, with a gagging clause preventing
her from telling her side of the story.

So the irony of the weekend’s stories is that a company granted at
least one sweetheart deal—and possibly two—by the Department of
Health has been the bringer of exactly the sort of bad news the
department would rather not have heard. There have been hints that
the department may have fallen out of love with Dr Foster. It lost
the NHS Direct contract in July 2008, and set up a rival website for
its Good Hospital Guide.

That said, is what Dr Foster says about the NHS to be relied upon?
Professor Jarman’s method of comparing hospitals is called the
Hospital Standardised Mortality Ratio, or HSMR. From the HES data the
number of patients dying after a range of different procedures in
hospitals is available. All things being equal, good hospitals will
kill fewer patients than bad ones.

The difficulty is that all things are not equal. Hospitals vary in
many ways—the area they serve, the age of their patients, the
efficiency with which they record the data. HSMRs are corrected, so
far as is possible, for these variables so that at the end of the
process one hospital can be compared directly with another. Professor
Jarman is a leading world authority on this subject, so nobody doubts
the probity of the process.

But many do question the results. One striking feature of this
year’s results is a sharp fall of 7% in HSMRs across England.
This is good news, if slightly too good to believe without careful
scrutiny, because it implies that many fewer patients are dying
in England’s hospitals than in previous years.

But the fact that HSMRs across the board have fallen so fast means
that the target has moved. The results are rebased so that a hospital
that exactly matched the national average would score 100—those with
more deaths above 100, those with fewer below 100. The rebasing means
that some hospitals who know they have improved appear to have done
worse, because they haven’t improved quite as much as the
average.

Others say the Dr Foster method misrepresents how good they are.
Alternative methods for calculating HSMRs exist, and in some cases
paint a much more flattering picture. One reason for this may be that
hospitals do not invariably "code" their patients fully. If a patient
has four or five co-morbidities (conditions that make survival less
likely) then this should be taken into account and corrected for. If
they are not recorded properly, then a hospital will have a higher
HSMR than it should. There may even—perish the thought—be the
suggestion that clever hospitals are exaggerating co-morbidities to
improve their HSMRs. It wouldn’t be the first time a target has
been "gamed" in the NHS.

Another possible source of error is that Dr Foster counts only
deaths that occur in hospital. If patients are discharged and
die the next day, at home or in a hospice, their deaths go
unrecorded. Professor Jarman has shown that in-hospital mortality is
a good proxy for all deaths within 30 days, but others are not so
sure.

So the Good Hospital Guide should be seen for what it is: a
sincere attempt to measure something very elusive, but very
important. It is not the last word, and nor should the mortality
figures be taken in isolation, without the other measures that
are included in the guide.

But is it more reliable than the Care Quality Commission verdicts?
Given the frequency with which good ratings from the CQC and
its predecessor the Healthcare Commission have been overturned
with a few months, Dr Foster provides an important corrective.
Both approaches have their virtues, but careful scrutiny and a
degree of scepticism should accompany the reading of either.

Cite this as: BMJ 2009;339:b5242

A version of this article was first published in the
Independent.

CiteULike
Complore
Connotea
Del.icio.us
Digg
Reddit
StumbleUpon
Technorati    What's this?

Related external webpages:

Follow the
debate on Dr Foster at doc2doc.bmj.com, BMJ Group's professional networking
site for doctors worldwide

This Article

Extract

Respond
to this article

Alert
me when this article is cited

Alert
me when responses are posted

Alert
me when a correction is posted

Services

Email
this article to a friend

Find
similar articles in BMJ

Find
similar articles in PubMed

Add
article to my folders

Download
to citation manager

Request
Permissions

Google Scholar

Articles by Hawkes, N.

PubMed

PubMed
Citation

Articles by Hawkes, N.

Related Content

Epidemiologic
studies

Related
external webpages

Bookmark with

What's this?

What's new

Last 7
days

Past weeks

Current print issue

Rapid
responses

Pandemic flu

Latest blogs

The
fight against HIV/AIDS

Funny
dead philosophers

Finca
bout it

Infant
mortality in Sierra Leone

Swine
flu in Bangladesh

Richard
Lehman's journal blog

Find BMJ on:

Latest
from BMJ

BMJ
simple search

Follow BMJ on
twitter

Services

Blogs

Podcasts

Submit an article

Subscribe/Activate

Email alerts

Request
permissions

Tools

Email
to friend

Print
this page

Online poll

Find out
more>>

Resources

Readers

Authors

Reviewers

Media

BMA members

Advertising and sponsors

Subscribers

Rapid responses for this article
There are no rapid responses for this
article.

Print issues

Current issue contents

Current cover image

Past issues

Subscribe

Contact us - Privacy policy - Web site terms & conditions - Revenue sources - Site map

HighWire Press - Feedback - Help - © 2009 BMJ Publishing Group Ltd.