Dec 21, 2023
In this episode Miles is joined by the National Statistician, Sir Ian Diamond, to reflect on what has been a busy and transformative year at the Office for National Statistics.
Transcript
MILES
FLETCHER
This is “Statistically Speaking”, the official podcast of the
UK Office for
National Statistics, I’m Miles
Fletcher.
This is our 20th episode, in fact, a milestone of sorts,
though not a statistically
significant one. What is significant is that we're joined, once again, to look back at the highlights
from another 12 months here at the ONS by none other than the
National Statistician himself,
Professor Sir Ian Diamond. Ian, thanks for joining us again. The
year started for you with being reappointed as the national
statistician.
As 2023 developed, how glad did you feel to be back?
SIR IAN
DIAMOND
Of course,
you know, I
was hugely
privileged to be invited to continue. It's one of the most exciting things
you could ever
do and I will
continue to do everything in my power to bring great statistics
to the
service of our nation.
MF
To business
then,
and this time
last
year, we sat
in this very room talking about the results of Census 2021, which were coming in
quite fresh then. And we've seen the fastest growth of the
population, you told us, since the baby boom of the early
1960s. Over the course of the year much more data has become
available from that census and this time, we've been able to make it available
for people in much richer ways, including interactive maps, create
your own data set tools. What does that say about the population
data generally and the way that people can access and use it now?
How significant is that there's that sort of
development?
SID
Well I think
we need to
recognise that the sorts of things that we can do now, with the use
of brilliant technology, brilliant data science and brilliant
computing is enabling us to understand our population
more,
to be able to make
our data more accessible. 50, 60, 70 years ago, 150 years ago, we
would have just produced in about six or seven years after the
census, a report with many, many tables and people would have just
been able to look at those tables. Now, we're able to produce data which
enables people to build their own tables, to ask questions of
data.
It’s
too easy to say,
tell me something interesting, you know, the population of Dorset is
this. Okay, that's fine, but actually
he wants to
know much more about whether that's high or low. You want to know
much more about the structure of the population, what
its
needs for
services are, I could go on and on.
And each
individual will have different questions to
ask of the data, and enabling each
individual to
ask those questions which are important to them, and therefore for the census to
be more used,
is I
think, an
incredibly beautiful
thing.
MF
And you can go
onto the website
there and create a picture...
SID
Anyone can go
onto the
website, anyone can start to ask whatever
questions they want of the data. And to get very clearly, properly
statistically disclosed answers which enable them to use those data
in whatever way they wish to.
MF
And it's a demonstration of obviously the
richness of data that's available now from all kinds of
sources, and behind that has been a
discussion of, that's gone on here in the ONS and
beyond this year, about what the future holds for
population statistics and how we can develop those and bring those
on. There's been a big consultation going
on at
the moment. What's the engagement with that
consultation been like?
SID
Well the engagement's been great, we’ve had around 700 responses, and it addresses some fundamental
questions.
So the census
is a really
beautiful thing. But at the same time, the
census, the last one done the 21st of
March 2021, was out of date by the 22nd of March 2021, and
more and more
out of date as you
go on and many of our users say to us, that they want more
timely
data.
Also
by its very
nature a census is a pretty
constrained data set. We in our country have
never been prepared to ask for example, income on the
census yet this is one of the
most demanded questions. We don't ask it because it is believed
that it is too sensitive. And so there are many, many, many
questions that we simply can't ask because of space. There are
many more questions that we simply cannot ask in the granularity
that we want to. We've been doing some work recently to
reconcile the differences between estimates in the number of Welsh
speakers from surveys with estimates on the number of people in the
census who report they speak Welsh. Frankly, it would be better if
we were able to ask them to get information in a more granular way. And so while the
census is an incredibly beautiful thing, we also need to recognise
that as time goes on, the technology and the availability
of data allowing us to link data becomes much more of a great
opportunity that we have been undertaking a
lot of research, a lot of research which was
asked for by the government in 2013, following the report by Chris
Skinner, the late Chris Skinner, Joe Hollis, and Mike Murphy, which is a
brilliant report. We said at
the moment we
need to do another census in 2021. That's what we have
done and I
believe it to be one of the best coverages
there has ever been. And yet we need to assess whether
administrative data could be used in future to provide more timely, more flexible and more accessible data
and that's what the consultation is about.
I will be making a recommendation to the UKSA (UK Statistics
Authority) board in the future. In the near
future we have to say, and I think it is worth
saying that what the consultation says to us is that people are
very, very, very much in favour of the direction of travel but at
the same time as yet accepting our prototype, unconvinced about the
data flows and the sustainability of those data flows to enable us
to do it and so, we are looking at how to respond to other very
important analyses and we will do so in the near
future.
MF
When can the people who
contributed to that
consultation, roughly when should they expect to hear
from us?
SID
I think the expectation
is
we'll
publish something by
the end of quarter one in
2024.
MF
Surveys have continued to
be a very
important part of what the ONS
does,
these
very large
national
surveys, and yet one of the biggest
challenges of the has been maintaining coverage and particularly
response rates and obviously, particularly with the Labour Force
Survey recently that has been a particular issue for the ONS
hasn’t
it. Where do
things stand
now as we move into
modernising the traditional Labour Force Survey and moving to a new
model because it's an issue statistics bodies
around the world have been dealing with, it's harder to get people to complete
surveys like they used to.
SID
I think it's a fair point that response rates
globally are a challenge and response rates globally, not only in
national statistics issues, but in the private sector organisations
that also collect data,
are a
challenge.
So we need to
recognise that. A part of that is that
historically, one could find people at home, knock on
doors, have that conversation
with
people,
and
perhaps post
pandemic
people
are less willing to
have a conversation at the house. Also, people are
very
busy. They
work in multiple occupations. They are not always
in,
they live in housing
accommodation which is more and more
difficult to access.
This there is no kind of single magic bullet here that we could
press all we would have. The first thing to say
Miles is that we recognise that
and that's why we worked with our
colleagues at His Majesty's Treasury to provide a project to go to what we
call a
transformed Labour Force Survey. And I think
that that's a hugely exciting project
for a
number of reasons. One, the
labour force
survey which has
been around for a long time, the questionnaire had become a little
bit unwieldy. And
also we
wanted to enable people to have much more flexibility at the time
of which they answered the question. We are in the field with the
pilots for that service. We've been pretty good. There are good response rates.
There are also some challenges around getting the questions right.
These aren’t challenges that stress me, that's why you do a pilot, but at the
end of the day we're hoping to be able to transform
into that new Labour Force Survey early in 2024, in the first
half
of 2024. We're working very closely in doing
that with our major stakeholders and the Bank of England, His
Majesty's Treasury and the Office for Budgetry
Responsibility
(OBR), is
you take a joint
decision on when people feel comfortable that we have had
enough dual running to enable us to move
forward. The other question that I'd have to raise around surveys
more generally, is on inflation, which we have all
been subjected to in many, many areas in the last couple of
years,
inflation in
survey
collection has
increased massively and so in the last year we've had to make real judgments about
how we maintain quality. And in the next few
years, we will really be needing to think through exactly how we
conduct our surveys and the cost of doing so.
MF
Yes. Of all the people who should be aware of
inflation are the people who report it, and certainly the impacts of
those relatively high
rates of
inflation have impacted us as much as anybody
else. The challenges not withstanding
of
running
surveys, the interest of government
bodies in getting that information directly from people does
continue to underline the unique value of
surveys.
Some people say
Oh, well, they
surely they
can get this information from other sources
I've even seen it suggested that
social media
could provide the answers, but there is a unique
value isn't there and actually
getting a
statistically representative sample of people and speaking to them
directly.
SID
It depends Miles, I think it absolutely depends on what the
question is you're trying to answer. If
you're
trying to get some
answers to a question where the answer can be obtained through
administrative data sources, then you
don't
need a
survey.
Surveys are
difficult to conduct and difficult to pilot and plan and extremely
expensive to undertake.
So you should
only do a survey if you can't get the information from
somewhere else. Therefore, you know, I do think that we need to
be very, very
careful in
thinking through when we need to do surveys. Does that mean to say
we don't need to do surveys? Absolutely
not. There are reasons why you need to do surveys. It may be that
you need to really spend some time identifying whether someone really is
eligible for hte
questions
you're
going to
ask or you
may want attitudes. I don't know how to get someone's
attitude without asking them. And so there are reasons why
you would want to do a survey, but I would argue that you should
only do a survey when you cannot get the data from elsewhere. And
you also mentioned social media. Social media is an incredibly
interesting and important source of data. Now, I
wouldn't
necessarily say it
was statistically representative, but we absolutely
have
to be
flexible in what we call data. We have
to be sure of
the quality of those data and we have
to be sure
that we are really
aware of what
the population is that are represented by those data.
So we are
using many, many, many types of data now that we would not have
used 50 years ago, we simply couldn't have used things like telephony
data, things like card data, things like data from satellites to
address questions which those data are the best way
of providing answers.
MF
And there are some fantastic
examples of that around the ONS. If you look at how we've
changed prices over the last couple of years, again, the
measurement
of inflation, bringing in new data sources
most recently from the US car industry, from the rail industry as
well and it all means that the estimates of inflation are now based
on many hundreds of thousands of price
points, where it used to be just a few
things.
SID
It doesn't matter what the numbers are,
frankly, it matters that you've got a good coverage it matters
that you have the most appropriate method
and that your data
are as accurate as possible. And I do think it
is incredibly important. We use a wider range of data sources. I
think it's incredibly exciting what those
data sources are, but we should only do so being unbelievably
careful about what the metadata are that go with
them,
what the coverage
is, why we are using them and whether
or not they represent an improvement over what we
could do before.
MF
Okay, so we've seen in
the area of
prices,
the measurement of inflation,
there's new innovative data sources coming from
outside, coming from industry. What sort
of an improvement does that represent in how we measure inflation,
when
it's
such
an important
time for cost
of living?
SID
Well, I think it helps because we have
more accurate data. We have more
timely
data, we have
data that are real.
So on rail
prices, we know what people pay as opposed to what the price as
advertised necessarily is and I think that is important. And so being able
to properly understand what the consumer is doing, therefore,
what inflation is, is to me, incredibly important. I would
say that all this effort that we're putting in would not necessarily
just be about prices. Here it is about do we understand more about
what is going on in the economy, and there are many more questions
that we can ask from those data when you've got them, and simply from some of the
fixed price point data that we have
previously.
MF
Now one massive change
we've
seen
lately, and this is another area
we've
managed to improve
coverage, is of course the private rental
sector. It’s become much more important
as we've seen house prices coming under
pressure and mortgages under pressure by high interest
rates and so forth. It's revealed a very interesting
picture
of
long-term change, and
also in more
contemporary terms, what's actually
going on with
the economy right now.
SID
Oh 100%
MF
Talking about areas where
we've
been able to form a
new view of what's really been going
on.
An area that
attracts a particular commentary during
the course of the year is expenditure
on research
and development. Regarded as a very important
area of
activity if you're talking about productivity,
future economic growth...we substantially upgraded
our estimates
of R&D. What was the story
behind that? Why was
that necessary?
SID
Well it was
incredibly important because we looked very carefully
at our data,
we look very
carefully at our samples, we looked at our coverage and
we decided that we needed properly to to
bring in a much
wider range of business. And we were reflecting very much those
businesses from a very wide range of areas who were able and
available to claim R&D tax credits, and therefore to be able to get
a decent sample, and the critical thing here is
not only were we making good estimates, but we were able to
understand much more about what, particularly for smaller tech
and creative industry companies, was R&D. And I think that is something that we need to
recognise particularly in those smaller companies where
there's
a much greater
flexibility about what people would call R&D.
MF
It’s a reflection perhaps
that startups are
the
sort of firms that
do R&D these days, and less so the sort of industrial
behemoths with huge
R&D departments. But there was an interesting
change nonetheless, and obviously considerable improvement in
measuring that very important area. This all I guess comes
under the umbrella of future proofing practices and systems and
this all came under a refreshed data strategy that we
launched during
the course of the year. One of the fundamental
principles underlying that, where is it taking
us?
SID
I think, I mean, just where I've
been coming from, are to do a much more holistic view of what data
are and how we really use data which are most appropriate to answer
the questions that we have, and we recognise that the
economy and indeed society are changing very quickly, and therefore
we need appropriate data to be able to answer those questions. For
example, if you look at employment, there are many, many people in
our society who have three, four, even five jobs, we need data which enable
us to find out what the distribution of the number of jobs people
have is, what they're spending their time doing, and how that
impacts on our understanding of the labour force.
MF
Worth perhaps recognising
some of the
particular
areas where
new data has also been able to shed new light and particularly
think of the payments industry which obviously digital payments
happen very quickly. They provide almost a daily update on the state of
consumer spending. With it obviously the state of the of the of the
wider economy. We've managed to strike up
partnerships with a huge cross section of the payments sector. What
is the particular
value of
that? And what do we say to perhaps other
data providers who
might wish to enter
into similar arrangements?
SID
Well I
think we’d say we do everything ethically,
and with complete privacy, but at the same time in the public good.
And that is, to me, incredibly important.
And
so understanding what the consumer
is spending money on understanding what the consumer is not
spending money on, and the
transitions, is incredibly important to
enable policy which impacts very positively on
all
of our fellow
citizens.
So we
are very
proud of
those partnerships. We value them greatly. We don't take them for granted. And those
data,
entirely ethically
provided, with great security but at the
same time enabling us to understand what is going on at an early
stage in the economy is incredibly
important.
MF
And
of course it’s worth restating, as
mentioned
already, that of course
all
of these data are anonymized and
aggregated,
and no individual
would ever be able to identify themselves or be
identified
from that fast
payments data which of course is helping to inform economics
policy.
MF
Providing data to the people who do make
policy and around government and to make sure that policies are
really informed by evidence of course that is the major purpose
behind the new Integrated Data Service, which was accredited
this year under the Digital Economy Act. And that's enabling data to be shared
around government in a way that simply wasn't possible
before.
SID
80 datasets available now and
indeed, that number going up more
or less by the day. And one of the most important
things here is that there are very few
challenges which government face which simply can be addressed by
data from one department. Therefore, what we need to be able to do
is to link data from different sources
to enable us in a
very granular way to be able to answer questions about topics for
which the answer requires data from many sources. And the
Integrated Data Service allows us to do that. It allows us
to do at a pace and
allows us to do it in a way which brings a wide variety of analysts
to the party. And I think that, you know, this year major
milestone in getting Digital Economy Act
accreditation. And we will be looking to
streamline the process of using it over the next
year,
as well as
seeing more
and more and more projects on it
having successful
results.
MF
And sharing between departments at
the national level is important, but also it's been a long-term aim of the ONS
to improve its
coverage at local levels. And again, there's another important initiative
kicked off this year, and that's the launch of
ONS
Local.
SID
Yes and I’d say that the two are linked.
It doesn't matter whether you are at a
national level or whether you are at a regional
level,
linked data
are important, but we are very pleased working with funding from our
colleagues at the Department of Housing, Levelling Up and Communities to have been able to
place ONS staff in regions.
So
we're not
talking about teams of people in Manchester or teams of
people in Exeter, but we are talking
about interlocuters
in the
southwest, northwest for example, who can
really work with the leaders there to ensure that we've got local
data for local leaders to make local decisions and that's
incredibly important because the questions that people wish to ask
are different in different parts of the country and therefore we
need to recognise that so it is a good initiative, which I hope
will bear fruit in 2024.
MF
And the importance of data in
government has been underlined by a big initiative, which takes in
everybody, not just statisticians and analysts, but everybody in
the civil service, has been engaged
in
what's called the One Big Thing campaign to spend time
learning about data that's important to the use of data.
How has that initiative been going? The ONS has been a central part of
that.
How's
it been going? How
important is it?
SID
It is critical.
We do not
need every public
servant to be able to be a brilliant statistician, but we need
every public servant to be data literate. We need every public
servant to be able to understand data and the best policy comes
about when analysts and policymakers and potential beneficiaries
work together. And that requires that you can have that data
literate conversation. And
so I
think One Big
Thing is a great thing.
MF
In fact that the need for people to
better understand data became evident early this year, of
course, when our GDP revisions were quite
dramatically revised in the early part of the autumn as the
estimates for the big peak pandemic years, 2021 and 22. There was quite a
reaction from some parts of the media and beyond, who reported that our original
figures were, because they had revised so
dramatically, were simply wrong. I mean, that's not the case. revisions of
course have always been integral part of the process.
Indeed the
OSR, the statistics regulator, found as part of its review
our approach to
be, and their
words
were
appropriate and well managed, however, it also found the
ONS could communicate better
the uncertainty in those early estimates of GDP
and
that's a learning point for the
future.
MF
We saw particular attention
recently for the natural capital outputs, measures of the natural
environment, and they attracted a degree of
media interest we haven't seen so
far,
helped by the fact
we're able to bring it to life with an analysis of time spent in
nature and so forth, and you spoke to BBC Countryfile
about that particular work. What's your overriding thoughts on that
release? Are we moving to the point where
these kinds of measures are getting more
exposure?
Are
they being recognised for
their value?
SID
I thought the
national Natural
Capital stuff was
brilliant. I've always
thought,
as I said last
year,
that we should put
alongside GDP measures of the environment and measures
of
well-being, but you need a concise picture
and
that's where we're moving in the
future.
MF
As we speak, we're heading into the bleak midwinter
of 2023. The nation is doing all it can to avoid a seasonal bout of
flu and the other viruses that traditionally do the rounds at this
time of year. And that's seen a revival of our surveillance
effort. The Winter Coronavirus
Infection Study (WCIS). Tell us about
that.
What's the purpose of it
and
what's happening?
SID
Yeah, working very much for our
colleagues at the UK Health Security Agency who asked us whether we
would be prepared to stand back up some of the work we do on
surveillance of winter flus, COVID and other
issues. and we're of course pleased and proud to
be asked.
We’re using a different strategy to the
one we
were
using
in
the
past,
this is
very much simply a
mail
out of tests
enabling people to take a test and then to make
estimates,
and at the
moment the good news is that the
estimates of positivity are relatively
low, but the bottom
line is we need to recognise that without some good
hard
data on those
levels
it's pretty
impossible
for government to plan, and so I think it’s a
really
exciting
initiative. It's a smaller survey
than
one
in the
past.
It's a survey which will make
national estimates rather than many regional estimates, but it's one that we think is extremely
exciting, and builds on some of the
work
we've
done in the
past.
MF
And now of course everyone knows
how to
self-administer
a COVID
test and that
ability makes it much easier to run these
big.
SID
Oh 100%. I do think we need to recognise
the way in which the world moves on. And certainly, when we first set up the COVID
infection survey in 2020. We were not aware of the extent to which
people could self-administer,
we
learned pretty
quickly
that's why we were able to
transition to self-testing, but I think we are in a world where we can do
this at pace and provide estimates very, very
quickly.
MF
Well, thank you very much for
joining us. Great to have you with us again at the end of the year.
You could choose just three words to sum up your
2023
SID
Exciting, full of change
and
high-quality
statistics.
MF
And looking ahead to 2024, which pieces of
work
are
you looking forward to
most?
SID
The economy is changing
quickly, society is changing quickly. We
will continue to change and to be ever more
effective. We've talked about some of the
things
we're
bringing on board
and looking forward to a brand-new website to improve our
communication. And I think it's going to be a very exciting time.
MF
Professor Sir Ian Diamond, thanks very
much for joining us.
That's it for another episode
of Statistically Speaking, you can subscribe to future
episodes of this podcast on Spotify, Apple podcasts and all the
other major podcast platforms and
also
follow us on
X,
previously known as
Twitter, via the @ONSFocus feed. I'm Miles Fletcher, and from myself, our producer Steve
Milne, and everyone here at the ONS, we wish you seasonally adjusted
greetings, goodbye.
ENDS