Statistically Speaking: In The Field: Surveying the nation

In The Field: Surveying the nation

Jul 31, 2023

In this episode we chat to members of the ONS Social Survey Collection Division about the importance, and challenges, of getting the general public to take part in crucial surveys that help paint a picture of what life is like across Britain.

Transcript

MILES FLETCHER

Welcome again to ‘Statistically Speaking’, the official podcast of the UK’s Office for National Statistics. I’m Miles Fletcher.

Now I don’t know about you - but it seems hardly a moment passes these days when we are not being asked to feed back. How was our service today? Are you satisfied with this product? Please fill in this short survey. Your responses matter.

Well, forgive the natural bias, but today we’re talking about surveys that really do matter.

ONS surveys – some of which are the very largest conducted regularly in the UK – don’t just inform economic and social policy, though they are hugely important to it. The data they gathered also represent a public resource of immense and unique value.

But persuading people – some unaware, some sceptical and even hostile, others just very busy – to take part in them is a growing challenge for statistical institutions worldwide.

In this episode then we’ll be discussing how the ONS gathers often personal data from members of the public right up and down the country.

Taking time out of their day to answer my questions, and to explain why it’s absolutely crucial that you participate in our surveys if you get the opportunity to do so, are Emma Pendre and Beth Ferguson, who head up the ONS’s face-to-face Field Operations;

and sharing their own personal experiences of life on other people’s doorsteps we have two of the ONS’s top Field Interviewers, Tammy Fullelove and Benjamin Land.

Welcome to you all.

Emma, if I come to you first – give us an idea of what exactly the field community in ONS is. Who are you and what do you do?

EMMA PENDRE

The Social Survey Collection Division is the largest division in ONS. We primarily collect data from households either online, face to face or by telephone using computer assisted interviewing, and also work at air, sea and rail ports collecting data from passengers. All the data collected is used to produce quite a number of our key ONS publications which help to paint a picture of what life is like in the UK. And these can include things like estimates of employment and unemployment, how we measure inflation, how we measure migration, and a key topic of interest at the moment is the cost of living. So while most of ONS relies on the data that we collect for our outputs and statistical bulletins, the statistics that we particularly generate also support research, policy development and decision making across government and other private sector businesses as well.

Now Beth, bringing you in here, when it comes to household surveys, presumably someone's deciding which households are going to be approached to take part. Who makes those decisions and how is it done?

BETH FERGUSON
So I'm not going to pretend to understand the clever people in the statistics team who work out how we get the right people to cover a broad spectrum of society. But yes, that's done by the sampling team and they choose a random sample for the surveys.

And that's generated presumably from using the electoral roll.

It's generated from something called PAF which is the Post Office Address Finder. I'll have to confirm exactly what that stands for. Yes, but essentially, it's a list of addresses across England, Scotland and Wales.

And when it comes to the passenger survey, it's a question of stopping what we hope will be a representative random sample of people as they pass through those ports.

Yes, it is. Yeah. But at the moment we're currently working on departures and arrivals. So yes, it's a random sample of individuals stopped and asked questions.

But to make the data really representative and really valid, of course, we've got to be covering the whole of the country. The country in this case being Great Britain. How do we ensure that that coverage is working day in day out?

That's our role as the kind of management of the face-to-face field interviewers. Different surveys are done over different frequencies. So we've got the Labour Force Survey and the transformed Labour Force Survey which addresses are issued for on a weekly basis and those surveys are delivered on a weekly basis. And then we've also got our other longer, more detailed financial surveys that we’re issued with a quota for on a monthly basis. So our job is to make sure we've got the right people, in the right places, to knock on the right doors, to get hold of those members of the public and, you know, encourage them to agree to complete surveys for us.

And luckily for us we’re joined by two of those “right people” here today. Tammy and Benjamin, welcome to our humble podcast. Now you are both at the sharp end of our survey data collection, working as field interviewers. I'm obviously really interested in what you do day to day, but first off tell us how you got into this line of work. What was the attraction for you Tammy, how did you become a field interviewer?

TAMMY FULLELOVE

So prior to working for the ONS - I've never worked in public sector before, I've always worked in the private sector - and I've actually got a finance background. But then after being on maternity leave, having a young family, seeing the job advertised and the flexibility working with people in a very, varied job sort of pulled me to it to apply to be honest. And that was seven years ago, and I can honestly say I enjoy every single day I'm out in the field. It's great.

And Benjamin how about you, what was your background?

BENJAMIN LAND

Well, I've done a variety of hospitality jobs in the past. I then applied to work on the Census at the start of 2021. And my manager at the time she had worked previously for the ONS on the basket of goods figures, and she recommended it as a really great place to work. It's funny how timing happened I saw a vacancy for a field interviewer, which I applied. And then I started in May 2021. So almost two and a half years ago now.

Okay, so you've both got quite a bit of experience already under your belt. I was wondering of both of you, is there such a thing as a typical day for a field interviewer?

I can honestly say no, every day is completely different. Depending on the area where you go into, where you may be working, streets apart, houses apart. You never know what door you knock on who can be behind that door, which makes every day completely varied, especially with the studies that you may be interviewing for, that they can be very different with the content. So yeah, two days are never the same.

I totally agree with Tammy. It varies. My week has a sort of flow to it. So I tend to get out quite a lot at the start of the week to visit various addresses. If it’s LFS they change every week. On the financial surveys it's monthly so you've got longer to familiarise yourself with the area. We tend to have a team meeting most Tuesday mornings just to check in and see how we're doing. And then obviously interviews are scheduled around respondents’ timetable so that can be any time up to sort of eight, nine o'clock at night and sometimes Saturdays, if that's when they're available.

Going out to people's houses on a daily basis, you no doubt encounter a wide variety of people. That must have led to one or two amusing episodes.

TF
I've had occasion where people will answer the door in not the most suitable attire, shall we say, for public viewing. I don't know how much further to go into this, but yeah, definitely opening the door in towels which have fallen off and dressing gowns which haven't been completely covered. It definitely happened a couple of times over the past few years.

Perhaps that's what they mean by “raw data”.

Beth, if I can come back to you, are there particular surveys which are considered to be especially important for us to be speaking to people in their homes in the way we’ve just been talking about? Ones that perhaps can’t be carried out in other ways.

BETH FERGUSON

It’s the more detailed financial surveys. So we've got the Family Resources Survey, the Living Costs and Food Survey, the Survey of Living Conditions, and the Household and Assets survey. They are quite long, more detailed surveys. The living costs and food survey, that requires the respondent to complete an interview, but then they also have to get hold of all their receipts of any expenditure for a two week period and annotate them and hand those over to the interviewer. So it's quite a detailed, involved survey. The Household and Assets survey, again it’s dependent on how many people in the household can, you know, take up to two hours to complete and ask lots of detailed financial questions around savings and pensions and other things. If you're in the home, you can ask them to get the documents, support them to review the documents, make sure that they're actually giving the right information which, if they were to go online and do it themselves, there's no guarantee that they would get the right detail that we're actually looking for.

So it's quite an intensive experience really, isn't it compared with simply asking someone to tick the boxes on a webpage? And I guess it very much depends on building a personal rapport with the survey participants?

Absolutely. And that's the key. That's the key to a really successful interviewer is that ability to build rapport in a snapshot on the doorstep. You know, before they've had the opportunity to give a polite no, no thank you or sorry, not today. They reckon it is approximately 10 seconds on the doorstep to get that engagement and build that rapport, and then maintain that through what can sometimes be quite a lengthy interview. Keep that friendliness, that rapport going so that the person being interviewed remains engaged and keen to do it.

Now Tammy, you’ve already told us about your previous financial background. Do you find that helps you when you're collecting data on economics or topics around money?

TAMMY FULLELOVE

Yes, I do. Like Beth’s already mentioned, a couple of our financial studies go into people's income and expenditure. So having that sort of background I feel does help me, especially when they're speaking about what benefits they receive, what sort of things they pay out. It definitely does sort of give me the edge I do feel.

That’s great, because it’s no secret is it Beth that the ONS, like other statistical organisations around the world, are finding it increasingly challenging to get people to take part in surveys.

BETH FERGUSON

Yeah, absolutely. I think it's got more and more challenging. Pre-pandemic it was getting more challenging, but the shift during and post-pandemic has been quite significant in terms of the number of willing people to do surveys for us.

A shift in what direction?

Fewer members of the public are willing to actually do surveys for us. Now whether that's because there's less trust in the government or actually, because of the pandemic, everybody's working from home and time is more limited. But no, it's definitely harder to get a response now.

What techniques do we use then to try and change people's minds to get them to take part?

At the moment we're doing a lot of work, certainly with the face-to-face field community - we're calling it a Respondent Engagement Programme. So looking out for clues and signs from, you know, when you approach the doorstep in the area, identifying the kind of things that may be key to them. Our statistics on things like CPI and RPI and, you know, the change in cost of food - that being constantly in the news gives us, kind of like, a lever to start an open conversation on the doorstep, particularly when we're looking at the financial surveys.

EMMA PENDRE

And also Miles. It's worth noting that all the surveys are voluntary, so the offer of incentives such as vouchers in exchange for the time taken to complete a survey will also continue to be significantly influential in maintaining our response rates.

Absolutely Emma - offering people a small incentive has actually been proven to work hasn’t it, and I guess in cost terms, it's better to spend some money on that rather than wasting it on chasing people who are never going to take part.

Yes, that's right. The vouchers are very significant. They do help maintain our response rates. And again, being in a cost of living crisis at the moment. Our respondents see them as very helpful.

But even with incentives, and as Beth has suggested, there’s still a reluctance by some people to be involved in our surveys. Coming to you Tammy and Benjamin, as our people on the front line every day - upon your shoulders falls the responsibility for persuading people in many cases to actually take part. Do you have a standard approach, or do you tailor what you do according to particular circumstances?

TAMMY FULLELOVE

We definitely have a doorstep introduction, which has to cover a few different points to obviously make sure respondents are aware of the confidentialness of obviously the answers that that will be providing. But I do believe having a smile as soon as they open the door is the biggest thing - you're obviously trying to get them on board and trying to get them to either go online to complete the study or to make an appointment if they can't do it there and then to do the interview. It definitely has to be tailored I think, compared to who answers the door and obviously what reasoning they would like to help complete the study. Whereas some people as soon as you knock on the door, they've had the letters, they're waiting for you. They really want to help. So yeah, it definitely does depend on who's behind that door and obviously why they would like to help the Office for National Statistics.

We live in a suspicious age and some people might think that there's something fishy afoot.

BENJAMIN LAND

That's the challenge Miles is people often initially they think it's a scam. I turn up with my badge and they're like, “Oh, you are real”. And taking the time to explain to people once we've done the doorstep introduction that it’s not a scam and it is legitimate, valuable research that we're carrying out and it certainly impacts everyone.

I can imagine how tricky it must be to convince people sometimes, but you strike me as someone who isn’t likely to be put off by that.

BL
Yes, yeah, I love a challenge. There was one lady last summer and every time she was like, “Oh, I'm busy. I've just come back from holiday, can you pop round the next week?” And it got to the point where she's like “I'm decorating my house.” I said, Ma'am, that's fine. I'll come and help you decorate your house if you complete this survey. And she's like, “Oh, you're so persistent.” I managed to get an interview and I was really pleased about that. So there's a little, you know, a little win in the bag.

Well done, though I should point out that painting and decorating is not officially one of the ONS’s services for getting people to take part in surveys. Tammy have you got experiences like that?

Yeah, I've never got into painting and decorating, I'm gonna admit that. But it is a great feeling when the first time you knock on the door people don't want to help they're too busy, especially now post-COVID, with the amount of people working at home. So like Benjamin said, you're interrupting a Teams call. You're interrupting them doing some work. So you have to get over that first hurdle. But, you know, making that appointment, and sometimes they will make the appointment but then they either won't answer the phone, or they won't be in when you turn up, which can be frustrating. But yeah, when you actually do complete that study and they do feel like, you know, they have helped and you've gone above and beyond to secure that interview, it is definitely a great feeling. So maybe I should be offering painting and decorating services, maybe that would help.

Don’t take my tricks. No, the sense of achievement or, like Tammy says, you do get people that break appointments, you know, due to personal circumstances, and you somehow have to chase people and encourage them, but when you do secure the interview, and you get the data. There's something about when something’s hard won you value it more.

Yes, indeed. But how many people have heard of the ONS would you say?

A lot of people now, because we were quoted a lot during the COVID statistics, regularly on the news and is quoted... I read the newspaper I appreciate not everyone does. But a lot of the data in newspapers it will state that it's been sourced from ONS.

That recognition factor has helped help you on the doorstep. Do people get that the ONS is an impartial organisation operating at arm's length, certainly from ministerial government?

No, no, I think we’re often tarnished with the same brush as the TV licencing people that come round, especially in certain areas where I knock on doors. You know that they were met sometimes with hostility, to put it politely.

Clearly some persuading to be done in a wider sense there as well. But is that your experience too Tammy?

Yeah, they do believe that we are a government body and that we are influenced by a particular minister, or by the government that's in power at the time. If people are very anti-government on the doorstep it does create that hostility as the first sort of part of your introduction.

Do you try to talk them around on that?

Yeah, exactly like Benjamin said that really, you know, we don't have a minister in control of us. We are separate to the government. Everything is private, confidential. We don't share the information. You know, there's different a few things that you have to try on the doorsteps to try and get that buy-in from the respondents.

Those of us who live and breathe statistics, of course, we wouldn't need to be persuaded to the value of taking part, but the challenge is to convince the whole population, or at least a representative sample of the whole population. It seems well removed from everyday life for a lot of people, but how many people do you think “get it” in terms of the value of statistics, you know, particularly economic indicators and high-level population data?

I think it's great if you can get to an area. And you know, that statistics, whether it's been from some sort of government funding, have helped in the area. So you can say to someone on the doorstep, well, the reason why this school was built or this doctor surgery, or this park, or some sort of local information, really does that help to sort of say why they are important to provide the information. But on the flip side, speaking to students who will obviously do research looking at the ONS data, they might be using, obviously, in their own work and people who work in sort of the public sector, I think, do understand to a degree how important it is. But then, I think the vast majority and I think Benjamin will help me with this, don't really understand why we’re collecting this or what benefit the information could have to them and where they live.

That's right. I think a lot of people they have a global sense of it, but they don't understand the impact it has on their life and I work quite a lot in Bournemouth. And there's a lot there's a big student population as we've got Bournemouth University and the Arts University College, and a lot of the students do actually know or use the ONS data. I was actually at a student house yesterday in Winton and that makes my life much easier if I can link it to their own studies.

In covering the whole of the country, of course, that means covering areas, which in statistical language are “hard to reach communities” - that’s the phrase that's used. And frankly, of course, that often means areas of considerable social deprivation. Emma... How do we target those areas in particular, does that require extra attention or special techniques?

EMMA PENDRE

So our vision is to be fully inclusive by design. So that ensures that both the data and our workforce are fully representative of the population that we serve. The pandemic actually opened up opportunities and challenged how we have historically done things in the ONS. So to give a specific example here is around one of our key data sources, which is the Labour Force Survey. Before the pandemic we would write to addresses randomly, selected from our database of all UK households, and invite people to take part, and then knock on their doors to follow up if we didn't hear back from them. During the pandemic, when face to face interviews became impossible. We had to rely on people responding to the letter and taking part in the telephone interview. We saw pretty quickly that this was leading to bias in the responses, with particular demographics, such as the older population being more likely to respond where we were less likely to hear from people who sort of rented their properties. We knew we needed to speed up the work already underway to improve the survey. So fast forward two years and we now have transformed the Labour Force Survey and making it an online first survey which is now supported by telephone collection where needed. We've proven that to make the survey inclusive and reduce bias we also need to be knocking on doors. So for households invited to complete the survey from November 2022 They now might get a visit from a field interviewer who encourages them to complete the survey online or via a telephone interview, and we call this mode of field work “knock to nudge”.

So in other words, it's not enough just to send somebody a letter inviting them to take part - that's likely to go unheeded. But a friendly face at the door and a little bit of gentle persuasion, can have a really useful effect.

Absolutely. Right.

And this is very important, because the ONS has committed with the Inclusive Data Task Force to make a special effort to ensure everybody is represented in official statistics, and field communities have been involved in that work.

Tammy, you operate in an area that's quite ethnically diverse. How do you bridge barriers in communities where English perhaps is not the first language for a significant number of people?

TAMMY FULLELOVE

So in the North West, we do have a number of regions where it's densely populated, very different cultural diversity, I suppose obviously, London would cover the same. And we do rely on interviewers who speak second languages, who can then translate the languages of the people on the doorstep to go through the interview, or even just to help on the actual doorstep to speak to people and advise what the study is about and to make the appointments.

MF

And Beth... when it comes to choosing field workers I guess it's very important as well that you've got people who are not only representative in the general sense of those communities, but actually have got some understanding, some feel, for the people they're dealing with.

BETH FERGUSON

That's part of the skill of a field interviewer. And I guess it comes from the fact that we've got interviewers from all different backgrounds, but it also comes as they learn the role, understanding which areas you know are going to be more challenging, where you're going to have to put a bit more effort in and understanding that actually... as an interviewer you can knock on a number of doors and, you know, you know who's going to be easy, you can get interviews relatively easy from various different sections of society. And you know that's going to be easy, but you also know that if you're going into an area that's more deprived, you're going to have to put more effort in, you know, and for some interviewers it comes immediately and, for others, it's learned over a period of time, where those more challenging areas are, what's actually going to work, what's going to resonate with the people behind the door that, you know, you're going to need to get that interview from to make sure the data is representative of everyone.

MF
Now, Emma, let's talk a little about the future of the field community, because obviously we hear so much now about big data and the ability to discover and gather insights from that. A mountainous array of data sources that can now give us rapid, fast data, covering just about every topic you can think of. But, nevertheless, the ONS sees value in continuing to run these very large, and very personal surveys face-to-face and over the phone.

EMMA PENDRE

Yeah, social surveys will continue to have an essential role to play in ONS’s future, but also as part of a joined up data acquisition approach as well. I don't feel it's any longer a competition between whether we use surveys or other data sources. We have now come to realise that we actually need to work together and complement each other. So surveys are still fundamental in collecting the data that other sources cannot provide. And whilst new types of data sources are allowing us to more rapidly take stock of what's happening in our society and economy, they can't tell us everything or provide insights on things like personal opinions, attitudes, or exactly how people might be feeling at a given point in time. That will only ever be possible from talking to people.

MF

And on that note, can I just say that it’s been a pleasure talking to all of you today.

[OUTRO MUSIC]

That’s it for another episode of Statistically Speaking.

Thanks once again for listening, and also thank you for taking part in our surveys. Without you all the incredibly valuable information we get from our surveys – which help to inform better decisions by your local council, for instance – would simply not exist.

If you haven’t yet had the opportunity to take part, and you get a knock on the door in future from one of our field interviewers, please do answer and take the time to respond.

And if you happen to be in the Bournemouth area of course, and need some painting and decorating doing, then Benjamin’s your man!

You can subscribe to new episodes of this podcast on Spotify, Apple podcasts and all the other major platforms. You can also get more information by following the @ONSfocus feed on Twitter.

Special thanks to producers Steve Milne and Julia Short.

I’m Miles Fletcher and until next time... goodbye.

ENDS