Food and You 2: Technical Report: Weighting

Overview of weighting

The same weighting approach has been taken in all waves of Food and You 2. Weights are initially calculated separately for each country in two stages:

1. Calculation of selection weights (described in the section on selection weights)
2. Calibration of selection weights to country population totals (described in section on population weights)

Next, weights are created for use in analyses of combined-country data by scaling the weighted country sample sizes to be proportional to the corresponding country population values (for adults aged 16 and over).

Because it is not possible to include all questions in the postal questionnaires (see the section called ‘Questionnaire development and cognitive testing’), four separate question-type weights are calculated in each country, and in the combined all-country sample. For most waves, four question-type weights are designed to be used as follows:

1. All-questionnaire weights to be used for questions asked of all sample members in all online and postal questionnaires
2. Online questionnaire weights to be used for questions asked only of online participants (i.e., questions not asked in the postal questionnaires)
3. Online questionnaire plus version 1 postal questionnaire weights to be used for questions asked only of online participants and postal questionnaire respondents receiving only one of the postal questionnaires
4. Online questionnaire plus version 2 postal questionnaire weights to be used for questions asked only of online participants and postal questionnaire respondents receiving only the other postal questionnaire

Four additional weights (one for each of these question types) are calculated for the combined all-country sample. 

However, in waves where the healthy eating module is included, one of the two postal questionnaires contains Northern Ireland-only questions, with the other postal questionnaire only used in England and Wales.  This means:

1. All-questionnaire weights are used for questions asked to all sample members in all online and postal questionnaires 
2. Online questionnaire weights are used for questions asked only to online participants
3. All-countries Defra questions weight is calculated for a selection of questions asked of all participants in England and Wales, but only online participants in Northern Ireland.
4. Combined England & Wales online questionnaire weight is used for questions which are asked only in England and Wales and only in the online questionnaires offered in those countries.

Online questionnaire plus Version 1 postal questionnaire weights and online questionnaire plus Version 2 postal questionnaire weights are not relevant in waves when there is a Northern Ireland specific postal questionnaire. 

Once the main weights are calculated as described above, supplementary ‘Wales & Welsh-England’ weights are calculated. These are designed to allow comparisons to be made between Wales and England (excluding London) after controlling for country profile differences in age within gender, ethnic group, number of adults per household, and urban-rural mix.

Calculation of selection weights

Selection weights are created to compensate for (i) variations in within-household individual selection probabilities and response propensities and (ii) the fact that, by design, some questions are not included in all questionnaires. As a maximum of two eligible adults are surveyed per household, adults in larger households are less likely to be included in the survey. So without this weight, individuals living in households in which some eligible adults are not interviewed would be underrepresented relative to individuals living in households in which all eligible adults are interviewed. They are calculated in the following stages: 

1. The all-questionnaire selection weight is calculated as: (number of eligible people aged 16 years or over in the household)/(number of participants in the household). 
2. The online questionnaire selection weight is calculated as: (number of eligible people aged 16 years or over in the household)/(number of online participants in the household). 
3. Next the Online questionnaire plus version 1 postal questionnaire weight and the online questionnaire plus version 2 postal questionnaire weight are calculated by doubling the value of the all-questionnaire selection weight for postal respondents relative to the corresponding value for online respondents (because the relevant questions are only asked in half the postal questionnaires).    

Values are capped to the range 1-3 for the all-questionnaire and online selection weights, and to the range 1-6 for the online questionnaire plus version 1 postal questionnaire and online questionnaire plus version 2 postal questionnaire weights to restrict variance inflation.

Calibration to population values

Next, selection weights are applied to the three individual country samples and each is calibrated to the corresponding country population values for the number of adults aged 16 or over by: 
(i)    age band within gender 
(ii)    geographic area (defined separately for each country) 
(iii)    deprivation quintile (calculated using each country’s multiple deprivation index)

These weighting variables are often used as standard in social surveys because they correlate reliably with both response propensity and a wide range of survey variables. We note that in some previous rounds of the face-to-face Food and You survey, working status was used as a weighting variable instead of deprivation quintile. In Food and You 2 it was decided not to use this variable for weighting the sample because early waves of fieldwork took place during the Covid-19 pandemic, during which rates of employment were likely to be unstable.  Deprivation quintile was used as a substitute indicator of general economic prosperity. This approach is expected to continue for the immediate future for comparability.

Weighting targets are taken from recent ONS Mid Population Estimates and NISRA Mid  Population Estimates. The estimates used are provided in the accompanying tables.

Initial calibration is carried out separately in each country for each of the four questionnaire type weights described above. For each questionnaire type weight, calibration adjustment factors are calculated by dividing the individual country weights by the selection weights. These adjustment factors are then capped at the 99th percentile value to limit variance inflation and applied to the selection weight to produce final individual country weights.

After calibration and adjustment factor capping, the individual country level weights are scaled to equalise unweighted and weighted sample sizes in each country.

The aim of these within-country calibration procedures is to match the profile of the weighted sample to that of the population aged 16 or over on gender, age band, geographic region, and deprivation quintile.  In practice, there are slight discrepancies between weighted sample totals and population figures as a result of the adjustment factor caps.

Creation of all-country weight

An all-country version of each questionnaire type weight is then constructed by combining the individual country samples and rescaling final individual country weights so that weighted sample country proportions match the respective country population (age 16 years or over) proportions.

The all-countries Defra weight is constructed using the same rescaling process described above but using the all-questionnaire individual country weights for England and Wales and the online questionnaire weight for Northern Ireland.

The combined England & Wales online questionnaire weight is constructed from the online only individual country weights for those countries, as described above. Where the all-countries weights are rescaled to match the respective (age 16+) population proportions for all three countries, this weight is rescaled to match just the respective (age 16+) population proportions for England and Wales.

‘Wales & Welsh-England’ standardisation weight

This weight is designed to calibrate English sample estimates to Welsh population characteristics for comparative purposes. It is calculated from the England sample as follows:

1. London cases are dropped (London being in many ways unique in the UK)
2. The non-London England sample proportions are calibrated to the weighted Wales sample proportions for four variables: number of adults in the household, ethnic group, urban-rural and age by gender. These four variables were selected when the ‘Wales & Welsh-England’ weights were first constructed in Wave 1. Weighted estimates for Wales and non-London England were compared across a range of candidate variables and statistically significant differences were found for urban-rural, ethnic group, household size and age within gender

The final weighting variables are defined as follows.

Table 1: Age within gender (male and female)

Number of adults in household:

• 1 adult
• 2 adults
• 3+ adults
• Question not answered

Ethnic group:

• White 
• Asian
• Black
• Mixed
• Other/not answered

Urban rural:

Urban: OA falls into a built up area with a population of 10,000 or more

Rural: All other OAs

‘Wales & Welsh-England’ weights were calculated only for respondents in England outside London and in Wales (where they were the same as the individual country weight for Wales).