Skip to main content
English Cymraeg
Final report for the efficacy of withdrawals and recalls evaluation

Efficacy of Withdrawals and Recalls: Conclusion

The tables below map the evaluation findings against the original evaluation questions based on evidence from the data collection, various interviews and focus groups.

Table 10: Evaluation findings - Objective 1

Evaluation question Evaluation findings
To what extent has the project delivered its objectives?
  • ESRG members were broadly positive that the planned outcomes had been met, both in the design of the new system and its outputs.
  • ESRG members observed that the guidance is clear, with the roles and responsibilities of participants clearly described. However, ESRG members from industry expressed concerns that smaller FBOs may have fewer resources to implement the new processes, as well as understand the legalities underpinning them.
  • The adjustments to the format and information included on the website and in the point of sale notices were highlighted as a positive step. However, as the regulators do not control where recall notices are placed within store, ensuring consistency is an ongoing challenge.
  • Raising consumer awareness is an iterative process. The delivery of the system redesign’s anticipated consumer awareness campaign was impacted due to other pressures (for example, EU exit and Covid-19).
  • The development of RCA guidance and the e-learning course were viewed positively. However ESRG members and enforcement officers suggested low numbers of FBOs undertaking the e-learning course, and that RCA findings were not always shared consistently.
To what extent has the project met expectations?
  • Overall, ESRG members considered the system redesign to have met expectations.
  • They acknowledged the inadequacy of recall and withdrawal systems prior to the system redesign, including inconsistencies and lack of awareness of roles and responsibilities.
  • The comprehensiveness of the process of building the evidence base was noted by several ESRG members (for example, live case study reviews, qualitative international benchmarking and consumer workshops). This meant that best practice was directly used to create the four planned outcomes.
Has the governance/ management of the process been adequate to ensure that the process was well run and supported?

Overall, ESRG members regarded the governance and management structures as robust and effective as:

  • the programme was a corporate priority for FSA/FSS, so it was assigned significant resource and support;
  • having the four workstreams was beneficial, as delivery was divided into manageable sections and aligned with clear and distinct objectives;
  • decision making was quick but thorough; and
  • there was good representation of all the relevant stakeholders within workstreams, including consumer and industry input.
What went well?
  • ESRG members were positive about the co-design element, including inputs from industry and consumers (for example, during the drafting of point of sale notices).
  • There was a high level of trust between stakeholders, which encouraged open and honest discussions at ESRG meetings.
  • Due to the extensive engagement and co-development with a range of stakeholders, there was no requirement to pilot the outputs of the project.
What could FSA/FSS have done differently?

ESRG members provided the following suggestions on how the process could have been improved:

  • more time to produce the guidance and templates, as these were delivered within tight timeframes;
  • more guidance offered to the industry-led workstream around requirements; and
  • more regular updates, as it felt as though several activities had progressed before an update was provided.
Were the inputs (people, time, money, resources) to process enough to deliver the project’s objectives?
  • As the programme was a corporate priority for FSA/FSS, all ESRG members considered the system redesign to be well-resourced and funded.
  • As smaller FBOs can find implementing recalls processes more challenging than larger FBOs due to resource, there could have been additional engagement with this group during the design process. The impact of EU Exit and Covid-19 were highlighted as limiting factors in the prioritisation of this work and industry’s capacity to implement outputs.

Evaluation findings - Objective 2

Evaluation question Evaluation findings
Regulators awareness and understanding of the package, including perception of industry awareness, understanding and use
  • All local authority enforcement officers were aware of the package, and often referred to the guidance during a recall incident, as this was considered comprehensive and straightforward.
  • Some enforcement officers questioned whether the guidance could be simplified or shortened to encourage implementation in practice.
  • On the whole, FBOs, ESRG members, and enforcement officers agreed that there was a clear understanding of roles and responsibilities.
  • Point of sale templates were helpful for providing consistency (particularly for smaller FBOs and FBOs who had never previously experienced a recall).
  • Enforcement officers suggested that many small FBOs were unaware that guidance was available, and that local authorities were required to signpost them to the FSA/FSS website.
Industry awareness and understanding of the new guidance, including preparedness in the event of a recall
  • The new guidance was regarded by FBOs as comprehensive, however, many were unaware of its existence prior to their own recall experience.
  • Findings from the FBO Tracker Wave 3 endorse this, as only 37% of FBOs were aware of guidance being available.
  • Contrary to many micro FBOs’ expectations, the process was less daunting than expected, due to the responsiveness of the regulators to FBO queries, in addition to support and guidance received from local authorities.
  • Smaller FBOs interviewed suggested that their recall preparation was limited, while larger FBOs were more likely to have some form of internal policy in place in the event of a recall.
  • All agreed that post-recall experience, their internal policies were strengthened and were clear on the actions required
Industry use of the new guidance and template in response to a recall, including any changes in the time taken to issue a recall notice
  • Some ESRG members shared that the new guidance worked for larger FBOs but raised concerns about smaller FBOs having understanding of the processes.
  • Some enforcement officers suggested that the point of sale template was more widely used by smaller FBOs, who had less experience of a recall, and welcomed the structure provided by the template.
  • No feedback on changes to timeliness was provided, however both enforcement officers and FBOs highlighted that the recall was a fast-paced process, suggesting that recall notices were issued in a prompt manner.
Industry use of the RCA, whether it has been successful in finding a cause and whether findings have been shared more widely
  • Enforcement officers suggested that RCAs were being routinely conducted by larger FBOs, but there was still some further work required to ensure that smaller FBOs also took part in this process
  • FBOs considered the completion of RCAs as beneficial for their individual businesses, as it helped to identify the root cause of the incident and enabled them to put specific measures in place to avoid future recall incidents.
  • However, the programme has not as effective in ensuring that the learnings from the RCA are being used to help other businesses avoid the same problems. There is currently no process to share the learnings more widely, nor a process to capture near-miss incidents
  • There appears to be limited awareness of the e-learning course amongst FBOs, with ESRG members reporting limited completion
Has the learning from RCA been used to help other businesses avoid the same problems? How does that process work? How could it work better?
  • Overall, learnings from RCAs do not appear to be shared in a consistent manner, meaning that there are no opportunities for cross-industry learning
  • There was some uncertainty expressed around who was responsible for sharing these RCA findings
  • ESRG members and enforcement officers suggested that the system has been less effective in ensuring industry-wide learning, as there is currently no formal process in place to share the RCA learnings.
Consumer awareness of recalls and actions they should take in response to a recall
  • Perceptions of consumer awareness differed between enforcement officers, FBOs and ESRG members, and consumers themselves.
  • Consumer focus groups suggested that those who had experienced a recall were cognisant of the process. However, the majority of participants had experienced a recent high-profile chocolate recall, during which steps were outlined in the media, which may have increased their knowledge.
  • Those who had not experienced a recall were less aware of the actions they should take, and many suggested they would rather dispose of the product than return it to the store.
  • However, data suggests that where consumers are aware of food recalls, they are increasingly returning food items: in 2021/22, 22% of consumers returned items to the store, compared to only 2% in 2018/19 (Public Attitudes Tracker & Food and You 2). This suggests increasing public awareness of required actions.
To understand how and why the overall package has made a difference (if any)? What was the process by which the package led or contributed to outcomes?
  • On the whole, the consistency of information for consumers has improved, but there are still some areas for future consideration
  • The guidance document sets out clear roles and responsibilities – previously there was no one resource that provided all necessary information
  • Continuous stakeholder engagement (from industry, consumer and local authority perspectives) and a commitment to the system redesign contributed to the attainment of outcomes.

To identify what are the most useful elements of the package and why?
  • The guidance was regarded as comprehensive and well-developed by FBOs and enforcement officers.
  • In particular, the flow charts were considered as accessible and easy to follow.
  • All FBOs praised their local authorities and/or FSA/FSS for being responsive and supportive during the recalls process.

Based on these key findings, these are some considerations for the future for the FSA/FSS:

  1. Process: For any future FSA/FSS project requiring partnership working, consider adopting a similar approach to that used in the system redesign (for example, clearly defined workstreams and engaging regularly with all key stakeholders).
  2. Guidance: Continue to raise awareness of the recalls guidance on the FSA/FSS websites amongst FBOs, as FBOs and enforcement officers suggested that current awareness was limited. Once aware that the guidance was easily accessible, it was well regarded by FBOs. Raising awareness could be done via trade organisations, Linkedin posts or during local authority inspections. Consider also designing separate guidance documents on new and emerging trends, to ensure that the guidance remains current and responsive to new challenges within the industry (for example, in the event of an online recall).
  3. Point of sale notices: Consider making the point of sale notice template mandatory for FBOs to improve consistency of the information provided to consumers. As more consumers shop online, consider producing guidance on where these notices should be displayed online. The point of sales notice template could also include a QR code, as suggested by consumer focus groups.
  4. Consumer awareness: Continue to raise consumer awareness of the steps to take during a food recall (for example, at FSA/FSS stands at food shows or advertisement campaigns), as data suggests that awareness is still lower than expected. Consider further promoting the current FSA/FSS text alert service, as focus group participants were responsive to this idea (as long as the alerts received were tailored to their food consumption habits).
  5. SME support: Consider providing more tailored support for smaller FBOs to raise awareness of their role within the recall and withdrawals process, as SMEs were less likely than larger FBOs to have internal processes or resources in place in the event of a recall. This could include a series of webinars, paid advertisements on social media platforms or additional posts designed for smaller FBOs on the FSA/FSS website. There may also be merit in producing simplified or shortened guidance to encourage smaller FBOs to complete RCAs.
  6. Communicating with consumers: Going forwards, ensure that a combination of communication channels are being used by FBOs to notify consumers during a recall, to reflect consumer preferences and shopping habits. As part of this, the FSA/FSS could create a communicating best practice guide, outlining the various methods that could be used, and local authorities should encourage FBOs to use a combination in-store notices, online notices and social media posts.
  7. Greater sharing of Root Cause Analysis findings: More clarity is required regarding who is responsible (FSA/FSS, local authorities or FBOs) for sharing RCA findings, and for confirming the types of forums these findings could be shared in. This would ensure continuous improvement within the system. Consider also developing a national database of RCAs, accessible by all local authorities, and consider developing a database of ‘near-miss’ incidents. This would be useful in monitoring any current recall trends, as well as noting any emerging trends.
  8. Further promotion of the e-learning course: To increase uptake of the RCA e-learning course, consider requesting local authorities share the RCA e-learning course with FBOs as part of the recalls process. Consider monitoring course completion rates, to explore if uptake increases post-promotion.
  9. Data collection: Consider standardised FSA and FSS data collection categories, so data can be directly compared going forward to monitor recall trends.