Skip to main content
English

Less than thoroughly cooked beef burgers: guidance for food businesses

Less than thoroughly cooked beef burgers guidance for food businesses and local authorities.

Last updated: 22 May 2023
Last updated: 22 May 2023
Summary of the less than thoroughly cooked beef burgers guidance for food businesses and local authorities.

Purpose

This guidance gives advice to food businesses and local authorities on controls and safe systems which can reduce the risks associated with less than thoroughly cooked (LTTC) beef burgers, sometimes referred to as rare, pink or lightly cooked beef burgers. 

This document contains Regulatory Compliance (i.e., how to comply with regulatory requirements) and Best Practice Guidance (for example, helpful examples of approaches you might employ, but which you are not legally required to follow)

Who is this publication for?

This guidance is for:

  • food businesses serving LTTC beef burgers
  • LA officers enforcing food hygiene official controls 
  •  the guidance may also be of interest to manufacturers, processors and distributors of beef, minced beef and beef burgers intended to be less than thoroughly cooked

Which UK countries does this guidance apply to?

  • England
  • Wales
  • Northern Ireland

Review date

We will review this guidance before June 2025

Key words

  1. food law
  2. monitoring and controls 
  3. hygiene and food safety 
  4. less than thoroughly cooked beef burgers
  5. burgers
  6. meat and livestock

Contacts

We welcome your feedback on this guidance. Please contact us by submitting an online enquiry if you are a business. LAs should use the normal procedures to contact the FSA, following the agreed hierarchy of enquiries which can be found on the smarter communications platform.

Downloading this guidance

You can:

  • use the ‘View as PDF’ option at the top of each page to download individual pages of this guidance
  • use the 'View entire guide as PDF’ option placed at the bottom of each page to download the entire guidance as one document

Revision history

  • May 2023 -Change of title and guidance amended to be clearer and easier to understand. Advice to buy minced beef and beef burgers from specifically approved premises added.
  • January 2022- Guidance amended to be easier to understand. Advice to buy minced beef and beef burgers from specifically approved premises added. Change of format from a PDF document to HTML webpages for accessibility.
  • June 2018 - Revision to consumer messaging
  • May 2016 - Original guidance published
An overview of different burger cooking methods, purpose of the guidance, its legal status and the intended audience.

This guidance has been produced by the Food Standards Agency (FSA), in consultation with key stakeholders including food businesses, trade organisations, food safety consultants and local authorities (LAs). 

The FSA first produced a guidance document on less than thoroughly cooked (LTTC) beef burgers in 2016, as it had become apparent that consumer demand for lightly cooked burgers was increasing and that food businesses and enforcement authorities (EAs) needed information to help them understand the necessary controls and systems. An update to the guidance in 2018 was limited to advice about consumer messaging.

When meat is minced and burgers are formed, harmful bacteria which are normally on the outside of a joint of meat can be spread throughout the burgers. If burgers are not thoroughly cooked there is a risk that harmful bacteria may survive and cause food poisoning.

Minced meat may be formed into a burger or into a patty. For the purpose of this guidance a burger consists of minced meat with added ingredients, while a patty consists of minced meat with less than 1% salt added. In this guidance when the term burgers is used it also includes patties. 

When burgers are thoroughly cooked, achieving a temperature of 70°C for two minutes, or equivalent all the way through, this will result in a 99.9999% (six-log) reduction in harmful bacteria which are a risk to human health and may be present in meat. This reduction in bacteria is generally considered to reduce the risk of food poisoning to an acceptable level.

The colour of a beef burger cannot be used as the sole indication of safe cooking. Some thoroughly cooked beef burgers appear to be LTTC as they are pink in the middle, for example when they have been cooked using the sous vide method which achieves a lower temperature but for a longer period of time.

The FSA acknowledges that some consumers wish to eat LTTC beef burgers, which are considered a greater risk of food poisoning than beef burgers normally consumed thoroughly cooked. This guidance aims to balance the risks to consumers with informed consumer choice.

Whilst there is no legal requirement to cook meat for a specific time or to a specific temperature, food businesses are legally obliged to produce safe food. Food businesses can serve LTTC beef burgers where they can demonstrate that they have suitable procedures in place and have controlled all hazards identified, including the microbiological risks, to an acceptable level.

This guidance provides advice on the source control method which can be used to produce LTTC beef burgers. This method can provide an equivalent level of safety to conventional thorough cooking, but strict controls and systems need to be in place. This guidance also includes overviews of two methods – sear and shave, and sous vide – which can be used to produce beef burgers that may appear to be LTTC as they are pink in the middle. The overviews are in Annexes 4 and 5.

For the purposes of this guidance, a LTTC beef burger is a beef burger which has been produced using the source control method.

Food businesses intending to serve LTTC beef burgers must inform their LA beforehand and must be able to demonstrate that suitable controls are in place. This will include a food safety management system (FSMS) which identifies the hazards and controls needed for producing and/or serving beef burgers that will be LTTC.

A consumer message is required at the point of sale to help consumers understand the potential risks of eating LTTC beef burgers so they can make an informed choice when ordering. The consumer message also aims to discourage consumers from eating LTTC beef burgers at home.

Food businesses with a primary authority relationship (England and Wales only) may request the support of the primary authority in meeting their obligations. This does not affect the responsibility that the food business has to comply with the law.

Intended audience

This guidance is intended for food businesses that serve LTTC beef burgers and beef burgers that appear to be LTTC, and the enforcement officers who approve, register and inspect such food businesses.

Guidance is also provided for the businesses which serve LTTC beef burgers that they make themselves on site, either from minced meat they have bought in, or by mincing beef themselves and making beef burgers from it. 

This guidance may also be useful to food businesses that supply beef, minced beef and beef burgers to be LTTC, and the enforcement officers who carry out official controls at these food businesses. 

Purpose of the guidance 

This guidance is intended to help food businesses and enforcement officers to understand the controls and systems which can be used to produce and serve LTTC beef burgers. It also gives an overview of methods that can be used to cook beef burgers which appear to be LTTC. The guidance will help food businesses to achieve consumer safety and legal compliance. 

Burgers made from meat other than beef are not within the scope of this guidance. FSA advice is that burgers made from other meats should be thoroughly cooked.

This guidance document has been updated and published since the UK exited from the EU and the end of the transition period. References to EU legislation have therefore been updated to reflect Retained EU Law (REUL). In Northern Ireland EU law continues to apply in respect to the majority of food and feed hygiene and safety law, as listed in the Northern Ireland Protocol (now referred to as the Windsor Framework), and retained EU law does not apply in these circumstances. [Please note that the UK Government published 'The Windsor Framework: A new way forward' - Command Paper 1245 (PDF) to set out new arrangements which will be applied in place of the original Protocol.]

Retained EU Law is identified in FSA guidance using the following formats: Retained Regulation (EU) No. XXX/XXXX or Regulation (EU) No. XXX/XXXX (REUL).

This guidance has been produced to provide:

  • guidance on the legal requirements relevant to the production and service of LTTC beef burgers and beef burgers which appear to be LTTC
  • links to the legislation that applies can be found in legislation boxes at the end of each page and in Annex 1
  • best practice guidance. You are not required by law to follow best practice guidance.

The guidance on legal requirements cannot cover every situation, food businesses and EAs may need to consider the relevant legislation itself to see how it applies in their circumstances. Food businesses with specific queries may seek advice from the relevant EA, which will usually be the environmental health/trading standards department of their LA.

Following this guidance will help food businesses comply with the law. Food businesses are not required by law to follow best practice guidance. Guidance on best practice is identified with a heading of 'Best practice': 

 
Bacteria associated with less than thoroughly cooked burgers, requirements for managing the risk to consumers and overview of cooking methods.

This section covers the bacteria which are associated with LTTC beef burgers, general requirements for managing the risk to consumers and an overview of methods which can be used to safely produce and serve LTTC beef burgers, or beef burgers which appear to be LTTC.

The main source of bacteria in meat is from the intestines of the animal. When animals are slaughtered, there is potential for harmful bacteria from the intestines and hide to contaminate the surface of meat. There is no way of knowing which animals in the slaughterhouse are carrying harmful bacteria as the bacteria cannot be seen without a microscope.

Certain harmful bacteria, such as Salmonella and Shiga-toxin producing Escherichia coli (STEC), including Escherichia coli (E. coli) O157, are associated with raw beef. These bacteria can cause infection in very low doses which can lead to serious illness and death in some cases.

When meat is minced to produce burgers, harmful bacteria from the surface of the raw meat may be spread throughout the burger. Unless the burger is cooked to a temperature of 70°C for two minutes or equivalent, all the way through, these bacteria may survive on the inside. 

Cooking burgers to a time/temperature combination of 70°C for two minutes, or equivalent, all the way through will result in a six-log reduction in bacteria and this is considered to reduce the risk of food poisoning to an acceptable level. If followed correctly, the methods of cooking and serving LTTC beef burgers, or burgers that appear to be LTTC, which are covered in this guidance should give similar levels of reductions in bacteria.

Reductions in bacteria are often expressed as log reductions to avoid using the massive numbers which are associated with micro-organisms. The table below shows how log reductions of bacteria can be expressed as percentages. For more information about log reductions please see Annex 2.

Log reductions of bacteria expressed as percentages

Reduction Percentage of bacteria eliminated
One-log 90%
Two-log 99%
Three-log 99.9%
Four-log 99.99%
Five-log 99.999%
Six-log 99.9999%

Requirement to inform LA of intention to serve LTTC beef burgers

If a food business plans to serve LTTC beef burgers, or burgers which appear to be LTTC, they are legally required to inform their LA because it constitutes a significant change to their business operation. New businesses should indicate on the registration form their intention to produce/serve LTTC beef burgers or beef burgers which appear to be LTTC.

Requirement for a Food Safety Management System 

Food businesses serving LTTC beef burgers, or beef burgers which appear to be LTTC, must produce and implement an appropriate FSMS which takes into account that burgers will be LTTC, or appear to be LTTC. The FSMS must be based on Hazard Analysis and Critical Control Points (HACCP) and the FSA website provides further information. 

This guidance summarises the requirement; it is not exhaustive, all hazards must be considered. 

HACCP

The HACCP approach provides a systematic way to identify food safety hazards and make sure that they are being controlled every day. 
It is a legal requirement to follow the seven principles of HACCP:

  • identify any hazards that must be prevented eliminated or reduced
  • identify the critical control points (CCPs) at the steps at which control is essential
  • establish critical limits at CCPs
  • establish procedures to monitor the CCPs
  • establish corrective actions to be taken if a CCP is not under control
  • establish procedures to verify whether the above procedures are working effectively
  • establish documents and records to demonstrate the effective application of the above measures

Pre-requisites 

Pre-requisites are effective policies and procedures that are essential for food safety and must be in force before and during the implementation of HACCP. Many pre-requisites are legal requirements. Once the pre-requisites are in place, HACCP can then be used to control steps in the food business which are critical in ensuring the preparation of safe food.

It is important that these pre-requisites are in place as without them, the HACCP based procedures for controlling hazards throughout food production will not be effective. Although not exhaustive, some examples which are particularly relevant to beef burgers which are LTTC or appear to be LTTC are:

  • staff training
  • cleaning and disinfection
  • temperature control, which could include achieving/maintaining the cold chain and cooking/hot holding temperatures 
  • suitable resources and facilities     
  • prevention of cross contamination 
  • personal hygiene of staff

As part of the food safety management system, consideration should be given as to whether any of the pre-requisites would also be critical control points.

The requirement to inform the LA of plans to serve LTTC beef burgers or beef burgers which appear to be LTTC can be found in:

The requirement for a HACCP-based food safety management system is included in:

Overview of methods used to cook beef burgers

Conventional thorough cooking - burgers are thoroughly cooked all the way through to a core temperature of 70°C for two minutes, or equivalent. This will generally result in a 99.9999% (six-log) reduction in harmful bacteria which may be present. This reduction in bacteria is generally considered to reduce the risk of food poisoning to an acceptable level. Examples of time/temperature combinations which are equivalent to 70°C for at least 2 minutes for cooking include:

  • 80°C for at least 6 seconds
  • 75°C for at least 30 seconds
  • 65°C for at least 10 minutes
  • 60°C for at least 45 minutes

Sous-vide cooking- burgers are vacuum packed and cooked in a water bath for a longer period and at a lower temperature than conventional cooking. A validated time/ temperature combination equivalent to 70°C for two minutes must be achieved. This will result in the burgers remaining pink in the middle while achieving a six-log reduction in bacteria. Further information can be found in Annex 4 of this guidance.

Sear and shave - the outer surfaces of a whole muscle piece of beef are to be sufficiently cooked to achieve at least a six-log reduction in bacteria, as determined by the food safety management system. The outer surfaces are then shaved off and the remaining meat is minced and used to make burgers which are lightly cooked. This method of preparation will achieve a six-log reduction in bacteria while the burgers remain pink in the middle. Strict controls and additional information can be found in Annex 5 of this guidance. 

Source control - beef, minced beef and beef burgers are produced with strict controls in place, which research has shown can reduce bacteria by two-logs. The beef burgers are then lightly cooked to achieve at least a four-log reduction in bacteria. The infographic below (Figure 1) gives an overview of the steps to be taken at each stage of the food chain when using this method. You can also download the Source control method infographic as a PDF.

Figure 1. Source control method - steps at each stage of the food chain

 A flow chart showing the control measures to be taken at each stage of the food chain when using this source control method.
This section explains how the source control method can be used to produce and serve LTTC beef burgers.

​Overview of the method

This method starts with beef/minced beef/beef burgers being bought from a supplier that has control methods in place to reduce bacteria and/or control the growth of harmful bacteria. Examples of these control methods include lactic acid rinsing of meat cuts, steam surface treatment and the separation of storage materials and equipment to reduce the risk of cross contamination. The food business must carry out microbiological testing to verify that controls are working. Research has shown that appropriate source controls can result in at least a two-log reduction in harmful bacteria.

The beef burgers are then cooked to a validated time/temperature combination as determined by the food safety management system, which will result in a four-log (99.99%) reduction in bacteria. Overall, this method can achieve a six-log reduction in bacteria. 

A consumer message is used to alert consumers, particularly vulnerable groups, to the increased risk of eating LTTC beef burgers.

The information below provides a quick checklist for food businesses preparing and/or serving LTTC beef burgers using the source control method. More detail on each step is given further on in the guidance.

Best practice

Food businesses may wish to appoint an expert food safety consultant as this is a complex process and must be carefully controlled.

Step 1: Purchase

  • if the supplier requires approval, then they must be specifically approved for the production of minced meat or beef burgers to be LTTC
  • specific controls in place at suppliers (for further information see Annex 3)
  • supplier’s FSMS specifically takes into account that product will be LTTC
  • product specification
  • microbiological testing has been carried out as per the requirements for minced meat and meat preparations

Step 2: Delivery to caterer

  • food safety intake checks 
  • temperature 
  • cross-contamination
  • product must comply with specification
  • product to have use by date and to be within use by date
  • malformed/misshapen beef burgers

Step 3: Storage

  • temperature                     
  • burgers becoming misshapen
  • cross-contamination
  • shelf-life

Step 4: Preparation of beef burgers onsite by the catering business

  • temperature/time combination
  • cross-contamination
  • shelf-life

Step 5: Cooking

  • validated system for reducing bacteria by at least four-logs

Step 6: Service

  • consumer message

Purchase

Best practice 

It is best practice to buy minced beef, beef burgers or patties from a reputable supplier that has been specifically approved by the EA to supply products that will be LTTC. Generally, food businesses which supply minced meat or burgers, such as cutting plants and meat processors, must be approved by either the FSA or by a LA. However, some food businesses are exempt from approval. If a food business does require approval, then they must also have specific approval for the activity of supplying minced beef or beef burgers to be LTTC. The list of approved less than thoroughly cooked meat establishments in the UK will assist food businesses with finding a suitable supplier. 

Hygiene standards

There are specific hygiene standards that apply to suppliers of minced beef or beef burgers to be LTTC and these are summarised in Annex 3.

Suppliers of minced beef or beef burgers which will be LTTC must have appropriate procedures in place which reduce the risks associated with raw beef. Suppliers must specifically identify relevant pathogens, such as salmonella and STEC, as hazards in their FSMS. There must be evidence that suppliers have identified and put in place controls for these hazards and that they monitor and verify that the controls are effective on an ongoing basis. Annex 3 provides further information about an appropriate HACCP-based FSMS.

Generally, food businesses handling products of animal origin must be approved by the relevant EA. However, some food businesses may be exempt from approval. If the food business serving the LTTC beef burgers uses a supplier which is not approved, they must be able to demonstrate that the supplier meets the stringent hygiene standards that would be required of a LTTC approved establishment.

Annex 3 provides further information about these hygiene standards and an appropriate HACCP-based FSMS.

Best practice 

It is best practice that suppliers are audited to ensure that appropriate controls are in place. 

Product specification

Food businesses serving LTTC beef burgers must be able to demonstrate that appropriate product specifications are in place. Examples include:       

  • making beef burgers a consistent shape, size and thickness, so that safety controls such as cooking times are easy to apply
  • using packaging which does not squash or misshape the beef burgers
  • clear labelling to distinguish them from burgers that require thorough cooking
  • using strict temperature controls and ensuring the cold chain is maintained to limit the growth of harmful bacteria
  • avoiding cross-contamination at all stages, for example ensuring outer packaging is not a source of contamination
  • shelf-life and storage instructions take into account the temperature that the beef burgers will be stored at in the food business serving the LTTC beef burgers

Validation and verification of the FSMS at suppliers

As part of the process for validation and verification, microbiological sampling and testing regimes are needed to validate and verify controls as part of the FSMS. The regime must include specific corrective action that will be taken in the event of unsatisfactory results. Although microbiological sampling and testing is not a guarantee of the safety of a product, it is an important means of verifying that the FSMS is effective. 

Microbiological testing requirements for food businesses that mince meat

Microbiological testing is required for minced meat/beef burgers that will be LTTC throughout each stage of production and including when the beef is minced, in order to validate the implementation of the FSMS.

The requirement for a HACCP-based FSMS is included in:

The requirement for food business operators throughout each stage of production to carry out testing for minced meat/beef burgers that will be LTTC is included in:

Retained Regulation (EC) No 2073/2005 and Regulation (EC) 2073/2005 provides specific microbiological criteria for salmonella levels in minced meat and meat preparations to be consumed raw.

These criteria also apply to minced meat and meat preparations which are to be LTTC, including minced meat with seasoning or additives, formed into beef burgers.

The food business serving LTTC beef burgers must be able to demonstrate that this verification sampling and testing has been carried out and that the results comply with the regulations. 

Any food business that produces minced meat or meat preparations to be LTTC using the source control method must always carry out verification sampling and testing, regardless of volume. The exemption in the regulations for small amounts of minced meat and meat preparations does not apply and sampling must be undertaken in all cases.

The FSMS must also consider controls for hazards other than salmonella, including STEC, which may include additional verification sampling and testing alongside the criteria in the regulations. There must be procedures in place for the appropriate corrective action to be taken if the results are unsatisfactory.

If sampling and testing results show that STEC are confirmed as present in a batch of minced meat or meat preparations, that batch of meat must not be used for burgers that will be LTTC due to the risk to public health. This should be considered as part of the FSMS.

More information about sampling for STEC and what to do in the event of presumptive or unsatisfactory results can be found on the FSA website. 

Delivery to caterer

  • if the product is delivered above the maximum temperature limit that has been set by the supplier and/or detailed in the FSMS, the delivery must be rejected  
  • it is a legal requirement for approved producers of minced meat to chill to an internal temperature of not more than 2 °C, for meat preparations this would be 4 °C, and to maintain that temperature during storage and transport
  • if third party distributors are used to transport the minced meat/burgers, the cold chain must still be maintained
  • the delivery must be placed into chilled storage as soon as possible, so the cold chain is not interrupted
  • the ‘use by’ date must be checked on arrival and if it has expired, the food must be rejected 
  • during transport, meat, or burgers to be less than thoroughly cooked should be separated from other raw meat and ready-to-eat products, to reduce the risk of cross contamination
  • if the delivery does not meet the product specification, such as size and shape, it must be rejected 
  • delivery intake procedures must be verified on a routine basis; monitoring records and any corrective actions taken should be checked by management

Best practice

It is best practice for catering businesses to carry out microbiological sampling and testing of uncooked minced beef or beef burgers they have received from the supplier. This is to verify they are meeting the required microbiological criteria.

Storage

  • fridges must be large enough and of a suitable grade to hold the required volume of product at the correct temperature
  • burgers must be kept at the temperature specified by the supplier, or the food business must demonstrate that they have appropriately assessed the hazards and provide evidence that the storage arrangements do not pose a risk to food safety
  • temperatures must be monitored using calibrated equipment and records kept
  • products must be used within the supplier’s specified shelf life
  • minced beef or beef burgers bought in fresh must not be frozen before use, unless instructions are provided by the supplier
  • products to be LTTC must be stored separately from other foods, including other raw meat products and ready-to-eat foods
  • if the shape and size of the burgers, and/or the way they are stacked or packaged, has been specified, then storage at the catering establishment must not affect this
  • management must make routine verification checks of storage practices to ensure that temperatures are being monitored and recorded correctly and remain within the set limits, staff are using the designated storage areas correctly and any corrective action required is being taken and recorded
  • if any deviation from the documented storage procedures is observed, the root cause of this should be determined and addressed to ensure the same issues do not recur

Best practice 

It is best practice to store products to be LTTC in a separate, designated fridge.

Production of beef burgers on site by the food business which serves LTTC beef burgers

  • controls must be specified in the FSMS, for example, the time taken to produce the beef burgers and the time the beef burgers are out of the fridge, or the temperature of beef burgers at the end of processing may need to be checked and recorded
  • beef burgers to be LTTC should not be prepared in the same area at the same time as any other raw meats, and the work surface must be thoroughly cleaned and disinfected between uses, detailed guidance on cleaning and disinfection can be found in the E. coli cross-contamination guidance
  • complex equipment such as mincers must be designated for products to be LTTC only and other pieces of equipment, such as knives and chopping boards, must also be designated, unless they can be completely dismantled, thoroughly cleaned and heat disinfected
  • designated utensils and equipment must be stored in a location where they are not at risk of being contaminated
  • the beef burgers must be labelled with a suitable shelf life that is supported by verification and it is the responsibility of the food business to determine the shelf life of the burgers in line with their FSMS, this date should be determined by a HACCP validation study
  • to reduce the risk of contamination and reduce the hazards to a safe level, the preparation methods must be validated before they are introduced to make sure they are going to work. It may be helpful to consider the following questions:

Is the preparation space adequate?
 Is the right equipment available?
Will safe temperatures be maintained?

Best practice

It is best practice to:

  • prepare beef burgers during times when the kitchen is quiet, as it is likely to be cooler and staff are less likely to be distracted by other tasks 
  • use separate, designated equipment and surfaces for the preparation of beef burgers/patties to be LTTC
  • remove meat from the fridge only when staff are ready to mince the meat and/or form the beef burgers. Staff should work on small batches at a time and place finished batches in the fridge before starting on another 
  • handle the meat as little as possible, to prevent it from becoming warm. Any equipment used should not be hot, for example after being washed in a dishwasher 
  • use a chilled room or a cool area of the kitchen as this will help keep meat cold and if any ingredients are added to the beef burger they should be as cold as possible before being used, e.g., onions

Cooking

The proposed time/temperature combination must be validated to show that a minimum of a four-log reduction in bacteria will be achieved. Further validation is needed to show that the proposed cooking method will consistently achieve the validated time/temperature combination. Validation must be carried out using the worst case conditions as several factors will affect the cooking such as:

  • the thickness of the burger
  • the percentage of fat in the burger
  • the initial burger temperature (fridge or room temperature)
  • cold spots on the hot plate/grill, the use of surface temperature probes on hot plates can assist with finding cold spots 
  • type of grill/hotplate

Validation and verification that the time/temperature combination achieves four-log reduction
When validating and verifying the proposed time/temperature combination to ensure that it will result in a four-log reduction in bacteria, evidence may be gathered as below. The list is not exhaustive and there may be other ways to provide evidence.

Challenge testing

Challenge testing can be used to demonstrate that the cooking method will result in a four-log reduction of bacteria throughout the beef burger. Challenge testing is the deliberate addition of specific microorganisms to monitor their growth and/or survival in a product. The challenge testing must take into account that the bacteria associated with raw beef are Salmonella and Shiga-toxin producing Escherichia coli (STEC), including Escherichia coli (E. coli) O157.The laboratory will be able to advise on how testing can be done safely. Challenge testing can be done using actual pathogens in certain accredited laboratories using actual processing equipment to be used by the premise. Alternatively, challenge testing can be done in a food processing area using a qualified non-pathogenic surrogate organism with similar thermal resistance properties to the pathogen under consideration. Genuine human pathogens should not be used for microbiological testing in food processing environments. Microbiological testing must be carried out by a laboratory accredited to ISO 17025 and the methodology used by that laboratory should also be accredited. A list of accredited laboratories can be found on the United Kingdom Accreditation Service’s (UKAS) website. Laboratories will be able to advise on tests in further detail.

Scientific data

The food business may consider scientific and/or technical data that has been produced by the food industry to support their validation and challenge testing. For example, where food businesses have a number of identical establishments, equipment and products, it may be that a validation for one establishment is suitable for other establishments.

It may also be possible to use wider industry validation data, if available, where products and cooking methods are similar. In this situation, it would be important to consider factors such as:

  • time and temperature
  • equipment 
  • method of handling
  • ingredients
  • fat content
  • uniformity of composition
  • size and shape of burger
  • the heating method

Modelling

Potentially, mathematical modelling could be used to support validation and challenge testing. Models used for this purpose must be validated and verified as appropriate for LTTC beef burgers. There must be sufficient data to predict the effects of different treatments. Factors such as, but not limited to, size and composition must be considered.

Note: If changes are introduced to the method or product specification, for example a new piece of equipment is provided, or a new supplier or product specification is used, then the process must be re-validated.

Verification sampling

A FSMS that is effectively implemented will ensure that food safety hazards are under control. An important part of any FSMS is verification - activities that ensure the system is working. Microbiological sampling and testing after cooking can be used to verify that the FSMS in place is working as intended. When planning a sampling and testing regime, the following matters may need to be considered:

  • quantity of LTTC beef burgers served
  • controls in place
  • purpose of the sampling regime
  • pathogens being tested for
  • against what parameters the final test results will be compared  
  • sample size to be tested each time
  • frequency of sampling and testing

It is important to recognise the limitations of sampling and testing. For example, a result that reports the absence of STEC does not necessarily verify that all the burgers are free from STEC. If STEC was not present in the burger that was sampled before cooking, it will not be present after cooking, regardless of the controls in place.

Validation and verification of the cooking method

Following the validation of the temperature/time combination to achieve a four-log reduction, the cooking method must be validated before it is introduced to show that the proposed time/temperature will be achieved consistently. Once the cooking method has been validated and it is introduced, food businesses must verify that the time/temperature combination is being met consistently.

Temperature checks to validate and verify the FSMS

  • the temperature at the centre of the burger will need to be tested, as this is likely to be the last part of the burger to reach the required temperature
  • the type of probe thermometer used must be suitable and instructions must be followed, it must be calibrated according to instructions to ensure it gives an accurate reading 
  • the probe thermometer must be cleaned and disinfected before and after each use to reduce the risk of cross contamination
  • temperature monitoring records should be kept in accordance with the FSMS
  • monitoring records should be verified by management, in accordance with the FSMS 
  • the method used to check burger temperatures should be verified on a regular basis by observation, in accordance with the FSMS
  • staff must be able to be identified and retrained if management identify any issues with records
  • verification checks should be recorded along with any corrective actions taken, in accordance with the FSMS

Cooking time check to validate and verify the FSMS

The validated cooking method may entail burgers being cooked at a certain temperature for a certain amount of time to achieve the time/temperature combination throughout the burger. Several factors need to be considered when using time checks including:

  • the type and temperature of equipment used - different grills and flat tops operate in different ways and some will have cold spots that will affect cooking time 
  • the number of burgers cooked together may affect cooking time
  • heat will distribute at different rates through burgers with different thicknesses, fat content and ingredients
  • the temperature of the burger before cooking - a burger removed from the fridge at 1°C will take longer to heat to core temperature than a burger at 5°C, the ambient temperature of the kitchen may also affect cooking time
  • if timers are used to monitor the time taken to cook burgers, the number of timers needed will depend on the number of burgers to be cooked at the same time and whether every burger is timed, a maximum cook load may need to be set

Validation of cooking time

If the cooking time is used to assess whether the beef burgers are cooked to an appropriate time/temperature combination, this system must be validated before it is introduced. This would involve checking that the correct time/temperature of the burger has been achieved after it has been cooked for the set time.

If anything changes which may affect the cooking time, such as the cooking equipment or the supplier of the burgers, the method must be validated again.

Verification of cooking time

It is important that the time taken to reach the required time/temperature is verified regularly. To verify that an adequate time/temperature has been achieved, a burger must be cooked for the set time. Then the core temperature must be checked and monitored for the required length of time, as per the time/temperature combination. If cold spots on the cooking plate/grill have been identified, the verification must be carried out in the known cold spot.

Consideration must also be given as to how the above process will be recorded.

Consumer message

A consumer message should be provided to alert consumers, particularly vulnerable groups, to the increased risk of eating LTTC beef burgers.

Food businesses have a legal obligation to provide information to the consumer concerning the avoidance of specific adverse health effects from a particular food or category of foods. Although it is not a legal requirement to provide this information for LTTC beef burgers by way of a consumer message, it is best practice so that the consumer can make an informed choice. 

This legal requirement can be found in:

A relevant extract from the legislation can be found below:

  • 'In determining whether any food is unsafe, regard shall be had:
  •  … to the information provided to the consumer, including information on the label, or other information generally available to the consumer concerning the avoidance of specific adverse health effects from a particular food or category of foods

And

  • … In determining whether any food is injurious to health, regard shall be had… to the particular health sensitivities of a specific category of consumers where the food is intended for that category of consumers’

When a food carries a level of risk that is deemed as elevated but still within acceptable levels of risk, and that risk is not communicated to consumers to allow them to consider that before they order it, this could be considered as not fully compliant with general food law.

Consumer messaging research 

The FSA conducted consumer focused research on the effectiveness of consumer advisory messages at the point of ordering, which was published in July 2016. The findings of the research were taken into consideration, along with comments from LAs and industry stakeholders, to develop the wording for consumer messages. Messages should be clear, meaningful and easily understood. The best practice box below contains the recommended consumer message to be used when LTTC beef burgers have been produced and served using the source control method. 

Best practice 

The FSA recommends using the following consumer message:
'Burgers cooked rare and medium rare carry a higher risk of food poisoning. Unlike a steak, a burger needs to be cooked through to reduce that risk.

The Food Standards Agency recommends that children, pregnant women and anyone with a weaker immune system have their burgers well done. Please ask us for more information.'

As children are classed as a vulnerable group, it would not be appropriate for LTTC beef burgers to be included on children’s menus. Food businesses should also consider if unaccompanied children can make an informed decision.

It is best practice that the consumer advisory message is prominent, clearly legible and considers the needs of those with impairments. The font size used for the consumer message should be legible to prompt discussion between staff and consumer if necessary. It is recommended the message is in the same sections of the menu in which burgers are listed and not obscured in any way. Where food is ordered remotely, such as online sales, the consumer message should be available and easily visible. 
 

 

Links to legislation and guidance related to less than thoroughly cooked burgers.

Legislation

GB-wide legislation

Retained EU legislation

Domestic legislation

England

Domestic legislation

Wales

Domestic legislation

Northern Ireland

EU legislation

Northern Ireland

Domestic legislation

Business guidance

England and Wales

Northern Ireland

An overview of logarithmic reduction in bacterial count.

For the purpose of this guidance, logarithmic (log) reduction is a way of identifying the decrease in the number of bacteria (bacterial count) following a method of processing- for example, cooking.

A one-log reduction means that the number of bacteria (bacterial count) has been reduced by 90%. If the meat originally has 1,000 bacteria before cooking, after a one-log-reduction cooking method, the meat will still have 100 bacteria left. A two-log-reduction cooking method would decrease bacterial count by 99%, that is, 10 bacteria left. In contrast, a six-log-reduction cooking method would decrease the bacterial count by 99.9999%, that is, the meat would have between one and zero bacteria.

Log reduction is a mathematical concept. Therefore, the bacterial count will never be described as an ‘absolute zero,’ but as a decimal instead. The smaller the decimal, the less likely it will be for the meat to have any bacteria left after cooking.

If the bacterial count of the meat used is very high (for example, one million), a four-log reduction would decrease the bacterial count to 100, whereas a six-log reduction would decrease the bacterial count to one; not all bacteria would be eliminated. This illustrates the importance of all controls through the food chain, including at supplier level as detailed in Annex 3.

‘Figure 2’ illustrates the effect of log reduction in bacterial count as explained in previous paragraphs.

Figure 2. Effect of log reduction in bacterial count

Graph showing the effect of log reduction on the bacterial count as specified on this page.
Suppliers of beef/minced beef/beef burgers to be LTTC must have hygienic procedures in place during slaughter, cutting, mincing and any other relevant process.

Suppliers of beef/minced beef/beef burgers to be LTTC must have hygienic procedures in place during slaughter, cutting, mincing and any other relevant process. Their procedures must focus on minimising contamination with pathogens.

Information about reducing potential contamination during slaughter, cutting and manufacturing of minced meat and beef burgers or patties is available in the Manual for Official Controls (MOC) (England and Wales) /Manual for Official Controls (VPHP) | Department of Agriculture, Environment and Rural Affairs (NI) and the FSA Clean Livestock Guide. The FSA has also published research to assess the significance of intervention methods to reduce the microbiological load on beef through primary production.

The key aspects of the MOC and the FSA Clean Livestock Guide are highlighted in this section, further details can be found in the original documents.

Clean beef cattle for slaughter

It is important that cattle presented for slaughter are clean and dry. This is because faeces and mud on the animal’s hide can contaminate meat or the environment of a slaughterhouse when the hide is removed. Further information can be found in the Clean beef cattle for slaughter guidance.

Hygiene during evisceration and skinning/hide removal

Most of the harmful bacteria in an animal can be found in the intestines and on the hide. The intestines must be removed carefully to avoid the contents being released and contaminating the carcass during the evisceration process. Similarly, the hide of the animal must be removed carefully to avoid contamination.

Steam vacuum can be used to remove minor visible contamination, dirt and hair from the surface of the carcass prior to post-mortem inspection. This must only be used to rectify accidental contamination of carcases and not as a substitute for good hygiene or inadequate dressing practices.

Surface treatments of the carcass

Suppliers may wish to apply treatments to the surface of the meat that reduce levels of contamination, prior to supplying it to catering establishments. Care must be taken to prevent re contamination after treatment, such as hygienic handling and dedicated mincing equipment.

Any surface treatment used must be legally allowed and the requirements in the relevant legislation must be followed. At present, lactic acid and potable water are the only substances that can be used to reduce the contamination on the surface of beef carcases in approved slaughterhouses.

Further information on the use of lactic acid to reduce microbiological surface contamination on bovine carcases can be found in the Manual for Official Controls. (England and Wales) and Manual for Official Controls (VPHP) | Department of Agriculture, Environment and Rural Affairs (NI)

Time limits on meat to be minced

Meat must be minced within a specific time period from the time of slaughter to minimise the potential for the growth of pathogens that might be present on the meat. In the case of beef minced meat produced from chilled meat, this is:

  • within no more than six days of slaughter, or
  • within no more than 15 days from the date of slaughter of the animals in the case of boned, vacuum-packed beef

Temperature control

Where possible, mincing should be carried out under temperature control to keep the meat and resulting minced meat at a temperature as low as is practical. As the presence of low numbers of harmful E. coli can cause illness, limiting growth of the bacteria by using low temperatures as a control throughout the mincing process and other processing, handling and storage, is an important safety control. This is because if the temperature rises, the bacteria will grow faster and, if it is present in higher numbers, there is a greater likelihood of the bacteria causing illness.
Strict temperature control during any handling, storage and transport will also limit the potential for growth of harmful bacteria.

Temperature control should be reflected in HACCP-based procedures throughout the supply chain to control the hazards of concern and these should be maintained throughout distribution and storage.

Separation to prevent cross-contamination

The separation of food and equipment, where appropriate, will help to prevent cross-contamination. This includes separation of meat destined to be LTTC from meat which is not, including during preparation, storage, packing and transport. The separation of equipment for use with meat destined to be LTTC and meat that is not will also help to prevent cross-contamination. Strict cleaning and disinfection procedures of equipment, tools and premises must be in place to prevent the build-up of bacteria and avoid cross-contamination. Meat packaged and for transportation to another premises should be stored in a clean, separate area. It must be ensured that outer packaging does not come into direct contact with dirty floors etc. prior to and during storage/transport to another premises. This is particularly important as it will help to prevent bacteria from dirty outer packaging contaminating the meat when opened.

Personal hygiene for food handlers

Slaughterhouses and meat processing establishments must ensure food handlers follow good hygiene practices such as regular, effective handwashing and wearing appropriate clean protective clothing. This is to minimise the risk of food handlers spreading bacteria.

Documentation

Suppliers must produce and implement a written HACCP-based FSMS which includes information about how the risks associated with LTTC beef burgers are controlled. Suppliers must specifically identify Salmonella and Shiga toxin-producing E. coli (STEC), and any other relevant pathogens, as particular hazards.

The requirement to mince meat within specific times is included in:

The requirement to produce and implement a written HACCP-based food safety management system can be found in:

This section gives an overview of the sous vide cooking method and how it can be used to cook burgers which appear to be LTTC.

Sous vide consists of placing food in a vacuum sealed bag, then cooking it in a water bath. The food is cooked for a longer time and at a lower temperature than conventionally cooked food. Burgers cooked this way may look pink in the middle, despite being cooked throughout to a time/temperature combination equivalent of 70°C for two minutes. Meats are often quickly grilled or fried after the sous vide process to caramelise and/or give the browned appearance expected by the consumer.

Best practice

Food businesses may wish to appoint an expert food safety consultant as sous vide is a complex process and must be carefully controlled to ensure it is safe.

The sous vide system must be validated before it is introduced to check it will work as intended. Checks must be carried out to ensure the burgers consistently achieve a time/ temperature combination of 70°C for two minutes or equivalent.

If burgers are prepared at the catering business, and are not cooked and consumed straight away, an appropriate shelf life is required. It is the responsibility of the food business to determine the shelf life of the burgers in line with their food safety management system. This date should be determined by a HACCP validation study

Best practice

It is recommended the burgers are used within the shortest time possible to minimise the growth of any harmful bacteria. 

Handling and storage of the burgers after cooking must be hygienic. Food businesses are advised to refer to the guidance on vacuum packed foods and the E. coli cross-contamination guidance.

Best practice

It is best practice to provide a consumer message to explain that the food business has used specific cooking methods to produce beef burgers which appear to be LTTC. This is to help consumers understand that cooking pink burgers at home is not recommended.

This section gives an overview of the sear and shave method and how it can be used as part of the process to prepare and cook burgers which appear to be less than thoroughly cooked. 

This method is based on the same principle as cooking whole pieces of steak - searing the outside of the meat to kill bacteria. Whole muscle cuts of meat must be used because any harmful bacteria will be on the outer surfaces of the meat and will be reduced to safe levels. Meat that has been minced, rolled, tenderised, or reconstituted in any way must not be used. 

The method has two steps:

  • step one is searing - cooking the outside surface of the meat by briefly heating it to a high temperature to destroy surface bacteria, while the deeper tissues remain raw
  • step two is shaving the piece of meat to remove the seared surface, and the remaining meat can be minced and formed into burgers which can be lightly cooked

The searing stage can be achieved by frying the piece of meat but blanching, boiling, deep frying or other types of cooking can also be used to heat the outside of the meat to a high temperature.

It is important that the surface of the cut of meat is not pierced, for example by using utensils to tenderise the meat. This could cause contamination to be pushed into the cut of meat, which could make the sear stage ineffective.

The combination of time and temperature applied at the searing stage must be sufficient to thoroughly cook the surface of the meat as this will reduce harmful bacteria to safe levels in accordance with the FSMS. The surface of the meat must be smooth to ensure searing is even.

Once the outside of the meat has been seared, the outer surfaces are sliced or shaved off, this must be done hygienically to prevent any potential for cross contamination. The meat can then be minced, formed into burgers and lightly cooked. If the burgers are not cooked and consumed straight away, an appropriate shelf life must be given.

It is the responsibility of the food business to determine the shelf life of the burgers in line with their food safety management system. This date should be determined by a HACCP validation study

Best practice
It is recommended the burgers are used within the shortest time possible to minimise the growth of any harmful bacteria.

Potential for cross-contamination must be prevented at all stages of the process. Mincers are considered to be complex equipment so a mincer which is used for raw meat to be LTTC must not also be used for other meats unless it can be fully dismantled and disinfected between uses.

This is because contamination can occur throughout the internal components of the equipment that cannot be adequately disinfected without a full dismantle. The burgers must be stored separately from other foods, including other raw meat products and ready-to-eat foods. Detailed guidance on cross contamination can be found in the E. coli cross-contamination guidance.

It must be remembered that any ingredients added to the minced meat to make the burger, such as raw onions or spices, will need to be free from microbiological contamination (ready-to-eat) as they will not be thoroughly cooked.

Best practice
It is best practice to use separate, designated equipment for meat which has been seared to reduce the risk of cross contamination.

It is possible to sear the outside of a whole cut of meat, and then mince the whole piece of meat without the 'shave' step to remove the cooked outer layer of meat. As with 'sear and shave', food businesses must ensure that the handling, preparation and storage of the meat after the searing process is safe and hygienic.

If the process is carried out hygienically a six-log reduction in bacteria can be achieved.

Best practice

It is best practice to provide a consumer message to explain that the food business has used specific controls to produce burgers that appear to be less than thoroughly cooked. This is to help consumers understand that cooking pink burgers at home is not recommended.

Glossary of terms used in the guidance.

A

Approvable activity

A POAO-handling activity undertaken at an establishment which does not fall within any of the exemption criteria as laid out in the regulations. Approvable activities are applicable to products of animal origin (POAO) for which requirements are laid out in Annex III of Retained Regulation (EC) No. 853/2004 and Regulation (EC) No. 853/2004.

Approved establishment

An establishment that has been approved pursuant to Article 4 of Retained Regulation (EC) No 853/2044 and Regulation (EC) No 853/2004 for handling, preparing, and/or producing products of animal origin.

B

Bacterial count    

Number of bacterial cells.

Burger

For the purpose of this guidance, a meat preparation consisting of minced beef and additional ingredients. In this guidance the term burger also covers patty, which is minced beef which has been formed into a burger shape to which less than 1% salt has been added.

C

Carcass

The body of an animal after slaughter and dressing.

Caterer or catering business

A food business directly preparing, cooking and supplying food to the final consumer such as restaurants, burger outlets and pubs.

Challenge testing

The deliberate addition of specific microorganisms to monitor their growth and/or survival in a product.

Competent Authority (CA)

Has the meaning as defined in Article 3(3) of Retained Regulation (EU) 2017/625 and Regulation (EU) 2017/625 to mean the Competent Authority responsible for the performance of official controls and of other official activities, in accordance with that Regulation and the rules referred to in Article 1(2)

Critical Control Point (CCP)

A step at which control can be applied and is essential to prevent or eliminate a food safety hazard or reduce it to an acceptable level.

Critical limit

A criterion which separates acceptability from unacceptability. Critical limits apply at Critical Control Points and could include the maximum or minimum temperatures required to eliminate a hazard and the time required to achieve this or could refer to the absence of contamination or damage. 

E

Enforcement Authority

Has the meaning as defined by Regulation 2(1) The Food Safety and Hygiene (England) Regulations 2013, Food Hygiene (Wales) Regulations 2006 and Food Hygiene regulations (NI) 2006 to mean the authority which is responsible for executing and enforcing the Hygiene regulations, for example a local authority or the Food Standards Agency.

Escherichia coli (E. coli)

A type of bacteria common in human and animal intestines. Many forms of E. coli are harmless, however a few types can cause serious illness. Some types of E. coli may also be toxin-producing. Avoiding E. coli can be achieved by chilling and cooking food correctly, avoiding cross-contamination and maintaining good personal hygiene

F

Food business

Means any undertaking, whether for profit or not and whether public or private, carrying out any of the activities related to any stage of production, processing, and distribution of food.

Food business operator (FBO)

The natural or legal person responsible for ensuring that the requirements of food law are met within the food business under their control.

Food Safety Management System (FSMS)

A systematic approach to controlling food safety hazards within a food business to ensure that the food produced is safe to eat.

H

HACCP

A food safety management system that identifies, evaluates and controls hazards that are significant for food safety.

Hazard

A biological, chemical or physical or allergenic agent in food, or a condition of food, with the potential to cause an adverse health effect.

L

Less than thoroughly cooked beef burger

A cooked beef burger which has been sourced and cooked using the source control method. 

Local authority (LA)

In England, a county council, a district council, a London borough council, or a parish council. In Wales, a county council, county borough council or community council. In Northern Ireland a district council.

Logarithmic reduction (log reduction)

For the purpose of this guidance, log reduction is a way of measuring the decrease in bacterial numbers following a method of processing such as a cooking procedure. A detailed explanation on how log reduction works is available in Annex 2.

M

Meat preparation (MP)

Defined by Annex I of Retained Regulation (EC) No 853/2004 and Regulation (EC) No 853/2004 as fresh meat, including meat that has been reduced to fragments, which has had foodstuffs, seasonings or additives added to it, or fresh meat which has undergone processes insufficient to modify the internal muscle fibre structure of the meat and thus to eliminate the characteristics of fresh meat.

Minced meat (MM)

Defined by Annex I of Retained Regulation (EC) No. 853/2004 and Regulation (EC) No 853/2004 as boned meat that has been minced into fragments and contains less than 1% salt.

Monitoring

A pre-arranged programme of observations or measurements to determine if the requirements of the FSMS have been achieved or whether control measures at critical points are in danger of failing, which will determine the need to take corrective actions

P

Pathogen

Microorganism that is capable of causing infection.

Patty

For the purpose of this guidance, it is boned meat that has been minced or otherwise reduced into fragments, compressed and given a flat shape and contains less than 1% salt. When the term beef burger or burger is used in this guidance it would include patties.

Primary authority (PA) partnership

A local authority that has formed a partnership with a business or co-ordinator and is nominated by the Secretary of State to exercise certain functions through that partnership.

Probe thermometer

Thermometer designed to be inserted into food to test core/internal temperature.

R

Raw

A raw food is one which has not undergone any processing which would reduce the levels of micro-organisms but which would not be consumed in that state. Raw foods would normally be washed, cooked or treated in some other way prior to consumption to remove dangerous bacteria. Foods which have not undergone any processing but are consumed in that state are considered to be “ready to eat.” Raw foods will include red meat, poultry meat and raw fish, whether or not they have been diced, cut or filleted. It would also include root vegetables and shellfish that have not been processed and are not consumed raw. 

Rare

Product subjected to some cooking, but where part of the product will not reach a sufficient temperature or is not cooked for a sufficient time to cook the product to a six-log reduction in bacteria.

Ready to eat food

A ready to eat food is one which is intended to be consumed without any further treatment or processing which would eliminate or reduce pathogens or their toxins to an acceptable level. Examples of ready to eat foods include cooked, sliced meats; cheese; washed salads; sandwiches; coleslaw and dips. 

Risk

The chance of somebody being harmed by a hazard, and how serious the harm could be.

S

Salmonella

A group of bacteria commonly found in human and animal intestines. It can cause food poisoning and is usually spread by inadequate cooking or cross contamination.

Sampling

The process of collecting a portion of product to be tested.

Sear and shave

The outside surfaces of whole muscle cuts of meat are briefly heated to a high temperature (seared), while leaving inner parts uncooked. The seared surfaces are then hygienically removed leaving the (raw) inner tissues to be used in the production of raw/rare products.

Slaughterhouse

An establishment used for slaughtering and dressing animals, the meat of which is intended for human consumption.

Sous vide

French term meaning “under vacuum.” This is low temperature cooking where the food is sealed in a gas impermeable bag under a vacuum. The food is then cooked in the bag (usually in a water bath) for a defined time at a defined temperature.

Steam surface treatment

Treatment applied to carcasses which involves the use of steam at a specific temperature for a minimum time period to reduce the potential microbial load on the surface.

Steam vacuum

The use of hot water or steam to loosen visible contamination from meat carcases and destroy certain bacteria, followed by the application of a vacuum to remove contaminants. 

STEC

Shiga toxin-producing Escherichia coli

T

Testing

The process of using biological or biochemical methods for the detection, identification or enumeration of microorganisms. 

Thorough cooking

Cooking process where a burger is cooked to 70°C for two minutes, or an equivalent time/temperature combination. 

V

Validation

Obtaining evidence that a control measure or combination of control measures, if properly implemented, is capable of controlling the hazard to a specified outcome. Revalidation may be required in case of changes.

Verification

The application of methods, procedures, tests and other evaluations, in addition to monitoring to determine compliance with the HACCP based procedures. Verification is conducted periodically to demonstrate that the HACCP system is working as planned. 

Vulnerable groups

For the purpose of this guidance, these are the following population groups: children, the elderly, immunocompromised people and pregnant women.

W

Whole muscle cut of meat

A piece of meat which has not been minced, chopped, rolled, pierced, tenderized, or undergone any process or procedure which could allow surface microbial contaminants to enter the inner parts of the muscle tissue, where a surface cook would not inactivate them.