Science governance: a checklist

The science checklist aims to highlight the points to be considered in the use of scientific evidence in the development and communication of policy papers and proposals which deal with science-based issues and those which draw on advice from the Scientific Advisory Committees (SACs).

The FSA has defined science governance as 'the methods by which we assure and demonstrate that scientific evidence and analysis are sought, obtained, interpreted, used and communicated appropriately and effectively by the FSA'.

Our Science and Evidence Strategy sets out our definition of scientific evidence and analysis. It includes:

  • the natural, physical, earth and social sciences, and may also cover other types of evidence (for example, market research) where this is felt to be appropriate
  • science from all sources, not just that commissioned by the Agency

What is our aim?

The Science Checklist is one of our key tools to support science governance. It's aim is to make explicit the points to be considered in the use of scientific evidence in the development and communication of policy papers and proposals, including preparation of FSA Board papers, other papers and submissions assembled by the Executive which deal with science-based issues and those which draw on advice from the Scientific Advisory Committees (SACs).

It addresses:

  • the processes to be followed and whether these are comprehensive
  • what the science says and what its limitations are
  • whether there are controversies and what weight to give to alternative views
  • whether those providing the risk assessment have clearly set out their conclusions

The checklist focuses on the use of scientific evidence and analysis, and particularly on the risk assessment process and how this relates to the wider processes of risk
management and risk analysis.

Governance of evidence gathering is covered by other tools and guidance covering horizon-scanning, prioritisation, engagement with experts and stakeholders, procurement, management and dissemination of evidence. These are outlined in the statement on FSA's framework for science governance and the FSA's Science and Evidence Strategy .

It reflects the 'Post Phillips Framework' published by the FSA in 2002, and in particular the need for a clear formal separation of risk assessment and risk management, allowing for dialogue and iteration between them in an open and transparent manner that respects their distinct roles.

How to use the checklist

The checklist will be used as a guide by the FSA Executive and the SACs. It provides a framework for discussion of the approach to ensuring papers and proposals are evidence based, including the input from SACs, and that this is clearly set out in communicating conclusions and proposals.

It also provides the framework for the assurance and challenge of this work, by the FSA Chief Scientific Advisor (CSA) and Director of Science, Evidence and Research (DSER) and the General Advisory Committee on Science (GACS), and ultimately by the Board.

Not all of the points in the checklist will be relevant for all proposals and papers. Use of the checklist at the planning stage helps to ensure that all relevant points are identified, and that plans to address them are factored into the approach. If it is not clear whether any issues arise, this can be discussed and an approach agreed at an early stage with the FSA DSER or CSA.

The checklist is intended to highlight the points that need to be considered, rather than to provide detailed guidance or instructions on the specific answers in all individual cases. Users will need to be aware of and refer to principles and guidance in specific areas. Some important example are listed in the framework for science governance (for example Codex principles, COT guidance on risk assessment; Magenta book, GSR Code, NICE guidelines).

Criteria for success

Success criteria for the checklist are that:

  • the CSA and the Board are assured that the specified work has been done, and to an acceptable standard
  • the CSA and the Board have confidence that the FSA’s assessment and use of the science-base is comprehensive and has been interpreted correctly
  • trust is maintained within the Board about the Agency’s collection and interpretation of scientific evidence
  • the FSA’s approach to and use of evidence is consistent, open and transparent to stakeholders
  • the external perception and reputation of the FSA as a science-and evidence- based organisation is maintained

The science checklist

Below are some of the questions that you may need to consider for your proposal and papers. Using this checklist at the planning stages will help you ensure that all relevant points are identified, and that plans to address them are factored into the approach.

Defining the problem and the approach

1. Has the problem been clearly defined?

2. Have all the different strands of scientific evidence and analysis needed to address the problem been identified?

3. Have different stakeholder views been taken into account when framing the issues and questions to be addressed?

4. If the issue falls between Scientific Advisory Committees (SACs) or to more than one SAC, has a satisfactory mechanism been put in place to ensure that all players are involved and that each SAC is aware of the views of the others?

5. Has the approach been agreed at the outset (including: the definition of any issues to be considered by SACs; the need for and approach to iteration between FSA and relevant SACs; and any changes from or additions to the framework for science governance)?

Gathering and assessing the evidence

6. What steps have been taken to ensure that all available and relevant scientific evidence has been considered by the appropriate body (e.g. SAC, FSA, external experts)? For the overall evidence base, this should include considering the following:

  • Has a comprehensive literature survey been undertaken?
  • Have external scientific experts been consulted who may know of relevant unpublished data? What steps have been taken to ensure that these data are reliable?
  • Has evidence been sought from the right stakeholders, and has this been assessed?
  • Is further research required?
  • Is there a need to consider different data sets for different regions of the UK or for different groups of the population?
  • Is the scientific evidence base transparent to stakeholders, and is it clear which evidence has been peer-reviewed

7. What steps have been taken to assess the individual pieces of evidence? This should include the following:

  • Has the appropriate methodology been used?
  • What is the strength of the quantitative scientific evidence; e.g. is it relatively weak such as anecdotal or from a single case study, or relatively strong such as from a large double-blind controlled experimental study?
  • Is qualitative evidence robust?
  • If evidence was collected outside the UK (or from contexts different from that under consideration), has the relevance to the UK situation and the current problem been assessed?
  • How do the findings contribute to the overall evidence base? Are there any contradictions between findings?

8. Are the conclusions consistent with the quantitative and/or qualitative evidence, both in character and emphasis?

Risk assessment

9. Are the evidence underpinning the risk assessment and the assumptions set out clearly?

10. Has an assessment been made of the likely impact and probability of occurrence?

11. Are all key scientific uncertainties including gaps in evidence or analysis highlighted?

12. Has any indication been given about the degree of uncertainty or consensus involved?

13. How have the areas of uncertainty including gaps in evidence or analysis been handled when reaching final conclusions and how do they impact on the advice?

14. Did the risk assessment consider the views of experts in all relevant disciplines, either as members of a committee or other invited experts?

15. Is it clear how the conclusion is reached, based on the evidence considered? In particular, is the extent to which judgement has been used clear?

16. Are there any alternative interpretations of the same evidence? Have they been considered? Why were they rejected?

17. How has the assessment committee (or other body) taken account of any conflicting views? Have any risk assessments carried out by others been cited? To what extent are there consensus/differing views?

18. Has the SAC or Agency consulted on the draft conclusions? How have the results been taken into account?

19. Are the conclusions/advice expressed in clear, simple terms? Do they include all important caveats and explanations necessary to properly understand the conclusion?

Interpretation

20. Is the scientific evidence and analysis (including SAC advice, any other assessments of risk, and evidence and analysis on other aspects of the proposals) presented and represented correctly and clearly in the paper?

21. Are significant limitations clearly explained?

22. If this is a review, have external factors that could affect the approach or interpretation changed since the last consideration, such that assumptions should have been challenged?

23. Is there a need for further iteration and dialogue between FSA and the relevant SAC(s) – for example to seek SAC views on whether (any of) the risk management options are consistent with the risk assessment, or to highlight any needs for research or for additional evidence or analysis?

24. If both risks and benefits were considered, is the degree of rigour and uncertainty associated with each set out clearly?Does the conclusion indicate whether the evidence base is changing or static, and when it may need to be reviewed? What are the views of the relevant SAC(s) (or other experts) on this?Has a picture of the external environment been given so that the Board (or other party being asked to make a decision on the basis of the proposals) knows the effect of any proposals or risk assessment, including ethical or sustainability issues, the responsibilities of all parties – government, enforcement and industry- or the need to educate consumers?

Related pages