Below are some of the questions that you may need to consider for your proposal and papers. Using this checklist at the planning stages will help you ensure that all relevant points are identified, and that plans to address them are factored into the approach.
Defining the problem and the approach
1. Has the problem been clearly defined?
2. Have all the different strands of scientific evidence and analysis needed to address the problem been identified?
3. Have different stakeholder views been taken into account when framing the issues and questions to be addressed?
4. If the issue falls between Scientific Advisory Committees (SACs) or to more than one SAC, has a satisfactory mechanism been put in place to ensure that all players are involved and that each SAC is aware of the views of the others?
5. Has the approach been agreed at the outset (including: the definition of any issues to be considered by SACs; the need for and approach to iteration between FSA and relevant SACs; and any changes from or additions to the framework for science governance)?
Gathering and assessing the evidence
6. What steps have been taken to ensure that all available and relevant scientific evidence has been considered by the appropriate body (e.g. SAC, FSA, external experts)? For the overall evidence base, this should include considering the following:
- Has a comprehensive literature survey been undertaken?
- Have external scientific experts been consulted who may know of relevant unpublished data? What steps have been taken to ensure that these data are reliable?
- Has evidence been sought from the right stakeholders, and has this been assessed?
- Is further research required?
- Is there a need to consider different data sets for different regions of the UK or for different groups of the population?
- Is the scientific evidence base transparent to stakeholders, and is it clear which evidence has been peer-reviewed
7. What steps have been taken to assess the individual pieces of evidence? This should include the following:
- Has the appropriate methodology been used?
- What is the strength of the quantitative scientific evidence; e.g. is it relatively weak such as anecdotal or from a single case study, or relatively strong such as from a large double-blind controlled experimental study?
- Is qualitative evidence robust?
- If evidence was collected outside the UK (or from contexts different from that under consideration), has the relevance to the UK situation and the current problem been assessed?
- How do the findings contribute to the overall evidence base? Are there any contradictions between findings?
8. Are the conclusions consistent with the quantitative and/or qualitative evidence, both in character and emphasis?
Risk assessment
9. Are the evidence underpinning the risk assessment and the assumptions set out clearly?
10. Has an assessment been made of the likely impact and probability of occurrence?
11. Are all key scientific uncertainties including gaps in evidence or analysis highlighted?
12. Has any indication been given about the degree of uncertainty or consensus involved?
13. How have the areas of uncertainty including gaps in evidence or analysis been handled when reaching final conclusions and how do they impact on the advice?
14. Did the risk assessment consider the views of experts in all relevant disciplines, either as members of a committee or other invited experts?
15. Is it clear how the conclusion is reached, based on the evidence considered? In particular, is the extent to which judgement has been used clear?
16. Are there any alternative interpretations of the same evidence? Have they been considered? Why were they rejected?
17. How has the assessment committee (or other body) taken account of any conflicting views? Have any risk assessments carried out by others been cited? To what extent are there consensus/differing views?
18. Has the SAC or Agency consulted on the draft conclusions? How have the results been taken into account?
19. Are the conclusions/advice expressed in clear, simple terms? Do they include all important caveats and explanations necessary to properly understand the conclusion?
Interpretation
20. Is the scientific evidence and analysis (including SAC advice, any other assessments of risk, and evidence and analysis on other aspects of the proposals) presented and represented correctly and clearly in the paper?
21. Are significant limitations clearly explained?
22. If this is a review, have external factors that could affect the approach or interpretation changed since the last consideration, such that assumptions should have been challenged?
23. Is there a need for further iteration and dialogue between FSA and the relevant SAC(s) – for example to seek SAC views on whether (any of) the risk management options are consistent with the risk assessment, or to highlight any needs for research or for additional evidence or analysis?
24. If both risks and benefits were considered, is the degree of rigour and uncertainty associated with each set out clearly?Does the conclusion indicate whether the evidence base is changing or static, and when it may need to be reviewed? What are the views of the relevant SAC(s) (or other experts) on this?Has a picture of the external environment been given so that the Board (or other party being asked to make a decision on the basis of the proposals) knows the effect of any proposals or risk assessment, including ethical or sustainability issues, the responsibilities of all parties – government, enforcement and industry- or the need to educate consumers?
Framework for science governance