Expert Elicitation on the Relative Risks of Processed Meat and Poultry Products Presented by Mary K. Muth, Shawn A. Karns, and Michaela C. Coglaiti RTI International Presented at Public Meeting on Risk Based Inspection (RBI)—Expert Elicitation June 26, 2007 U.S. Department of Agriculture, Washington, DC 3040 Cornwallis Road ■ P.O. Box 12194 ■ Research Triangle Park, NC 27709 Phone 919-541-7289 Fax 919-541-6683 E-mail muth@rti.org RTI International is a trade name of Research Triangle Institute 2 Presentation Outline „ History of the expert elicitation for processed meat and poultry products „ Development and contents of the expert elicitation worksheet „ Recruitment and members of the expert elicitation panel „ Process for conducting the expert elicitation „ Highlights of the expert elicitation results 3 Expert Elicitation History „ 2001: RTI conducted initial expert elicitation, ranking relative risks of processes but not processed meat type (species). „ 2005: RTI conducted an expert elicitation on the relative risks posed to public health by various types of processed meat and poultry products, including meat type (species). „ 2006: FSIS held a public workshop on plans for implementing risk-based inspection and received feedback on the 2005 expert elicitation. „ 2007: FSIS contracted with RTI to conduct a new expert elicitation with modifications to address comments on the 2005 expert elicitation. 4 Primary Modifications to the 2005 Expert Elicitation „ Experts were equally divided among individuals from public health, academic institutions, and industry „ Two additional worksheets were added to z Include risk ranking for vulnerable consumers z Address attribution of foodborne illness to individual product categories „ Scoring range was limited to 1 to 10 (instead of open- ended scoring) 5 Expert Elicitation Worksheets: Development and Contents (1) „ RTI and FSIS held a series of conference calls to discuss required modifications and additions to the worksheets „ Initial draft of the worksheets was peer reviewed z Peer reviewers included a senior advisor for regulatory support, veterinary epidemiologist, deputy director for research, and senior scientist z Peer reviewers were from FDA, EPA, USDA/ERS, and USDA/APHIS 6 Expert Elicitation Worksheets: Development and Contents (2) „ Worksheets were modified in response to the peer reviewers’ comments „ Worksheets were internally reviewed at FSIS „ FSIS conducted a pilot test of the instruments with three FSIS scientists z Based on pilot test feedback, a worksheet was added on attribution of foodborne illness to various categories of processed meat & poultry products „ RTI made final changes to the worksheets 7 Expert Elicitation Worksheets: Development and Contents (3) „ Worksheet 1: Ranks the public health risks posed by bacterial hazards for 25 product categories for healthy adults „ Worksheet 2: Ranks the public health risks posed by bacterial hazards for 25 product categories for vulnerable consumers „ Worksheet 3: Obtains estimates of the percentages of illnesses caused by consuming or handling foods in each of the 25 product categories z Salmonella (non-typhi), Salmonella (multidrug resistant), E. coli O157:H7, Listeria monocytogenes, and Campylobacter jejuni 8 Expert Elicitation Worksheets: Development and Contents (4) „ Differences compared with the 2005 expert elicitation: z Added Worksheet 2 (vulnerable consumers) and Worksheet 3 (attribution) z Added product category “Thermally processed, commercially sterile” z Limited scoring to 1 to 10 (fractions allowed) z Considered only bacterial hazards (not viruses or physical or chemical hazards) z Experts indicated level of confidence in their estimates 9 Expert Panel: Recruitment and Members (1) „ Identified 45 potential experts z 15 each in public health, academia, and industry z Criteria for inclusion: ‹ advanced knowledge and professional recognition in a branch of science related to public health and food safety ‹ an understanding of food science, meat and poultry processing, and foodborne illness z List generated by FSIS, RTI, and the National Advisory Committee on Meat & Poultry Inspection z Of those contacted, 14 declined or dropped out and 14 did not respond to calls or e-mail 10 Expert Panel: Recruitment and Members (2) „ Recruited 17 experts z 4 public health, 5 academic, and 8 industry „ All 17 recruited experts completed the worksheets z Names are provided in the report prepared for FSIS „ To ensure a balanced panel, 4 experts from each group were randomly selected for inclusion in the summary statistics 11 Expert Elicitation Process (1) „ Provided experts with the following materials: z 3 worksheets to be completed z List of examples of products for each of the 25 product categories ‹ Same examples as the 2005 expert elicitation except that “Thermally processed, commercially sterile” was added z List of assumptions to be used while assigning risk scores in Worksheets 1 and 2 12 Expert Elicitation Process (2) „ Scheduled and hosted series of teleconferences with groups of the experts to discuss purpose of the data collection, review the worksheets, and respond to questions „ Experts completed the worksheets within 1 week after the teleconference „ RTI received completed worksheets and entered data into a spreadsheet 13 Expert Elicitation Process (3) „ Assumptions provided to the experts: 1. Consider only bacterial hazards (different from 2005 expert elicitation) 2. Products will reach consumers without further processing at another establishment or at retail 3. Products are produced in a USDA-regulated plant with HACCP and SSOPs 4. Incoming source material comes from a supplier with average or typical food safety controls 14 Expert Elicitation Process (4) „ Assumptions provided to the experts (continued): 5. Processing plant’s food safety controls are average or typical 6. Products receive typical handling from the time they leave the processing plant until they are consumed 7. Raw products are cooked before consumption 8. Products are not irradiated 15 Expert Elicitation Process (5) „ Assumptions provided to the experts (continued): 9. For RTE products: a. Products are exposed to the environment after lethality treatments (unless specifically noted otherwise) b. Products do not contain additives to inhibit growth of Listeria monocytogenes c. Products do not receive postlethality treatment to destroy Listeria monocytogenes 16 Highlights of Expert Elicitation Results (1) „ Experts’ scores for product categories with highest likelihood of illness among healthy adults Product Type Median Score Minimum Score Maximum Score Raw ground, comminuted, or otherwise nonintact chicken 10 4 10 Raw ground, comminuted, or otherwise nonintact turkey 9 4 10 Raw ground, comminuted, or otherwise nonintact poultry―other 8.5 1 10 17 Highlights of Expert Elicitation Results (2) „ Experts’ scores for product categories with lowest likelihood of illness among healthy adults Product Type Median Score Minimum Score Maximum Score Thermally processed, commercially sterile 1 1 1 RTE meat fully cooked without subsequent exposure to environment 1.6 1 4 RTE poultry fully cooked without subsequent exposure to environment 1.6 1 4 18 Highlights of Expert Elicitation Results (3) „ Experts’ scores for product categories with highest likelihood of illness among vulnerable consumers Product Type Median Score Minimum Score Maximum Score Raw ground, comminuted, or otherwise nonintact chicken 10 4 10 Raw ground, comminuted, or otherwise nonintact beef 9.5 4 10 Raw ground, comminuted, or otherwise nonintact turkey 9 4 10 19 Highlights of Expert Elicitation Results (4) „ Experts’ scores for product categories with lowest likelihood of illness among vulnerable consumers Product Type Median Score Minimum Score Maximum Score Thermally processed, commercially sterile 1 1 1 RTE meat fully cooked without subsequent exposure to environment 2 1 5 RTE poultry fully cooked without subsequent exposure to environment 2 1 5 20 Highlights of Expert Elicitation Results (5) „ For both risk-ranking worksheets: z Results were very similar between healthy adults and vulnerable consumers. z Raw products were assigned higher risk rankings than RTE products. z Poultry products were assigned higher risk rankings than red meat products. z Opinions of experts varied substantially for some products (wide range of scores). 21 Highlights of Expert Elicitation Results (6) „ Product categories with highest attribution percentages for Salmonella (non-typhi) Product Type Mean Minimum Maximum Raw intact chicken 22% 10% 50% Raw intact turkey 14% 3% 40% Raw ground, comminuted, or otherwise nonintact chicken 9% 1% 20% Mean level of confidence = 2.2 (1=low, 2=medium, 3=high) 22 Highlights of Expert Elicitation Results (7) „ Product categories with highest attribution percentages for Salmonella (multidrug resistant) Product Type Mean Minimum Maximum Raw ground, comminuted, or otherwise nonintact beef 20% 1% 74% Raw intact chicken 19% 1% 60% Raw ground, comminuted, or otherwise nonintact chicken 8% 0% 16% Mean level of confidence = 1.7 (1=low, 2=medium, 3=high) 23 Highlights of Expert Elicitation Results (8) „ Product categories with highest attribution percentages for E. coli O157:H7 Product Type Mean Minimum Maximum Raw ground, comminuted, or otherwise nonintact beef 57% 0% 92% Raw ground, comminuted, or otherwise nonintact meat—other 14% 0% 75% Raw intact beef 8% 0% 20% Mean level of confidence = 2.3 (1=low, 2=medium, 3=high) 24 Highlights of Expert Elicitation Results (9) „ Product categories with highest attribution percentages for Listeria monocytogenes Product Type Mean Minimum Maximum RTE fully cooked meat 30% 0% 95% RTE fully cooked poultry 25% 0% 58% RTE acidified/fermented meat (without cooking) 6% 0% 40% Mean level of confidence = 2.4 (1=low, 2=medium, 3=high) 25 Highlights of Expert Elicitation Results (10) „ Product categories with highest attribution percentages for Campylobacter jejuni/coli Product Type Mean Minimum Maximum Raw intact chicken 36% 5% 90% Raw intact turkey 13% 2% 40% Raw ground, comminuted or otherwise nonintact chicken 12% 0% 25% Mean level of confidence = 2.0 (1=low, 2=medium, 3=high) 26 Questions? Full report will be available at http://www.fsis.usda.gov/Regulations_&_Policies/RBI _in_Processing/index.asp