Peer Review Report of Preliminary Estimation of Perchlorate Dietary Exposure Based on FDA 2004/2005 Exploratory Data

Peer Review Charge and Questions

In May 2006, FDA sent a draft of its 2004-2005 Exploratory Survey Data on Perchlorate in Food and the Preliminary Estimation of Perchlorate Dietary Exposure Based on 2004/2005 Exploratory Survey Data to three expert reviewers (see Part C below)

The charge to the reviewers was as follows:

The reviewers are asked to evaluate whether the exposure method (i.e., probabilistic Monte Carlos simulation described in the document) used to calculate exposure estimates for the 27 foods and beverages is tenable. The reviewers are also asked to ensure that scientific uncertainties are clearly identified and clarified in the exposure estimate. For example, the reviewers are asked whether the nature of the estimate (the uncertainty in the exposure estimate derived) is adequately qualified due to the limited perchlorate data available on the levels of perchlorate in foods. However, the reviewers are not to provide advice on the policy (e.g., the amount of uncertainty that is acceptable or the amount of precaution that should be embedded in the analysis).

Reviewer Comments and FDA Responses

Reviewer 1

Evaluation:

The perchlorate exposure estimates are calculated from data obtained from FDA's FY04 and FY05 exploratory surveys. Chemical exposure from consumption of the 27 foods and beverages was estimated by Monte Carlo simulation using @Risk software. The report is clearly identified as an estimation of perchlorate dietary exposure, and defines the limitations of the study. The data are clearly identified as having been obtained from a specific number of foods collected and analyzed for perchlorate residues under FDA programs. The report clearly states that sampling was done in regions of the U.S. where soil and water sources are believed to contain perchlorate contamination, and therefore the subject samples were expected to contain perchlorate residues. The analytical results, therefore, are identified as biased on the side of expected perchlorate residues and higher average perchlorate residues. You correctly state that further sampling and analyses will better characterize perchlorate distribution and provide a more precise assessment.

Conclusion:

We find that the Monte Carlo Simulation probabilistic model addressing variability and uncertainty is a reasonable, common, and widely used computational method in toxicology and its use in the report are tenable and desirable when considering the stated limitations of the data set. The uncertainties are clearly identified and are adequately qualified due to the limitations of the data set. We find the report to be adequate with respect to addressing the uncertainties of the estimation of perchlorate dietary exposure.

FDA Response: No response needed.

Reviewer 2

2.1 Comment: Discrete uniform distribution may not be appropriate.

Observation: The model is described as using a discrete uniform distribution to model perchlorate levels. Some of the products have a large sample set (e.g., milk: n = 125); others have a small set (e.g., potatoes: n = 6). Because the discrete uniform only resamples the original data, this distribution will give the appearance of much less variability regarding the value for milk than for orange juice. Collecting more data for orange juice would have the paradoxical effect of increasing the variability or uncertainty.

Recommendation: Allow upper and lower limits to express possible variation or uncertainty in empiric distributions.

FDA Response: We agree that distribution for a larger sample set (e.g., milk: n=125) will give the appearance of much less variability than for a smaller sample set (e.g., potatoes: n=6). However, for this preliminary assessment, without additional data for products having small sample sets, we believe the use of discrete uniform distribution is appropriate because there is no indication at this time that these small sample sets are not indicative of the distribution for larger sample sets. Additional sampling will reduce the uncertainty of this assumption, but we also recognize that the appearance of variability of the perchlorate levels could increase (i.e., the upper and lower perchlorate levels found for each food type may change) and note that there will always be variability for a given samples set, whether that sample set is large or small. The use of the uniform distribution is no less arbitrary than the choice to use only upper or lower limits for each food. As the 27 food groups have been examined, there is no indication that using a limit for any individual food would have a meaningful impact on the estimated total perchlorate exposure.
2.2 Comment: Only 32 percent of the diet is represented.

The text states:

"Although consumption of the 27 foods and beverages represents only about 32 percent of the total diet for the U.S. population, ages 2 years and older, these foods and beverages with high water content would be expected to contain higher levels of perchlorate, a highly water soluble chemical, and to contribute more to the overall dietary perchorate exposure than foods with lower water content."

Because two-thirds of the dietary intake is not considered, the true average intake could be much higher or lower than listed in the document. Although the document states that the sampled foods have a higher water content than average, there are certainly other high water content foods that were not considered, such as celery, pears, and peaches. Furthermore, products such as flour and corn meal should not really be considered as having high water contents.

Recommendation: Extrapolate the consumption to 100% of the diet. Explain the limitations inherent in the approach.

FDA Response: Currently, we do not have any information on the distribution of perchlorate in foods that constitute the remaining two-thirds of the total diet. We agree that the true average intake of perchlorate could be higher than the calculated average intake based on the foods for which we currently have data. We have added a statement reflecting this uncertainty in our preliminary exposure estimation document.

We do not agree, however, that we should extrapolate the consumption to 100 percent of the diet. First, it is not clear how we would do this, as any assumption about the levels of perchlorate in the remainder of the diet would be merely a guess. Simply multiplying the mean intake by three (to "correct" from 32 percent to 100 percent) would not provide a sound, scientific estimate of intake at 100 percent of the diet. Besides, while accounting for only about 32 percent of the total diet, the 27 foods and beverages with high water content would be expected to contain perchlorate and contribute more to overall dietary exposure than foods with lower water content. As such, we have chosen to describe this preliminary assessment on perchlorate exposure estimates based on available data for the 27 types of foods and beverages that constitute about 32 percent of the total diet and not extrapolate to estimate the total exposure from all foods in the diet.
2.3 Comment: Consideration of uncertainty insufficiently documented.

Observation: The document does not appear to consider the effect of uncertainty.

Recommendation: Include documentation discussing the major sources of uncertainty and their effects on the model. This could include either quantitative or qualitative considerations. For example, all five reported levels of perchlorate in orange juice came from samples from just a single city.

FDA Response: We have added additional language in the document to identify some major sources of uncertainty and ways to further reduce that uncertainty.
2.4 Comment: Correlation of intakes not considered.

Observation: The document does not discuss whether draws from the distributions were correlated. As the number of products increases the sum of the draws from any type of distributions tends to approach the sum of the means of the individual distributions. As an example, assume that individuals that eat more of one type of high water content food would tend to eat more of other high water content foods. So the individual would eat more lettuce, and carrots, and apples, and oranges, and spinach. Modeling these products as if they are not correlated means that we would sample combinations at the 90^th percentile or greater at a frequency of (1-0.90)⁵ or 0.001% of the time. One would expect such combinations to not be included in simulations of 5000 iterations. It is important to consider such correlations.

Recommendation: Include an analysis that assumes perfect or varying degrees of correlation between consumption levels.

FDA Response: For a contaminant, i.e., perchlorate, distributed widely in the diet, we believe the effect of correlation on the exposure estimates is minimal, and therefore we did not examine any correlation effect in this assessment. For example, if it were the case that specific types of foods, whose consumption patterns were likely to be correlated (e.g., soup and sandwich or peanut butter and jelly), contained high levels of perchlorate, and perchlorate intake was likely to arise from consumption of only these foods, then correlation in the intake model would be appropriate. However, in this case, we do not think correlation is an issue that needs to be factored into the exposure assessment because perchlorate is so widely distributed the food supply that any alternative food choice is almost equally likely to present some exposure potential. Therefore, we do not expect it to affect the overall estimates at the mean or 90^th percentiles.
2.5 Comment: The following comments refer specifically to the implementation of the Monte Carlo model as referred to in the document, "Preliminary Estimation of Perchlorate Dietary Exposure Based on FDA 2004/2005 Exploratory Data." The model performs as described in the documentation. There is one food described in the documentation (rice) that appears in one of the spreadsheets but is not included in the final calculations. I found no other problems in the implementation. Better documentation of model variables and calculations could improve transparency.
2.6a Comment: Rice is referenced in the documentation but not included in model calculations. Lima beans are included in the model calculations but not referenced in the documentation.
Observation: The documentation refers to 27 different products that are included in the exposure assessment and lists them in Table 2 as Lettuce, Milk, Tomatoes, Carrots, Spinach, Cantaloupes, Apples, Grapes, Oranges, Strawberries, Watermelon, Fruit Juices (Apple & Orange), Broccoli, Cabbage, Greens, Cucumber, Green Beans, Onions, Potatoes, Sweet Potatoes, Corn Meal, Oatmeal, Rice (Brown & White), Whole Wheat Flour, Catfish, Salmon, and Shrimp. Three different spreadsheets are included in the model. These represent the three different exposure groups: 2+, 2-5, and 15-45. The group 2+ includes a calculation for rice but the resultant value is not included in the total for the group. Groups 2-5 and 15-45 do not include a calculation for rice. I have run the model for 5000 iterations with rice included and it does not appear to make a significant difference.

Recommendation: Ensure that the correct number of food products is given in the documentation and calculated in the model.

FDA Response: All of the rice samples were non-detects for perchlorate. Therefore, because of the expectation that inclusion of rice in the calculated intake values for group 2+, as well as for groups 2-5 and 15-45, would not make a significance difference in the total intake of perchlorate (as the reviewer noted), we did not include calculations for rice in this assessment. We have made a notation reflecting this fact at the bottom of Table 2 in our preliminary exposure estimation document. In addition, although we included lima beans in the model calculations, we did not include it as part of this assessment because perchlorate levels were available for only two lima bean samples that would not make a significant difference in the total intake of perchlorate.
2.6b-d
1. 2.6b Comment: Documentation of the model could be improved.
  
  Observation: Although the model is straightforward it does take a little time to determine how it works.
  
  Recommendation: Consider adding some explanatory notes to the worksheets in the form of comments, text boxes, or call-outs.
2. 2.6c Comment: Maximums for the cumulative distribution should use a consistent formula for transparency.
  
  Observation: The cumulative distribution for consumption is entered in the form RiskCumul(min, max, values_array, probabilities_array). The minimum is 0 for each product and the arrays reference the consumption data. The maximum, however, is different for each product and appears to be related to the maximum value given in the data.
  
  Recommendation: Calculate the maximum value in the cumulative distribution based on the maximum value in the data (e.g., max=Max(values_array)*1.1). This will ensure consistent treatment of the distributions, improve transparency, and help avoid errors if the arrays change.
3. 2.6d Comment: Cells should be used to hold inputs rather than hard-coding values into formulas.
  
  Observation: The RiskDiscrete function for intake relies on hard-coded values for "% eaters" values that are available elsewhere in the worksheets
  
  Recommendation: Have the function take values from the appropriate worksheet cells rather than hard-coding them. As with the recommendation above, this will ensure consistent treatment of the distributions, improve transparency, and help avoid errors if the arrays change.
FDA Response to Comments 2.6b-d: We appreciate these comments and recommendations for improving the implementation of the model and will consider them for future use of the model. However, as the reviewer noted, the model performs as described in the documentation and implementation of the reviewer's recommendation would not affect the outcome of the exposure estimates. We note that the model is sufficiently transparent that modelers, such as the reviewers, are able to review it and understand how the model performs. Therefore, no changes in the implementation of the model are needed for the purpose of this preliminary perchlorate exposure assessment.
We agree with the comment in 2.6c and will implement the formula in future runs of the model. Right now the maximum is simply a round number higher than the 99^th percentile in the distribution of food intakes. In response to comment 2.6d, unfortunately, we are not able to use an input in all of the cells and have been forced by the software to hard-code some values (notably the percentage of eaters of each type of food).

Reviewer 3

3.1 Comment: It was unclear how the authors determined whether foods were consumed "primarily" as an ingredient or whole. Though some are intuitive, others are not. For example, there is probably little argument that cornmeal is most often consumed as part of a mixed-ingredient food dish. Conversely, however, whether cantaloupes are consumed primarily as an ingredient is less clear. For example, of the following, can you easily discern those items that are consumed as an ingredient vs. those that are consumed whole: spinach, strawberries, broccoli, cucumbers, green beans, carrots? The document makes the distinction, but it was unclear how.

FDA Response: The foods were determined to be consumed primarily as an ingredient or as whole so as to maximize the reported intake of the food, i.e., whichever way gave the higher intake of perchlorate was used. For example, because spinach is consumed in higher amount as an ingredient than as whole (USDA Continuing Survey of Food Intakes by Individuals 1994-1996 and 1998 Supplemental Children's Survey), it was categorized as an ingredient that resulted in higher perchlorate intake. Therefore, this preliminary assessment provides highly conservative perchlorate exposure estimates. We have added this explanation to the text of our preliminary exposure estimation document.
3.2 Comment: A reference (or web site at the very least) was needed for the "Environ Dietary Exposure Assessment." In short, wherever a database, software application, reference dose, recommended exposure levels, etc. is mentioned, a reference must be provided.

FDA Response: Web site references have been added for the Environ software application and the NAS recommended reference dose adopted by the Environmental Protection Agency.
3.3 Comment: The meaning of a "stable" exposure result was not clear. It would have been best to define it explicitly (e.g., 500 model returns within 0.5%, etc.).

FDA Response: The meaning of a "stable" exposure result has been explicitly defined by adding the phrase "(the change in statistics monitored every 50 iterations is less than 1% after 2300 iterations)" to the sentence.
3.4 Comment: Units should have been given in instances such as, "all persons aged 2 and above." Though most readers would, I suspect, read this as two years, inclusion of units removes any chance for confusion.

FDA Response: The unit "years" has been added.
3.5 Comment: Specifics should have been given for the geographic locations of produce samples. Stating that they "were collected primarily from regions where water sources are believed to be contaminated" was woefully insufficient. Furthermore, this quoted statement underscores the needs for references (who's belief?).

FDA Response: FDA's FY04/05 perchlorate survey data (see "2004-2005 Exploratory Survey Data on Perchlorate in Food"), which this preliminary perchlorate exposure assessment is based on, provide names of cities and states where produce samples were collected. We have revised the above statement to say "were collected particularly from regions (i.e., Southern California and Arizona) where water sources are known to be contaminated" with a web site reference added.

Reviewers

John D. McCurdy, Ph.D., D.A.B.T.
Regulatory Chemist & Toxicologist
Office of Surveillance and Compliance
Center for Veterinary Medicine
Food and Drug Administration

Dr. McCurdy is a board certified toxicologist with expertise in the evaluation of chemical manufacturing controls and target animal safety studies with respect to food additives, novel feed ingredients and feedingstuffs. He performed major safety evaluations and risk assessments for the irradiation of animal feeds, single cell proteins, and the use of formaldehyde in feed. He is an observer/participant, ILSI North America Subcommittee on Acrylamide, Toxicology and Metabolic Consequences Working Group. He currently devotes most of his review time to the chemistry, technical and safe use of industrial enzymes in feed manufactured through rDNA and fermentation processes.

Wayne Schlosser, DVM, MPH, DACVPM (Epidemiology)
Senior Risk Analyst
Office of Public Health Science
Food Safety and Inspection Service
United States Department of Agriculture

Dr. Schlosser is a board certified veterinarian in epidemiology and preventive medicine, and has over twenty years experience in animal health, public health, food safety, and risk assessment in military and government. Dr. Schlosser, an expert in probabilistic modeling, has developed several quantitative microbial risk assessments and is experienced in developing exposure assessments that utilize Monte Carlo methods. He is an instructor in food safety, epidemiology, and risk assessment, and author/coauthor of over 15 publications, over 25 published presentations, and 6 book chapters.

Carl Schroeder, Ph.D.
Microbiologist
Office of Public Health Science
Food Safety and Inspection Service
United States Department of Agriculture

Dr. Schroeder trained as a microbiologist and has been with the Food Safety and Inspection Service since 2002. During this time he has assisted with development of several risk assessments, including leading the agency's risk assessments for Salmonella in eggs and liquid egg products and Salmonella in ready-to-eat meat and poultry products. He earned a Ph.D. from Marquette University and completed postdoctoral training at the University of Maryland.