2023 FDA Science Forum
DeepAmes: Deep learning-powered Ames test predictive models for regulatory application by harnessing a comprehensive and consistent dataset
- Authors:
- Center:
-
Contributing OfficeNational Center for Toxicological Research
Abstract
Mutagenicity is an important endpoint to assess the safety of consumer products (e.g., drugs and environmental chemicals). It is often required by regulatory agencies, such as FDA, EPA, etc. Ames assay is a widely used method to assess mutagenicity of chemicals. However, Ames assay requires approximately 2 g of sample for a dose-finding study and main study, which poses a challenge for chemical compounds of small amount, such as impurity of a drug and food flavor. Therefore, in silico approaches such as Quantitative Structure-Activity Relationships (QSARs) have been widely used to predict Ames test results. International Council for Harmonization (ICH) also provides guidelines to conduct such in silico analysis for regulatory application. We report here a high performance and robust model, called DeepAmes which is developed with a novel deep learning approach, for its potential utility in regulatory science. Specifically, we developed DeepAmes with the largest and most consistent Ames dataset (>12,000 compounds). We compared DeepAmes with QSAR models from 5 standard Machine Learning (ML) methods; these are k-nearest neighborhood (KNN), logistic regression (LR), support vector machine (SVM), random forest (RF), extreme gradient boosting (XGBoost). All the six models were evaluated and compared using a test set of 1,543 compounds. DeepAmes achieved the top performance in prediction. With respect to applicability domain, DeepAmes yielded the best and stable performance till the compounds were 30% outside of training domain. The regulatory application of a model is focused on context-of-use, where false negatives are of utmost concern since positive prediction could be eliminated by downstream methods such as experimental methods. Compared to 5 standard ML models, DeepAmes yielded the highest sensitivity of 0.47, which was a 19.5% improvement in comparison with the largest sensitivity of the other five ML methods on the test set. More importantly, we provided a revised version of DeepAmes with a high sensitivity of 0.87. This improvement in sensitivity is significant given the fact that there are only ~15% mutagenic compounds in the test set. In conclusion, DeepAmes has the potential for regulatory application of predicting the Ames test result.