A major challenge in hazard assessment of drug and material discovery is the identification of potentially toxic compounds. To assess potential toxicity of the compounds, different assays are conducted; however, they are usually costly and time-consuming. Therefore, machine learning (ML) is often used to estimate the effect of a compound on a specific endpoint based on molecular features. However, finding an appropriate ML model, compound representation, and dimensionality reduction (DR) method remains challenging. Therefore, we benchmark a wide range of ML algorithms, molecular representations, and DR approaches to identify effective combinations for predicting toxicity across ToxCast endpoints. In particular, we consider 52 hormone receptor activity assays (e.g., estrogen or androgen receptor), each having roughly 500-9,000 tested compounds. To characterize the compounds, we compare transformer-based embeddings, structural fingerprints (Morgan and MACCS), and physicochemical descriptors. For these features, we apply different DR techniques (i.e., principal component analysis, variance-based selection, mutual information, and minimum-redundancy-maximum-relevance). Finally, we train different ML models (i.e., random forest (RF), multi-layer perceptron (MLP), CatBoost, support vector machine (SVM), and TabPFN). Using a nested 5×4 cross-validation setup, we evaluate 100 model–feature–DR combinations per assay. Performance is assessed using the Matthews correlation coefficient (MCC), one of the most appropriate metrics for imbalanced data. Despite the fact that the winning algorithm of the toxicity prediction challenge Tox21 was a deep neural network (NN), our results show that RF trained on physicochemical properties without DR is the combination that is most often ranked as top 1 (in 12/52 assays). Moreover, it ranks among the top 5 out of 100 combinations in half (26/52) of the tested assays. When ranking models per assay, RFs significantly outperform all other models, achieving an average rank of 1.4 and an MCC of 0.207. SVM, CatBoost, follow with ranks around 3 and MCC values near 0.15, while MLP and TabPFN show the worst performance with ranks around 4 and average MCC values around 0.13. Given that tree-based methods also show a short runtime and high interpretability, we recommend employing them for toxicity prediction.
Read the full preprint for free at Research Square
Lisa-Marie Rolli is a PhD student working on the RADAR project, this work describes the background she will use to develop a computational model for the prediction of toxicity of chemicals and if necessary redesign of these chemicals.
Benchmarking Machine Learning for Toxicity Prediction
This study benchmarked 100 combinations of machine learning models, molecular representations, and dimensionality reduction methods across 52 ToxCast hormone receptor assays to identify the most effective approaches for toxicity prediction. Contrary to expectations based on previous deep learning successes, random forest models trained on physicochemical descriptors without dimensionality reduction consistently performed best, achieving the highest average ranking and predictive performance (average MCC = 0.207). Given their strong accuracy, short runtime, and high interpretability, the authors recommend tree-based methods, particularly random forests, as a preferred approach for toxicity prediction in hazard assessment.
Link to the RADAR project:
Lisa-Marie Rolli is a PhD student working on the RADAR project, this work describes the background she will use to develop a computational model for the prediction of toxicity of chemicals and if necessary redesign of these chemicals.
Summary text
