Due to outstanding performance in cheminformatics

machine learning algorithms have been increasingly used to mine molecular properties and biomedical big data. The performance of machine learning models is known to critically depend on the selection of the hyper-parameter configuration. However

many studies either explored the optimal hyper-parameters per the grid searching method or employed arbitrarily selected hyper-parameters

which can easily lead to achieving a suboptimal hyper-parameter configuration. In this study

Hyperopt library embedding with the Bayesian optimization is employed to find optimal hyper-parameters for different machine learning algorithms. Six drug discovery datasets

including solubility

probe-likeness

hERG

Chagas disease

tuberculosis

and malaria

are used to compare different machine learning algorithms with ECFP6 fingerprints. This contribution aims to evaluate whether the Bernoulli Naïve Bayes

logistic linear regression

AdaBoost decision tree

random forest

support vector machine

and deep neural networks algorithms with optimized hyper-parameters can offer any improvement in testing as compared with the referenced models assessed by an array of metrics including AUC

F1-score

Cohen’s kappa

Matthews correlation coefficient

recall

precision

and accuracy. Based on the rank normalized score approach

the Hyperopt models achieve better or comparable performance on 33 out 36 models for different drug discovery datasets

showing significant improvement achieved by employing the Hyperopt library. The open-source code of all the 6 machine learning frameworks employed in the Hyperopt python package is provided to make this approach accessible to more scientists

who are not familiar with writing code.