Improving de novo molecular design with curriculum learning

Journal article, 2022

While reinforcement learning can be a powerful tool for complex design tasks such as molecular design, training can be slow when problems are either too hard or too easy, as little is learned in these cases. Jeff Guo and colleagues provide a curriculum learning extension to the REINVENT de novo molecular design framework that provides problems of increasing difficulty over epochs such that the training process is more efficient. Reinforcement learning is a powerful paradigm that has gained popularity across multiple domains. However, applying reinforcement learning may come at the cost of multiple interactions between the agent and the environment. This cost can be especially pronounced when the single feedback from the environment is slow or computationally expensive, causing extensive periods of non-productivity. Curriculum learning provides a suitable alternative by arranging a sequence of tasks of increasing complexity, with the aim of reducing the overall cost of learning. Here we demonstrate the application of curriculum learning for drug discovery. We implement curriculum learning in the de novo design platform REINVENT, and apply it to illustrative molecular design problems of different complexities. The results show both accelerated learning and a positive impact on the quality of the output when compared with standard policy-based reinforcement learning.