Robust Predictable Control (RPC) for optimizing fed-batch penicillin production

Abstract

Biochemical processes, characterized by nonlinear dynamics and uncertainties, pose significant optimization challenges. This work explores Robust Predictable Control (RPC) as a Reinforcement Learning (RL) algorithm to enhance a fed-batch penicillin production process utilizing the simulation model IndPenSim. Unlike some RL implementations that constrain exploration based on prior knowledge, the selected RPC approach allows the RL agent to explore freely and identify optimal control strategies by itself. We trained the RL agent under disturbance-free conditions and evaluated its performance against various unseen initial process conditions and disturbances. Results show that RPC significantly outperforms other process control methods, including other RL implementations, achieving higher yields with fewer necessary measurements as input for the RL agent. Analyzing two reward functions - penicillin concentration and yield - revealed that using concentration in the reward function improved agent training for maximizing yield, highlighting the importance of reward design in RL. Additionally, the trained RL agent effectively adapted to different action intervals, demonstrating robustness in dynamic environments without retraining. Our findings underscore RPC's potential for optimizing biochemical processes, especially in scenarios with few measurements, paving the way for AI-driven control systems in industrial applications.