Prediction-driven adaptive double booking strategy for outpatient scheduling: a multi-objective reinforcement learning approach

다국어 초록 (Multilingual Abstract)

Patient no-shows disrupt clinic operations, reduce productivity, and can delay
necessary medical care. Overbooking and double-booking are common strategies to
reduce the impact of no-shows, but poor use of these strategies may lead to
overcrowding and longer waiting times. Many existing approaches rely on static rules
that fail to adapt real-time scheduling dynamics or individual no-show risks. To
address this limitation, this study develops an adaptive double-booking strategy for
outpatient scheduling using a multi-objective reinforcement learning (RL) integrated
with individualized no-show prediction. We formulate the scheduling problem as a
Markov Decision Process. Patient-specific no-show probabilities generated by a
Multi-Head Attention Soft Random Forest (MHASRF) model are incorporated directly
into the decision process. A Multi-Policy Proximal Policy Optimization (MPPPO)
algorithm with a Multi-Policy Co-evolution Mechanism (MPCEM) is implemented. A
KL-based τ mechanism in MPCEM enables selective knowledge sharing among
behaviorally similar policies, improving convergence and trade-off diversity.
knowledge-sharing mechanism to support training and use SHAP to explain the agent’s
behavior. SHAP analysis further used to interpret the RL decisions. Overall, the
proposed framework provides a dynamic, data-driven scheduling system that learns
when to single-book, double-book, or reject appointments, offering more efficient and
reliable alternative to traditional scheduling approaches in outpatient care.