Abstract:
Ambushes, in the form of improvised explosive devices (IEDs), have posed grave risk to targeted vehicles operating on supply routes in recent theaters of war. History shows that this is an enduring problem that U.S. military forces will likely face again in the future. This research provides an underpinning argument and model demonstration of a previously unexplored approach to the attack prediction problem when conducting repetitive operations on a contested transportation network. The problem being addressed goes beyond the typical objective of maximizing IED detection and avoidance, or minimizing damage and delay. Rather the problem is re-framed to focus on using the defender's activities (that are being observed by the attacker) as a direct means to shape the attacker's expectations and therefore his attack choices. Thus, in contrast to most previous work, there is an explicit assumption of dependence between the defender's actions and the attacker's choices. Approximate dynamic programming (ADP) is applied in a reinforcement learning (RL) construct to determine convoy schedules and route clearance assignments in light of a responsive attacker. There are currently few analytical approaches for this problem in the literature, but RL algorithms offer opportunities for meaningful improvements by optimizing individual movements across an extended planning horizon, accounting for downstream attacker-defender interaction. Computational results show meaningful performance improvements over a one-step, myopic decision rule. Further, the decision policies that are discovered by the RL agent would be difficult for unaided human planners to duplicate.
