Leveraging multi-LLM orchestration for automated risk identification in supply chains

Abstract

Plant engineering supply chains, characterized by complex, project-specific networks, are increasingly exposed to disruptions such as shortages, delays, and geopolitical events. Traditional supply chain risk management (SCRM), often based on historical data, struggles to address such dynamic risks. This paper explores the use of large language models (LLMs) for automated risk identification through news analysis. A modular infrastructure integrated 24 open-source and proprietary LLMs under identical conditions. The methodology included (1) model orchestration for consistent data processing, (2) inter-model consistency analysis with Cohen's Kappa, and (3) aggregation strategies such as majority voting. Results show that ensembles outperform single models by reducing outliers, indicating uncertainty, and providing more robust classifications. Open-source ensembles achieved performance comparable to proprietary systems, suggesting effective SCRM is possible without costly commercial tools. The study demonstrates a scalable, reproducible approach to AI-based risk analysis. Future work will integrate company-specific data and validate the method in real-world plant engineering projects to support a more resilient and proactive management of construction supply chains.