Project Risk Assessment of a Multi-Agent AI Platform: Measured Failure Rates and Contingency

Abstract

Artificial Intelligence (AI) software is entering construction project delivery without the failure-rate evidence a project risk assessment requires, because suppliers publish none. This paper carries a five-agent AI platform through an ordinary construction risk assessment and obtains the missing evidence by measurement. The case is a highway bridge rehabilitation program of USD 180 million across twenty-four structures. Sixteen risks are identified across the agents and the interfaces between them, organized by the four functions of the National Institute of Standards and Technology (NIST) AI Risk Management Framework, and scored on the probability and impact scales the program already operates. Where a failure mode admits an objective ground truth, its probability is measured rather than judged. An evaluation harness was built for the specification compliance agent from provisions of 29 CFR 1926, and 288 trials were run across 72 items, five designed failure modes, two grounding conditions and two model configurations. Determination accuracy for the smaller model rose from 69.4 percent ungrounded to 93.1 percent grounded, significant on a Fisher exact test at p equal to 0.0005, while the change for the larger model was not significant. Error concentrated in one mode: where a general provision is displaced by a more specific subpart, the smaller model was correct in only 26.7 percent of ungrounded trials, and its wrong answers cited real but inapplicable provisions. Observed rates were mapped onto the probability scale, so five register cells record observations rather than judgments. Simulation returns inherent exposure of USD 7.91 million at the eightieth percentile, 4.39 percent of contract value, falling to USD 1.43 million after treatment. Four of the five measured risks proved the smallest contributors.