Human-Machine Collaboration in Technical Drawing Analysis

Paper in proceeding, 2025

Interpreting complex engineering drawings requires substantial manual effort in industrial workflows, with engineers spending hundreds of hours verifying correlations between visual elements and structured specifications across thousands of documents. This challenge is particularly acute in the Engineering, Procurement and Construction (EPC) industries, where interpretation errors are propagated into costly procurement and construction mistakes. We propose a cybernetic systems approach using Large Multimodal Models (LMMs) as cognitive partners in engineering documentation workflows, enhancing human capabilities through intelligent assistance in verification and information extraction tasks. To validate this solution, we systematically evaluated five LMMs on 60 piping isometric drawings with varying template structures, measuring both optical character recognition accuracy and correlation capabilities between drawings and their associated Bills of Materials. The results demonstrated significant performance variation between systems, with Claude 3.5 Sonnet and GPT-4o achieving greater accuracy 70%, while open source alternatives faced challenges with complex layouts and ambiguous visual information. These findings establish benchmarks for humanmachine collaboration in engineering documentation processing and provide a framework for integrating intelligent systems into technical workflows across multiple industrial domains.