Sow Smarter, Not Harder: Evaluating LLM-Generated Seeds for Fuzzing Critical Infrastructure

Abstract

Software vulnerabilities in critical infrastructure components can lead to severe disruptions. While fuzzing effectively identifies such weaknesses, the quality of initial seed inputs significantly impacts its effectiveness. This study evaluates how large language models (LLMs) can generate better fuzzing seeds for critical infrastructure software. We compared seven LLMs—ChatGPT-4-Turbo, Claude 3.0 Opus, Claude 3.7 Sonnet, DeepSeek-V3, Gemini 2.0 Flash, Grok 3, and Mistral 7B—with manual baselines across six programs, including industrial control libraries, routing components, and network firmware. Over 20 independent 24-h campaigns per model and program, LLM-generated seeds achieved 14.8% higher code coverage, detected 56.3% more unique crashes, and reached first crashes 373.9% faster than manual methods. Performance patterns emerged across different infrastructure protocols, with certain models excelling at complex SCADA data formats while others performed better for network security components. The 56.5% computational efficiency improvement benefits resource-constrained operational technology environments. These findings demonstrate that LLM-generated seeds can meaningfully enhance vulnerability detection in software underlying critical infrastructure systems, offering a practical approach to strengthening resilience against cyber threats .