SoftBank Automates SOC Triaging With Cisco's Open-Source AI Model

SoftBank Automates SOC Triaging With Cisco's Open-Source AI Model

SoftBank Corp. has automated a critical security operations workflow by deploying Cisco Foundation AI's open-source large language model on-premises, eliminating manual software categorization that previously consumed analyst time. The implementation demonstrates how organizations can achieve end-to-end automation in security operations while maintaining data privacy requirements.

Why it matters

Security teams face mounting alert volumes that strain analyst capacity. This deployment shows that specialized, compact AI models can automate complex categorization tasks without cloud dependencies or extensive retraining—a practical blueprint for enterprises balancing automation benefits against data sovereignty concerns.

The Workflow Challenge

SoftBank's Security Operations Center previously required analysts to manually categorize detected software, verify policies, and execute responses—a time-intensive process that diverted attention from high-priority investigations. While automation frameworks could handle policy checks and actions, software categorization remained stubbornly manual due to overlapping functionalities and organization-specific rules across thousands of potential applications.

The Foundation-sec-1.1-8B-Instruct model became the missing piece. SoftBank needed an on-premises solution due to data privacy requirements, ruling out cloud-based LLMs. The model's security-specific pre-training and compact 8-billion-parameter size offered operational cost advantages while outperforming general-purpose open-source alternatives on security tasks, according to details first reported by Cisco.

Achieving 90% Accuracy Through Prompt Engineering

The deployment required solving three technical challenges. First, output formatting: Cisco's team used few-shot examples in prompts to ensure the model returned only valid category names from the 17-option taxonomy, combined with validation loops that retry inference on malformed outputs.

Second, category disambiguation: The team embedded analyst logic into prompts to resolve overlaps. For instance, cloud storage like OneDrive should trigger "Forbidden Internet Service" policies rather than "File Sharing" despite sharing functionality. The prompt includes decision trees—"Does it output vulnerability reports? → Yes: Vulnerability Scanning"—to guide the model through ambiguous cases.

Third, handling edge cases: A catch-all "Undetermined" category exists for software outside the 16 defined types. The model initially over-assigned specific categories to avoid this label, creating false positives. SoftBank implemented preprocessing whitelists and postprocessing filters to catch organization-specific misclassifications.

The Foundation-sec-1.1-8B-Instruct model alone achieved 80.75% accuracy on a dataset of historical detections and manually verified labels—matching cloud LLM performance on the same task. Combined with rule-based systems and the new filtering steps, overall workflow accuracy reached 90%.

Beyond File Detection

SoftBank plans to extend this approach to intrusion detection system response automation. The model's on-premises deployment and security focus align with handling sensitive network data in these expanded workflows.

Hajime Uematsu, Director of Security Verification at SoftBank Corp., confirmed the model achieved over 85% accuracy at the workflow-action level during proof-of-value testing, with further improvement expected through preprocessing and policy controls.

These details were first reported by Cisco on the Automation Watch blog.