ITBench-AA: Frontier Models Score Below 50% on the First Benchmark for Agentic Enterprise IT Tasks — by Artificial Analysis and IBM

ITBench-AA evaluates AI models on agentic enterprise IT tasks, with all scoring below 50% on SRE benchmarks.

• •59 SRE tasks test Kubernetes incident diagnosis using logs, traces, and topology
• •Claude Opus 4.7 leads at 47%, GPT-5.5 at 46%, Qwen3.7 Max at 42%
• •Models are penalized for false positives: extra entities beyond root causes reduce precision
• •Gemma 4 31B scores 37% at $0.14, outperforming Gemini 3.1 Pro Preview
• •83-turn vs 58-turn: 30% vs 37%, more investigation doesn't improve results

This summary was automatically generated by AI based on the original article and may not be fully accurate.