CircuitSense: Benchmarking Visual Circuit Understanding Through Symbolic Equation Derivation

Abstract

Engineering design requires translating visual representations into mathematical models across hierarchical levels. While Multi-modal Large Language Models (MLLMs) excel at natural image tasks, their ability to extract equations from technical diagrams remains untested. We present CircuitSense, a benchmark of 8,006+ problems evaluating circuit understanding across three tasks—Perception, Analysis, and Design—with emphasis on deriving symbolic equations from visual inputs. Additionally, we propose a hierarchical synthetic pipeline that generates schematics and block diagrams with guaranteed ground-truth equations. Evaluating six state-of-the-art MLLMs reveals a critical gap: models achieve 85%+ accuracy on component recognition but fall below 19% on equation derivation. This performance collapse exposes the fundamental barrier to engineering AI. The correlation between equation derivation capability and design task performance confirms that mathematical understanding, not pattern recognition, defines engineering competence.

Model Performance Across Tasks

Performance comparison of six MLLMs across Perception, Analysis, and Design tasks. The chart reveals strong perception capabilities but catastrophic failure in mathematical analysis tasks.

Key Findings

Perception task results

Model	Component Detec. (%)	Connection Ident. (%)	Function Class. (%)
GPT-4o	100	94	95
Gemini-2.5-Pro	100	100	95
Claude-Sonnet-4	100	88	86
InternVL3-72B	95	76	12
Qwen2.5-VL	95	68	20
GLM-4.5V	100	78	26

Design task results

Model	Schematic- level (%)	Block- level (%)	Hierarchical- Design (%)
GPT-4o	10.52	36.36	18.92
Gemini-2.5-Pro	36.38	67.27	51.35
Claude-Sonnet-4	17.54	51.83	29.83
InternVL3-72B	7.01	52.73	29.73
Qwen2.5-VL	8.76	30.91	18.92
GLM-4.5V	15.79	50.91	32.35

Curated Analysis Task Results

Results of curated problems for Analysis task with multiple choice and open-ended format.

Model	Level 0 (Resistor)	Level 1 (RLC)	Level 2 (Small Signal)	Level 3 (Transistor)	Level 4 (Block)	Overall Accuracy
Multiple Choice Format (%)
GPT-4o	39.80	49.58	32.88	48.80	39.58	45.07
Claude-Sonnet-4	66.72	71.22	61.64	72.01	66.67	69.67
Gemini-2.5-Pro	74.04	87.39	78.08	81.72	89.58	80.71
InternVL3-78B	23.16	20.59	13.70	13.11	14.58	18.06
Qwen2.5-VL-72b-instruct	29.53	41.60	30.14	35.94	29.17	34.90
GLM-4.5V	24.63	29.20	9.59	17.28	31.25	22.42
Open-ended format (%)
GPT-4o	29.59	29.83	19.18	13.96	17.81	22.84
Claude-Sonnet-4	35.56	50.21	12.33	27.04	33.33	34.76
Gemini-2.5-Pro	76.98	84.87	73.97	55.85	72.92	70.32
InternVL3-78B	20.79	19.54	6.85	14.47	10.42	17.26
Qwen2.5-VL-72B-Instruct	28.73	31.30	16.44	13.71	22.92	22.85
GLM-4.5V	34.44	39.71	13.70	19.50	25.00	28.83

Synthetic Problems Performance

Performance comparison on our hierarchical synthetic problems with symbolic equation ground truth.

Model	Level 0 (Resistor)	Level 1 (RLC)	Level 2 (Small Signal)	Level 4 (Block)	Level 5 (System)	Overall
GPT-4o	1.50	3.33	5.80	7.33	9.65	4.98
Claude-Sonnet-4	2.83	5.16	5.80	11.64	7.89	6.29
Gemini-2.5-Pro	3.49	11.67	38.00	12.33	35.96	19.06
InternVL3-78B	1.50	3.67	6.68	3.72	0.44	3.50
Qwen2.5-VL-72B-Instruct	0.83	4.17	6.03	6.64	10.09	4.96
GLM-4.5V	0.33	7.33	4.00	4.50	5.70	4.09

Benchmark Examples

Task: Analysis-synthetic Level: 5 (System)

Question: What is the Transfer function of the provided block diagram?

Ground Truth: $$\boxed{\frac{C(s)}{R(s)} = \frac{(\frac{-10}{(s^2+2s+1)}) }{ (1 - \frac{50}{(s^2+2s+1).(s+2))}}}$$

Task: Analysis-synthetic Level: 4 (Block)

Question: Derive the nodal equation for node 3 in the s-domain. Express the equation using only the circuit elements and their values as labeled in the diagram. Make sure the final answer is just the symbolic equation Vn3(s) = ..., where the right side contains only the labeled components and sources from the circuit diagram.

Ground Truth: $$ \boxed{Vn3(s) = \frac{R1*V1}{(s*(C1* (R4*(L2*s^2 + R1*s + s) + R1) + L2*s+R1+R4))}}$$

Task: Analysis-synthetic Level: 2 (Small Signal)

Question: What is the transfer function from V1 to R2 in this circuit?

Ground Truth: $$ \boxed{H(s) = (s/(s + 1/(C1*R2)))*1}$$

Task: Analysis-curated Level: 3 (Transistor)

Question: The simplified schematic of a feedback amplifier is shown. Use the method of feedback analysis to find $R_{\text {out }}=V_2 / I_2$. Assume that all transistors are matched and that $g_m=1 \mathrm{~mA} / \mathrm{V}$ and $r_{d s}= \infty$.

Ground Truth: $$50 \mathrm{k} \Omega$$

Task: Analysis-curated Level: 3 (Transistor)

Question: For the common-drain amplifier of the provided circuit image, assume $W / L=10$ and $\lambda=0$. Find the dc output voltage $V_o$ while including the body effect and with $R \rightarrow \infty$.

Ground Truth: $$1.5 \mathrm{~V}$$

BibTeX

@misc{akbari2025circuitsensehierarchicalcircuitbenchmark,
      title={CircuitSense: A Hierarchical Circuit System Benchmark Bridging Visual Comprehension and Symbolic Reasoning in Engineering Design Process}, 
      author={Arman Akbari and Jian Gao and Yifei Zou and Mei Yang and Jinru Duan and Dmitrii Torbunov and Yanzhi Wang and Yihui Ren and Xuan Zhang},
      year={2025},
      eprint={2509.22339},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2509.22339}, 
}

CircuitSense A Hierarchical Circuit System Benchmark Bridging Visual Comprehension and Symbolic Reasoning in Engineering Design Process