Experiment - Yutarop/turtlesim_agent GitHub Wiki

Experiment

To evaluate the drawing capabilities of the turtlesim_agent, we defined 10 levels of shape complexity. The table below presents how each LLM agent performs shape drawing via natural language prompts in turtlesim.

Level Shape gpt-4o-mini gemini-2.0-flash mistral claude-3-opus
1 Circle
1 Square
1 Triangle
2 Rectangle
2 Parallelogram
2 Pentagon
3 Ellipse
3 Star shape made of lines
3 Circle inside a triangle
4 Hexagon
4 Octagon
4 Decagon
5 Circle inside a triangle
5 Triangle inside a square
5 Square inside a circle
6 Cube (2D representation)
6 Cylinder (2D view)
6 Cone (2D view)
7 Star-shaped polygon
7 Overlapping circles
7 Combined triangles
8 Spiral
8 Wave shape
8 Part of a fractal
9 Irregular polygon
9 Complex curve combination
9 Geometric pattern
10 Torus (2D view)
10 Mandelbrot fractal section
10 Non-Euclidean geometric shape

⭕ = Successfully drawn
❌ = Failed or incorrect output
— = Not yet tested