Large Language Models and Robots: What They Can and Cannot Do

Short answer: a large language model (LLM) can turn a plain instruction like "bring me a drink from the kitchen" into a list of steps, but it does not control motors or know the robot's physical limits. Another layer must turn the steps into safe actions. See LLMs in robotics and foundation models.

What LLMs are good at

• Understanding requests in everyday language.
• Breaking a goal into steps: open fridge, grab bottle, close door.
• Explaining what the robot is doing, which helps people trust it.

What they are bad at

• Knowing physics. An LLM may suggest an action the robot cannot reach or lift.
• Precision. Moving an arm to millimetre accuracy is the job of control software, not language.
• Being reliable every time. Models can produce a plausible but wrong plan.

How the pieces fit

1. The LLM proposes steps in words.
2. A set of skills (pick, place, navigate) the robot already knows are matched to the steps.
3. A safety layer checks each action before it runs.
4. Classic control executes the motion.

Without the safety layer

If an LLM says "pour the kettle" while a person's hand is in the way, nothing in the language model knows to stop. Hard limits and sensors must be outside the model.

What is changing

Newer research trains models that take camera images and instructions together and output robot actions directly. They are promising, but still being tested for reliability in varied settings.

FAQ

Can I connect ChatGPT-style models to my Arduino? You can use one to produce commands, but check them in code before sending anything to motors.

Will this replace programming robots? It changes how high-level tasks are described. Low-level control remains classic engineering.

Overview: AI and robotics, the complete guide.