Sensor fusion
417 words · 3 min read · 2 sources
Sensor fusion is the process of combining data from multiple sensors — cameras, lidar, radar, IMU — into a single, more reliable picture of the world than any one sensor could produce alone.
A doctor does not diagnose a patient using a single test. They combine a blood test, a physical examination, an X-ray, and the patient's own description of their symptoms. Each source has gaps; each source has strengths. Together they build a picture no single source could. Roboticists discovered the same principle is essential for machines that must underst
A doctor does not diagnose a patient using a single test. They combine a blood test, a physical examination, an X-ray, and the patient's own description of their symptoms. Each source has gaps; each source has strengths. Together they build a picture no single source could. Roboticists discovered the same principle is essential for machines that must understand the physical world.
Sensor fusion is the technique of combining raw data from multiple sensors — cameras, lidar, radar, inertial measurement units (IMUs), GPS, microphones — into a unified, more accurate and more reliable representation of the robot's environment than any individual sensor could produce on its own.
Why no single sensor is enough
Every sensor has characteristic weaknesses:
- Cameras produce rich colour and texture but struggle in darkness, glare, and heavy rain, and they cannot directly measure distance.
- Lidar measures distance precisely and works in the dark, but cannot read colour or text and struggles with glass and mirrors.
- Radar works in any weather and measures relative velocity accurately, but its resolution is low and it cannot distinguish object shapes well.
- IMUs (gyroscopes and accelerometers) measure motion with no external reference, but their errors accumulate over time — a problem called drift.
Sensor fusion lets each sensor cover the blind spots of the others.
How fusion works
The most famous fusion algorithm is the Kalman filter, developed in 1960 and still used in almost every autonomous vehicle on the road. It maintains a probabilistic estimate of the robot's state (position, velocity, orientation) and updates that estimate every time a new sensor reading arrives, weighting each reading by how trustworthy the sensor is in the current conditions.
A real example
A self-driving car approaching a busy junction uses sensor fusion to handle a scenario no single sensor could manage: a pedestrian in a dark raincoat stepping off the kerb at night. The camera sees almost nothing. The lidar sees a shape but the rain creates noise. The radar sees a moving object clearly but cannot tell it is human. Fused together, the system correctly identifies a pedestrian and brakes.
Why robots cannot skip it
A robot acting on a single sensor is brittle. Sensor fusion is what turns individual unreliable data streams into the confident situational awareness a robot needs to operate safely in the real world.
The Kalman filter was originally developed for NASA's Apollo navigation computer in the 1960s — the same algorithm now helps electric scooters know where they are on city streets.
Next: one minute
Watch how Sensor fusion actually works
A one-minute video, an interactive demo you can poke at, and one question to check it landed.
Watch the one-minute lesson →Plain English. Free. Unsubscribe in one click.
Learn this in the Academy
🔌W-02: Sensor Integration
Hands-on lesson · Wire track
Keep going
Complementary Filter
A complementary filter blends two sensors that fail in opposite ways: it trusts a gyroscope for fast changes a…
ConceptCamera Calibration
Camera calibration measures a camera's exact lens and geometry so a robot can turn pixels into real directions…
ConceptPoint cloud
A point cloud is a collection of millions of individual 3D coordinates, each one marking a surface that a lase…
Last updated · 2026-05-19
Spotted something off? Report an error →