TL;DR
- Computer vision is software that learns to read images. It learns this by seeing thousands of examples on which people have marked what is visible.
- In waste collection it does three things: find objects, name them and outline materials that lie on top of each other.
- The system gets better with every confirmed detection, but it does not see through an opaque bag.
Computer vision is the part of artificial intelligence that reads images. Just as a driver recognises a piece of cardboard because he has seen thousands of them, software learns this in the same way: through examples. In this article we explain how that works on a refuse truck, without formulas.
A computer first sees only pixels
For a computer, a photo is a grid of coloured dots. It sees no box, no bottle, no bag. It sees thousands of numbers that represent colour and brightness.
To find meaning in that, it needs a model. That model learns links between patterns in those pixels and what they represent. A brown, flat shape with straight edges and fold lines is probably cardboard. A shiny, crumpled, translucent shape is more likely film.
Learning from examples
A model learns from examples. People look at images of real lifts and mark what they see: this is paper, this is PMD, this is a gas cylinder. This is called annotation. The model looks at all those examples and adjusts itself until it gives the right answers on images it has never seen.
That is why local knowledge makes such a difference. A Flemish PMD bag looks different from a German yellow bag. Commercial waste differs from household waste. The Rematics team combines specialists in computer vision, software and waste management to train models on the fractions collectors actually pick up.
Three tasks: find, name, outline
On a refuse truck, computer vision does three things at once.
- Detection. The model looks for where an object lies in the image and draws a box around it. That way it finds a gas cylinder or a battery among the rest.
- Classification. The model names what it has found. Is that box cardboard, film or wood?
- Segmentation. The model marks which material each part of the image is. This is needed when materials lie on top of each other, such as film among cardboard. You cannot count them as separate objects, but you can measure how much of the image is non-conforming.
D4T combines these three tasks and so recognises more than 40 waste fractions, non-conformities and hazardous materials.
Why it happens on the truck itself
Computer vision requires a lot of computing power. That used to sit in a server room. D4T calculates on a rugged AI computer in the refuse truck itself. This is called edge AI. The truck does not need to send video, and the analysis keeps running even when the mobile connection drops for a moment. On the product page you can read which hardware this requires.
Better with every confirmed detection
Every detection in the Daisy Portal comes with the image. A customer can confirm or dispute a detection. Confirmed detections go back to the team, which uses them to train the models further. That way errors stay visible instead of being hidden in an average.
What computer vision cannot do
A camera does not see through an opaque bag. D4T can recognise and count such a bag, but cannot say what is inside. And whatever disappears completely under other materials remains invisible. That is why D4T analyses the whole lift at 40 images per second: the more moments you see, the less stays hidden. More on this in the frequently asked questions.
Curious how computer vision reads your waste streams? Request a demo and see real detections.