The Age of Auto-Labeling is the Age of Automation

A paradigm shift for AI

Photo by David Levêque

Conventionally, AI comes in the form of neural networks that require labels for their training data, which are created via human labour. Human labour, therefore, limits (bottlenecks, in fact) the computational size of neural networks and, therefore, their performance, i.e., their ability to instantiate intelligence.

Under the auto-labeling regime (which is a superset of the self-supervised learning regime), labels are applied to data automatically. (The means can vary quite widely.) The bottlenecks to neural network performance become data, compute, and ingenuity. These terms defined:

Data: e.g., video, human input to a steering wheel, actions taken by humans in a video game.

Compute: i.e., cycles of a GPU, a neural network accelerator chip, or, more rarely, a CPU.

Ingenuity: the ability of machine learning engineers and researchers to design neural networks.

In the research space, auto-labeling has produced such feats as AlphaStar and GPT-3.

In the commercial robotics space, the most interesting use of auto-labeling (to me, anyway) is in vehicular automation (i.e. self-driving cars and ADAS systems). However, in principle, the auto-labeling regime can and most likely will, eventually, spread to other kinds of robotics applications.

The impact of robots (powered by very well-designed neural nets utilizing large amounts of compute and auto-labeled data) will be much larger than the impact of disembodied AIs (like AlphaStar and GPT-3) because the physical economy is much larger than the digital economy. Most of human activity still involves carrying out physical tasks, which is something only robots and not disembodied AIs can do.

The age of auto-labeling will also be the age of robotics automation. A robotics age has been frequently imagined and depicted, almost enough to make one forget that a real one has never yet transpired. We don’t really know what it will be like, except the broad contours, like that its impact on human lives and the human economy will be very big.

Knowing that, however, is news enough to be worth writing about.