The frontier of physical AI is a Jenga game in a warehouse in San Leandro, California, where Encord is testing a headset that measures brain waves while a pilot carefully pulls wooden blocks. The headset, built by German neuroscience startup Zander Labs, tracks brain activity to deduce mental states like error, intent, and surprise, aiming to produce more useful training data for robots. Encord is betting that the real constraint on humanoid and warehouse robotics is not model architecture but the scarcity of real-world physical training data. Instead of just managing existing data, Encord is manufacturing the data companies lack.

Vineeth Velmurugan, Encord‘s head of robot learning and a veteran of OpenAI’s robotics lab and Berkshire Grey, says the data simply does not exist. He estimates that a data set five times the size of YouTube’s video corpus is needed to break through the robotics data bottleneck. That scale turns data generation into a business. Encord was founded to help companies building machine-vision applications annotate data and evaluate models. As their customers began applying end-to-end learning to robotic manipulation, executives realized they would have to produce training data themselves.

Encord already collects egocentric video from workers wearing cameras in several factories worldwide. At its San Leandro facility, it experiments with new modalities. Pilots use leader-follower rigs—paired robotic arms, one controlled directly by a human and one that mimics—to create data for tasks like pouring coffee (very sloshy) and stacking poker chips. “Every humanoid company has asked us for these pieces,” Velmurugan says. Storage racks hold cartons of fake flowers, plastic vegetables, kitty litter trays, and bags of wires for training household manipulation. Another pilot works on plugging ethernet cables, highlighting the difficulty of precise manipulation with robotic pincers.

Encord is also developing a forearm sensor that detects electrical signals in muscles to build a 3D depiction of hand position, providing richer training data than video alone. Data sets are annotated with physical descriptions like “right hand tightens bolt” to aid LLM-based models. Velmurugan estimates dense annotation is worth 100 times as much as “junky ego data” for specific tasks, but costs 20 times more to produce. That cost is the catch: scraping text from the internet cost frontier labs almost nothing, but generating physical training data is expensive and changes the economics of building models.

The brain wave work with Zander Labs is a trial run. Encord will build an initial brain wave-tagged data set, run it through customer robotics models, and evaluate whether it improves performance before deciding to scale. Lucas Gehrke, a Zander neuroscientist, says the amount of brain activity during a task offers clues for when to deploy the highest-effort models. Velmurugan calls this the “bleeding edge” of solving the robotics data bottleneck.

Velmurugan says progress is being made across the industry, and Encord‘s visibility into many robotics programs gives it a vantage point to spot which data techniques gain traction. That position keeps the pilots busy, including former Scale AI employees. The Jenga tower eventually topples, but the pilot, Andrew Ceja, says he enjoys the challenge of solving training tasks for robots.