Mohamad Dunggio recently strapped a mobile phone to his head and recorded himself doing his dishes over and over again for four hours.
asserted
himself → strap → hours
It left him with wrinkly fingers, but he did not mind.
asserted
he → leave → fingers
"I think it's OK because I exchange my time to earn money," said the 23-year-old from Gorontalo on Sulawesi in Indonesia.
asserted
old → think → Indonesia
Mr Dunggio is one of the thousands of gig workers around the world feeding the robotics industry's insatiable demand for training data.
asserted
Dunggio → feed → data
Since April he has been earning extra cash making "egocentric training" videos, used to teach robots how to do things like household chores.
asserted
he → earn → chores
Egocentric training data is sensor or video recordings made from the perspective of the person doing the task, often captured with a head or chest-mounted camera with a focus on keeping the subject's hands in the frame.
asserted
data → make → frame
Mr Dunggio said he was paid by three companies from Argentina, India and the US between $US3 ($4.22) and $US10 an hour to record himself performing tasks like dishwashing, cleaning, cooking, making the bed, ironing, folding clothes and sweeping.
asserted
himself → say → clothes
"I think it's a good thing to do because you don't have to have any experience," he said.
asserted
he → think → experience
As the recent World Humanoid Robot Games in China demonstrated, robots have been advancing in literal leaps and bounds in recent years.
asserted
robots → demonstrate → years
But the dream of a multi-function humanoid robot that can move around the home efficiently and reliably taking on a variety of domestic tasks has so far proven elusive.
asserted
that → move → tasks
Dana Kulić, a robotics professor at Monash University, said that for robots to navigate the huge variety and complexity of people's homes they needed to learn how to observe and react to their environments.
asserted
they → say → environments
"Even if the task is just to fold the laundry, or wash the dishes, every person has a different set of dishes.
asserted
person → fold → dishes
Then the place where [the clothes] should be folded looks different," she said.
asserted
she → fold → ?
Professor Kulić said robotics researchers were using egocentric data in a way similar to how generative AI researchers taught large language models (LLMs) like ChatGPT to predict the next word or letter in a sentence.
asserted
researchers → say → sentence
While egocentric data produced by actual robots, teleoperated by a human, tended to be more relevant, human egocentric data was cheaper, easier and could provide a much-needed variety of scenarios, she said.
uncertain
she → produce → scenarios
The workers producing the data now include entire factories in South Asia, house cleaners in the US and home delivery drivers.
asserted
workers → produce → US
However, Professor Kulić said the ways human egocentric data was collected were not without issues.
asserted
data → say → issues
They ranged from whether the contributors truly understood what the data was being used for to privacy concerns about filming in people's homes and data being sold on to third companies.
asserted
data → range → companies
Strict requirements for videos
Mr Dunggio said overall he was happy with the side-hustle and did not feel exploited, even if others might feel differently.
"If people don't have work and are looking for income this is just an option," he said.
uncertain
he → say → income
However, he said he felt a bit embarrassed going to the mall with a phone mounted on his head and was disappointed when some of his videos were rejected.
asserted
some → say → videos
He also complained that some companies charged a transfer fee of about $US1 to withdraw accumulated earnings of under $US10.
asserted
companies → complain → US10
On the plus side he got to learn some new skills.
asserted
he → get → skills
"I never cooked for myself before, but now I know how to cook," he said.
asserted
he → cook → myself
Changes in demand for data
Sergio Bruccoleri is vice-president of delivery for Appen, an ASX-listed Australian company that collects and annotates datasets for the tech industry, including egocentric data for robotics.
asserted
that → list → robotics
Mr Bruccoleri, who is based in Barcelona, said the company could collect around 60,000 hours of video per month from as many as 5,000 contributors all over the world, including India, Canada, the US, Brazil, Mexico and the Philippines.
uncertain
company → base → India
China was Appen's biggest market for robotics data with the US second, he said.
asserted
he → say → data
He said Appen's contributors produced videos in homes and increasingly in workplaces but also in controlled studios, sometimes with specialised cameras and equipment like sensor gloves.
asserted
contributors → say → gloves
However, he said Appen was moving away from the kind of data that was being collected at an industrial scale in "lower-cost countries" because it had become commoditised and de-valued.
asserted
it → say → countries
The company was increasingly focused on wealthier countries like the US and Canada, where the robots were likely to be sold initially, while still gathering high value data elsewhere.
asserted
robots → focus → data
"The dishwashers in the US look very different from the dishwashers you would have elsewhere," he said.
asserted
he → look → dishwashers
In the past, Appen has faced criticism over pay and conditions.
asserted
Appen → face → pay
Contributors on forums still complain of issues from late payments to poor communication and unexplained account cancellations.
asserted
Contributors → complain → communication
Mr Bruccoleri said some of Appen's systems were not perfect and the company tried to address contributor issues with open lines of communication.
asserted
company → say → communication
He said Appen had a code of ethics and fair-pay standards and tried "as much as possible" to ensure pay was "always above the minimum wage set in that particular country".
asserted
pay → say → country
Appen also had privacy guidelines for contributors and a tracing system which he said meant they should be able to request the data they produced be withdrawn, even after it had been sold.
asserted
it → have → contributors
Asked whether it was unfair that contributors were sometimes forced to pay fees to withdraw their earnings, Mr Bruccoleri said he believed it was but it was the payment platforms that took the fee.
asserted
that → ask → fee
"It's not limited to Appen.
asserted
It → limit → Appen
You see it in other platforms all the time," he said.
asserted
he → see → platforms
When can I have a robot to fold my clothes?
asserted
I → have → clothes
Robotics companies, particularly those in China, have this year been unveiling more and more competent humanoid models that appear close to the ideal of a Jetsons-style robot to do all our household chores.
asserted
that → unveil → chores
…and 5 more, not listed.
The Aporia
