Problem
A global technology company needed a dataset of egocentric (first-person) videos, capturing a broad range of real-world activities across different languages, countries, and environments.
The data needed to combine a consistent first-person perspective with natural human activity, concurrent spoken narration and detailed contextual metadata. It also needed to represent variation in how everyday tasks are performed across homes, cultures, geographies, and individuals.
Quality was a priority. Poor camera positioning, motion blur, inadequate lighting, incomplete narration or false footage could significantly impact AI performance. On the flipside, contributors needed enough freedom to perform activities naturally rather than creating overly scripted or repetitive data.
Solution
DATAmundi designed and managed a global egocentric data acquisition program involving DATAtalent AI contributors across 16 countries and seven languages, including English, Spanish, Portuguese, Polish, French, Italian, and Turkish.
- Contributors were recruited across Europe, North and South America and other locations, providing a mix of cultures, living environments, and everyday experiences. Participants came from urban and rural locations, had varying levels of technical expertise and used a range of consumer devices.
- The collection was designed to capture everyday activities as they naturally happen from the participant’s point of view. Contributors recorded tasks in homes, kitchens, living rooms, offices, garages and outdoor settings.
- Their activities ranged from cooking, cleaning, laundry and household organization through to gardening, arts and crafts, and DIY tasks. Physical AI systems need to recognize more than individual objects.
- Contributors used head-mounted smartphones and in some cases, action cameras, smart glasses and other wearable devices. Client guidelines established requirements around resolution, orientation, camera perspective, lighting, stability and continuous capture.
An important layer was the concurrent narration.
While completing each activity, participants described the actions they were performing, the objects they were interacting with and, where relevant, their intentions and the progression of the task. The recorded speech was transcribed, creating video, audio and text data that could support multimodal model development and evaluation.
Each recording was enriched with structured metadata covering factors such as language, geography, activity category, device and mounting configuration, recording duration and quality status. Data, transcripts, metadata and associated project files were securely managed through AIDA Hub, DATAmundi’s data operations platform.
Sample Activity: Preparing an Omelet
Language: English
1
00:00:01,600 –> 00:00:03,360
I’m going to cook an omelet.
2
00:00:03,360 –> 00:00:06,400
And for that, I’m taking a pan.
3
00:00:07,320 –> 00:00:12,600
And I will need to take the eggs from the fridge.
4
00:00:12,640 –> 00:00:21,600
I’m taking the egg out of the container, and I like my eggs with butter and chives.
Sample Activity: Renting a Bike
Language: French
1
00:00:00,000 –> 00:00:08,660
Alors je vais louer un Bike pour aller en vélo dans un magasin.
2
00:00:10,500 –> 00:00:17,660
Donc je suis près de la station de Bike, je prends mon téléphone, j’ai l’application de Bike ouverte.
3
00:00:19,080 –> 00:00:25,400
Je choisis le vélo, c’est le vélo 4905, je le sélectionne.
4
00:00:28,660 –> 00:00:35,240
Je sélectionne, j’ai un abonnement, et oui, je veux louer le vélo.
High-Quality Across Collection
DATAmundi’s quality team reviewed recordings for image quality, camera perspective, stability, lighting, narration and adherence to the collection guidelines and to check there was a strong relationship between the actions and the narration.
Using DATAmundi’s AI-enabled transcription tool, transcripts were then human-validated against the source audio for completeness and accuracy.
Recordings that failed to meet the required standards – for example, because of third-person perspectives, extended silences, poor visuals, or weak narration – were rejected. DATAtalent contributors received targeted feedback and were asked to re-record or resubmit where appropriate.
Result
The program hundreds of quality-controlled multilingual egocentric video data, in seven languages and across 16 countries.
The resulting dataset combined first-person video, natural speech, aligned transcripts and structured metadata, to create a multimodal view of how people interact with the physical world.
For AI systems expected to make that leap and perceive, reason and ultimately operate in real-world environments, this type of data acquisition program provides an important bridge between understanding the digital world and understanding how humans actually act within the physical one.
If you want to discuss how to create and collect multimodal egocentric training data for your AI model
