First-Person Household Video, Brazil (3.6 h starter set)

# First-Person Household Video, Brazil Head-mounted phone video of people doing household tasks in real homes in Brazil, with the wearer's hands in frame. ## At a glance | | household | |---|---| | Recordings | 25 | | Episodes | 82 | | Hours | 3.58 | | People | 11 | | Tasks | 10 | ## Capture - Phone on a head mount, ultra-wide lens requested, landscape. 1920x1080 for every episode. - 10 phone models: Apple iPhone 12 Pro, 12 Pro Max, 13, 14, 14 Pro Max, 16; Samsung SM-G991B, SM-G996B, SM-S901E, SM-S911B. - Every episode file carries the original audio track of the recording. - Environments declared by the person recording: kitchen, bedroom, laundry area, living room, yard or balcony, salon, workshop. ## Tasks Personal care (nails, skin care, make-up), washing dishes, sewing, washing and hanging laundry, tidying the kitchen, cooking, folding clothes, hand-washing clothes and shoes, crafts, cleaning a work area. ## Hands Measured on one frame per second of everything delivered. - At least one hand visible 97.0% of the time, both hands 79.6%. Every recording has one or more hands in at least 90% of its time. - Per episode: `hands_any_pct` and `hands_two_pct` in the episode manifest (`household.lerobot/package/metadata/episodes.jsonl`). ## Annotations - Task label per recording. - Task steps: 1,100 steps in 81 of 82 episodes, each with start, end, verb, object, acting hand and one sentence in English and in Portuguese. AI-generated and marked `annotation_source: ai_generated`; 55 validated by people (`human_validated: true`). - Camera intrinsics for all 25 recordings: 16 self-calibrated, 9 from the lens metadata (`intrinsics_origin`). None measured with a calibration target. - Short stretches where the hands leave the frame are kept in the video and listed in `flagged_segments` (2.3% of the time). ## Formats and folder structure ``` household.lerobot/ <h264 root>/ LeRobotDataset v3 root: meta/, data/, videos/ (82 episodes, 82 video files) derived/ frame maps of re-sampled episodes package/ manifests, QA report, consent ledger, terms, Croissant card index.json episode_index to episode_id, byte copy or derived ``` - The LeRobot root is the only folder in `household.lerobot/` whose name ends in `-h264`; its name gives resolution, frame rate and codec. - One LeRobot v3 root, 82 episodes in 82 video files. Camera key `observation.images.egocentric`. Tables carry `timestamp`, `frame_index`, `episode_index`, `index`, `task_index`. This is human video: no `action` or `observation.state` column. - Every episode is H.264 at exactly 30 fps. 59 episodes are byte copies of the phone file; 23 recorded at a variable or non-integer rate were resampled to 30 fps and re-encoded in H.264 (CRF 18, audio copied). `index.json` marks each one and `derived/<episode_id>.frames.csv.gz` maps every frame to its source frame. - Episode length from 15.8 s to 180 s, median about 3 minutes. - `package/` describes the full native package, which has one more episode (`V-78C5F03FFBD2-02`) that is not in this edition. - Croissant 1.1 card with the Responsible AI extension in `household.lerobot/package/croissant.json`. ## Loading ```python from pathlib import Path from lerobot.datasets import LeRobotDataset root = next(Path("household.lerobot").glob("*-h264")) ds = LeRobotDataset("local/household", root=root) ``` ## Not included No IMU, depth, camera pose or 3D hand pose. No robot actions. ## Rights and consent - First-party collection. Every person is an adult who accepted written participation terms covering recording, AI training, commercial licensing and sublicensing. - Each recording links to the terms version and the SHA-256 of the accepted text in `household.lerobot/package/consent_ledger.csv`; the terms text is in `household.lerobot/package/terms/`, with an English courtesy translation. - Before delivery a person watched every recording to the end, with sound, and confirmed there is no other person's face, reflection, document, screen with personal data or spoken personal data. - Participant ids are pseudonymous and valid only inside this dataset. ## License Kinetic Blocks Commercial Standard licence, non-exclusive. The Kinetic Blocks Data Licence Agreement governs. ## Available per order This is a starter set from our ongoing collection in Brazil. More hours, other tasks and environments, and a specific device mix are available as a commission, recorded to your specification.