Human interaction. Agent intelligence.
Android / Accessibility data
Mobile AI.
Human interaction data.
For mobile AI agent training.
Mobile AI provides consent-based Android interaction data for teams training and evaluating mobile agents. Explore real human sessions, accessibility trees, and event sequences, with task labels prepared separately.
- 2,000 apps observed per week
- App agnostic schema
01 / On the phone
Something worth your time.
02 / Structured accessibility log
- package
- ai.mobile.demo
- timestamp_ms
- 2714
- source.text
- "Form"
- resource_id
- ai.mobile.demo:id/
book_form - class_name
- android.widget.Button
- clickable
- true
MATCHED NODE / UI TREE
01 / The data
From what happened
to what it means.
Study the corrections and backtracking in real human sessions. Add structure and annotation to create the examples your agent needs.
Observed signals
Source data- Screen structure
- Node hierarchy, text, resource identifiers, bounds, and element states where captured.
- Interaction events
- Clicks, scrolling, text changes, and the order in which events occur.
- App and time context
- Package names, event timestamps, and associated screen context.
Derived structure
Prepared layers- Episodes and steps
- Events grouped into sequences and paired with available screen states.
- Task and outcome labels
- Added through inference or annotation. These are not raw accessibility fields.
- Evaluation suites
- Curated episodes with defined checkpoints and scoring criteria, scoped to your use case.
Field coverage and continuity depend on the app and capture. Sample exports establish the available signals; annotation and evaluation requirements are agreed separately.
02 / The format
A sequence.
A step.
A basis for evaluation.
Bring the data into your training or evaluation workflow, with source events and derived labels kept distinct.
Illustrative schema only. These examples are not production exports or real user data. Task type and outcome remain unset until annotated; an evaluation suite requires its own curation.
{
"episode_id": "illustrative_001",
"source": "android_accessibility",
"step_ids": ["step_01", "step_02"],
"derived_labels": {
"task_type": null,
"outcome": null,
"status": "not_annotated"
}
}
{
"step_id": "step_01",
"event": {
"type": "TYPE_VIEW_CLICKED",
"target_node": "result_a"
},
"state_ref": "tree_before",
"next_state_ref": "tree_after",
"alignment": "requires_validation"
}
{
"suite_id": "illustrative_suite",
"episode_ids": ["illustrative_001"],
"curation": {
"checkpoints": [],
"success_criteria": null,
"scoring_rules": null,
"status": "to_be_defined"
}
}
03 / Start a conversation
What are you
building for mobile?
Tell us about your agent, your data requirements, or the workflows you want to understand.
Sample review
Pilot scoping
Data partnerships