Embodied representation: the environment is more than input
GBB Embodied Interaction AI Institute2026.10 · Research agenda
Research question
Moving an object onto a table requires more than recognizing its name. Reachable areas, contact relationships and motion constraints all matter. Without these conditions, even correct recognition can produce an impossible plan.
Research method
Represent observations, goals, spatial relationships and preconditions around task-relevant states. Preserve the source and time of every update. Keep inference separate from observation so unconfirmed judgments do not become the basis for subsequent action.
Validation approach
Build comparison scenarios with position changes, occlusion and ambiguous instructions.
Compare object lists and relational representations in task reasoning.
Record whether errors originate in observation, state updates or missing constraints.
Express the same task as an object list and a relationship graph. The list records categories and coordinates; the graph also captures support, obstruction, contact and reachability. Keep goals and environments identical and compare omitted constraints in the resulting plans.
Experimental design & observations
Stage
What to record
What to check
Tabletop manipulation
Object pose, grasp region, supporting surface
Is the goal reachable, and are all action preconditions present?
Occlusion recovery
Last observation time, occlusion source, confidence range
Does the system mistake a past position for the current state?
Goal change
New goal, executed actions, remaining space
Can the system explain which plans are no longer valid?
Making results comparable
Fix the initial map, endpoints and input version, then change one environmental condition at a time. Record the baseline before introducing obstacles or conflicting inputs. Include successful paths and failure cases. Compare strategies using path length, explored cells and execution traces.
Reproducibility checklist
Input and map snapshots: record scene versions, task conditions and random seeds.
Strategy configuration: record algorithms, constraints, timeouts and human intervention points.
Execution traces: retain state changes, action sequences, failure causes and recovery steps.
Comparison notes: document variable changes, observations and follow-up validation.
