Studying how a robot can build, rearrange, and recover.
How can a robot build a structure, take it apart, and rebuild it when each move changes what it can reach?
A six-axis robot arm and gripper to simulate a system for planning, controlling, and evaluating construction tasks.
Findings related to reliable manipulation, better design, and improved test completion. Controlled experiments identified adjustments leading to task completion.
An example of a complete recorded program, shown with rendered and diagnostic camera views. This run used object-state observations; the RGB, depth, and point-cloud panels visualize the recorded scene.
This independent research project utilized tools across mechanical design, physics simulations, implementation, and analysis. It consists of an editable six-axis arm and gripper in Fusion, and a MuJoCo sim. It started with simple tasks of picking up objects, before moving into building bridge and tower structures. Each step improved motion, planning, and learning.
Connect the CAD design to a physics model, with joint limits, motor control, and objects held through simulated contact.
Check grasps and motion paths, choose temporary placements, and revisit the plan as the arrangement changes.
Train from demonstrations and interaction, then compare learned movement and action selection with simpler controllers.
Compare matched layouts, replay the recorded physics, and keep failures alongside successes to see what actually improved.
Each comparison uses its own matched task set, with motion, contact, and completion checks. Success rates are rounded to the nearest percent.
46% → 100%
Across 24 matched transfer trials, before and after motion-control and grasp-selection improvements.
6% → 63%
Across 16 matched tasks, after a combined controller, sensing, and gripper revision. Qualification includes motion and contact checks.
38% → 63%
Adding a learned planning cost improved completion across 8 matched conditions, drawn from six base scenes.
I redesigned the gripper in Fusion, moving its motor behind the fingers and reducing the tool’s width by about 23% while keeping the same opening. With the controller unchanged, construction cycle completion increased from 50% to 63% across eight programs from four new scenes, each tested with object-state and camera observations.
I added Mink to control the arm’s position and orientation while accounting for nearby objects. Across 15 matched simulated trials, Mink with collision limits achieved 60% transfer success, compared with 20% for the original controller.
Putting a block in its final position can obstruct the next grasp. I began treating temporary placement as part of construction, and testing whether each move preserved access for the rest of the sequence.
I compared alternative moves from identical saved states, then let the robot continue. That helped trace a failure to a placement choice, a control problem, or the planner itself.
A learned push became much more successful in isolation, but did not improve the construction programs. Skill accuracy, where the planner can use it, and the condition it leaves behind all matter.
More search and influence for a learned score needed to be assessed for regressions. Keeping simpler controls and failed attempts made those results useful for deciding what to investigate next.
A useful action should make the remaining work easier to complete. Construction Lab provides a practical way to simulate this across mechanical design, planning, and learning.