Predicting movement with RSO Pose

Learning from human movement

We trained RSO Pose on human motion capture to generate movement one pose at a time. Our controller drives a character in simulation, with walking, running and turning as its foundation. A change in the requested direction or pace alters the movement the controller generates.

Motion capture records how the body changes over time. A walking sequence includes the timing of each step and the coordination between the legs and upper body. A turn develops across several moments as the character changes its facing and continues moving. Those sequences provide examples from which a model can learn.

Training the controller

Our work builds on the published DeepPhase architecture. We trained our own phase representation and motion controller on human movement, then adapted the controller for live use in simulation.

The phase representation describes patterns in the rhythm of movement across the body. It gives the controller information about where the movement is in its progression. That matters because the next pose depends on how the character arrived at its current position, as well as where it is being asked to go.

The controller uses recent movement together with the requested direction and pace to predict what follows. Training on recorded sequences gives us examples of that relationship. Running the controller interactively lets us examine how it behaves when its own generated poses become the history for the next prediction.

Generating continuous movement

Each predicted pose becomes part of the input for the next step. The character therefore moves through a continuing sequence of model outputs while responding to steering. We can change direction during a run and examine how the turn develops through the body.

This repeated prediction is an important part of testing. A plausible individual pose gives only a small view of the behaviour. We also examine whether movement remains coherent over a sequence, how transitions look at normal speed and how the character responds as the controls change.

Walking and running provide a foundation for that work. Their repeated steps make it possible to inspect timing and foot contact over several cycles. Turning adds another demand: the character has to change its course while continuing the movement already under way.

Meeting the ground

Generated movement also has to fit the simulated world. We combine the model’s poses with contact adjustments and control refinements to work on foot placement and transitions. These parts of the controller help place the animation within the scene.

We test the result in a live, controllable simulation. The learned pose sequence and the adjustments used to fit it to the ground have separate roles. Keeping that distinction clear helps us understand which part of the system needs attention when a foot slides or a transition looks wrong.

Towards more natural robotics

We want to make robot movement easier to read and anticipate. Familiar timing and coordination may help people understand an approaching body or recognise that it is about to turn. That is a direction for further work, including studying how people respond to the movement.

Games, digital avatars and animation tools are also possible applications. Our current controller runs in simulation. A physical robot would bring further requirements for balance and contact, as well as the control of its particular hardware. Those will need their own development and evaluation.