A good motion retargeting cannot be reached without reasonable consideration of source-target differences on both the skeleton and shape geometry levels. In this work, we propose a novel Residual RETargeting network (R2ET) structure, which relies on two neural modification modules, to adjust the source motions to fit the target skeletons and shapes progressively. In particular, a skeleton-aware module is introduced to preserve the source motion semantics. A shape-aware module is designed to perceive the geometries of target characters to reduce interpenetration and contact-missing. Driven by our explored distance-based losses that explicitly model the motion semantics and geometry, these two modules can learn residual motion modifications on the source motion to generate plausible retargeted motion in a single inference without post-processing. To balance these two modifications, we further present a balancing gate to conduct linear interpolation between them. Extensive experiments on the public dataset Mixamo demonstrate that our R2ET achieves the state-of-the-art performance, and provides a good balance between the preservation of motion semantics as well as the attenuation of interpenetration and contact-missing.
|
| Overview of the proposed network R2ET, which has three decoupled modules, i.e., the skeleton-aware module $\Delta \mathcal{F}_{s}$, the shape-aware module $\Delta \mathcal{F}_{g}$, and the balancing gate $\mathcal{F}_{w}$. The Distance Matrix (DM) and the Distance Field (DF) are two types of distance measurements that guide the network to learn the information of semantics and geometry. |
@inproceedings{zhang2023skinned,
title={Skinned Motion Retargeting with Residual Perception of Motion Semantics \& Geometry},
author={Zhang, Jiaxu and Weng, Junwu and Kang, Di and Zhao, Fang and Huang, Shaoli and Zhe, Xuefei and Bao, Linchao and Shan, Ying and Wang, Jue and Tu, Zhigang},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={13864--13872},
year={2023}
}