CNN-based Visual Servoing for Simultaneous Positioning and Flattening of
Soft Fabric Parts
Abstract
This paper proposes CNN-based visual servoing for simultaneous
positioning and flattening of a soft fabric part placed on a table by a
dual manipulator system. We propose a network for multimodal data
processing of grayscale images captured by a camera and force/torque
applied to force sensors. The training dataset is collected by moving
the real manipulators, which enables the network to map the captured
images and force/torque to the manipulatorâ\euro™s motion in Cartesian
space. We apply structured lighting to emphasize the features of the
surface of the fabric part since the surface shape of the non-textured
fabric part is difficult to recognize by a single grayscale image.
Through experiments, we show that the fabric part with unseen wrinkles
can be positioned and flattened by the proposed visual servoing scheme.
 *The complete version of this preprint paper is in proceedings of ICRA
2023. Please refer to
“https://ieeexplore.ieee.org/document/10160635”.Â