The rapid growth of the live streaming industry brought about the VTuber trend, where content creators use character avatars to stream. One of the most accessible way to move a character in real time is by using a vision-based facial motion capture technique. However, previous works still suffer from jittering issues, which hinder the quality of the character’s movements. This work aims to develop a smoothed facial motion capture system that works with a live2D VTuber model. The system combines Mediapipe Face Mesh and OpenCV solutions to capture facial landmarks, which are then used to calculate head pose estimation using the Perspective-n-Point (PnP) function. In addition, the system uses EAR and MAR functions to detect facial features. The motion values obtained from this process are then filtered using a Kalman filter. Finally, the filtered motion data is sent to the Unity engine, which drives the Live2D VTuber model by adjusting the character’s motion parameters. The developed system successfully captures and drives the Live2D VTuber model with smoother motion, overcoming the jitter problem prevalent in previous facial motion capture approaches. The system’s improved motion capture quality makes it a more viable option for a wide range of potential uses.

The work description of virtual reality technology is superior to static reference pictures and videos, which can improve the predictability of decision makers and become a powerful performance tool for urban lighting design. The urban scene is constructed by Unity 3D, and the intelligent lamp pole and Rhododendron landscape lawn lamp are taken as the research objects. On the basis of maximizing the application effect of lamps with the HTC VIVE virtual headset, the city and lamps are digitally mapped 1:1, After applying the model material self-luminous, setting the type of light source and the position of the light source point, setting the transparency mixing, adding glow, and changing the light effect, a highly restored lamp and lamp simulation model are formed. The research results show that this method provides a convenient research method for the three-dimensional reconstruction and visual simulation of urban landscape lamps. It can make decision makers have a clear understanding of the application effect, application quantity and interval of lamps. Immersive observation experience can make the design scheme more honest and reliable. The efficient simulation of lighting lamps will play an important role in the dynamic development of smart cities in the future.

This paper presents a novel real-time facial feature extraction algorithm, producing a small feature set, suitable for implementing emotion recognition with online game and metaverse avatars. The algorithm aims to reduce data transmission and storage requirements, hurdles in the adoption of emotion recognition in these mediums. The early results presented show a facial emotion recognition accuracy of up to 92% on one benchmark dataset, with an overall accuracy of 77.2% across a wide range of datasets, demonstrating the early promise of the research.