Design of real-time VR monitoring SDK

Due to the lack of monitoring tool currently, performance test on VR applications is often limited to pre-publish state via Profiler in Unity 3D engine, while application performance monitoring in operation in user environment is not practical. To make things worse, actual performance operating in user environment may much differ from performance test result carried out by developer before publish. Unprecedented crash may also occur, resulting in a more dissatisfied user experience. User interaction in VR scenes is more diverse and natural compared with traditional ones, thus it is of great value to monitor user behavior and further excavate demand from users, which developers are exactly looking for. A real-time VR monitoring SDK is built up for collecting and uploading data to analysis platform including user hardware, application performance, crash statistics and user behavior in interacting with VR applications.
The SDK runs as scripts assigned to Camera(head), Controller(left) and Controller(right) respectively, each script runs at the same pattern as Unity 3D engine and collects data from device. The SDK can be divided into the following modules (as shown in Fig.3):
  1. Configuration, which is set prior to the use of SDK. Developers shall register the VR application on test platform and obtain a unique appId which is designed to distinguish each application. Only when the appId is set in SDK can the monitor begin. Calling for public information including device serial number, scene number and scene name is also realized via configuration module.
  2. User hardware, which is designed to collect hardware information in user environment via Awake method by calling SystemInfofunction provided by Unity 3D. In Awake method, all elements in VR scene are called upon completion of setup. SystemInfocollects information of user device serial number, CPU, GPU and operating system so as to assist developers better modify VR applications to achieve better user experience.
  3. Application performance, which represents the operation quality of VR applications in user environment. Frame rate as well as Mono stack memory are monitored real-time. Here, frame rate is defined as number of frames rendered by Unity 3D engine in one second[10]. The higher frame rate, the smoother graphic can be presented. Mono stack memory[11]can be expanded dynamically and automatically according to need of system. Such information can reveal whether there’s flaw in application performance or not.
  4. Crash statistics, which is designed to collect user environment information when crash occurs. Here, crash is defined as unprecedented error occurred in C# script assigned to VR applications. Such error may not lead to malfunction on Unity 3D engine itself, but is likely to result in application error (i.e. unable to interact with object).
  5. User behavior, which collects information in interaction including scene information, joystick button information, object interaction, which represents user demand and better assist developers to modify VR applications.
  6. Data upload, is responsible for generating JSON file based on collected data and upload to analysis platform via HTTP agreement.