Design of real-time VR monitoring SDK
Due to the lack of monitoring tool currently, performance test on VR
applications is often limited to pre-publish state via Profiler in Unity
3D engine, while application performance monitoring in operation in user
environment is not practical. To make things worse, actual performance
operating in user environment may much differ from performance test
result carried out by developer before publish. Unprecedented crash may
also occur, resulting in a more dissatisfied user experience. User
interaction in VR scenes is more diverse and natural compared with
traditional ones, thus it is of great value to monitor user behavior and
further excavate demand from users, which developers are exactly looking
for. A real-time VR monitoring SDK is built up for collecting and
uploading data to analysis platform including user hardware, application
performance, crash statistics and user behavior in interacting with VR
applications.
The SDK runs as scripts assigned to Camera(head), Controller(left) and
Controller(right) respectively, each script runs at the same pattern as
Unity 3D engine and collects data from device. The SDK can be divided
into the following modules (as shown in Fig.3):
- Configuration, which is set prior to the use of SDK. Developers shall
register the VR application on test platform and obtain a unique appId
which is designed to distinguish each application. Only when the appId
is set in SDK can the monitor begin. Calling for public information
including device serial number, scene number and scene name is also
realized via configuration module.
- User hardware, which is designed to collect hardware information in
user environment via Awake method by calling SystemInfofunction provided by Unity 3D. In Awake method, all elements in
VR scene are called upon completion of setup. SystemInfocollects information of user device serial number, CPU, GPU and
operating system so as to assist developers better modify VR
applications to achieve better user experience.
- Application performance, which represents the operation quality of VR
applications in user environment. Frame rate as well as Mono stack
memory are monitored real-time. Here, frame rate is defined as number
of frames rendered by Unity 3D engine in one
second[10]. The higher frame rate, the smoother
graphic can be presented. Mono stack memory[11]can be expanded dynamically and automatically according to need of
system. Such information can reveal whether there’s flaw in
application performance or not.
- Crash statistics, which is designed to collect user environment
information when crash occurs. Here, crash is defined as unprecedented
error occurred in C# script assigned to VR applications. Such error
may not lead to malfunction on Unity 3D engine itself, but is likely
to result in application error (i.e. unable to interact with object).
- User behavior, which collects information in interaction including
scene information, joystick button information, object interaction,
which represents user demand and better assist developers to modify VR
applications.
- Data upload, is responsible for generating JSON file based on
collected data and upload to analysis platform via HTTP agreement.