Cyrille Rossant

and 2 more

IntroductionMost scientific disciplines are facing exponentially increasing amounts of data, which require scalable interactive scientific visualization technology. The development of massively parallel graphics processing units (GPU), fostered by the video game industry and artificial intelligence, represents a remarkable opportunity in this respect.Real-time computer graphics technology has been used in scientific visualization for decades, mostly via OpenGL, a popular open-source graphics library created in 1992. It has been used by many video games, graphics applications, and scientific visualization software. During its first decade, OpenGL provided a fixed function pipeline that was simple to use, but not particularly powerful because of the lack of control of the rendering pipeline. In 2004, OpenGL 2.X introduced a programmable pipeline that gave the user a way to customize the various stages of rendering. Since then, there have been various open-source libraries providing OpenGL-based scientific visualization, mostly focused on 3D rendering. Mayavi is a popular example in Python \cite{Ramachandran_2011}.In 2013, Luke Campagnola, Almar Klein, Cyrille Rossant, and Nicolas P. Rougier wrote a new visualization library: VisPy \cite{np2015}. This library took full advantage of the GL ES 2.x API to achieve both fast and scalable rendering of the most common plotting objects, in both 2D and 3D (lines, scatters, images, colormaps, volumes, meshes, etc.). Within a few years, VisPy reached a large scientific audience and became the main real-time 2D/3D scientific rendering library in Python.Although usage of OpenGL is still widespread in the graphics and scientific communities for legacy reasons, the industry is steadily moving to newer low-level graphics APIs such as Vulkan (Khronos), WebGPU (W3C), Metal (Apple), and DirectX 12 (Microsoft). In this context, VisPy is now facing the same problem as Mayavi faced a few years back: it must decide on its future.The Khronos group introduced the Vulkan API in 2016 (https://www.khronos.org/news/press/khronos-releases-vulkan-1-0-specification). This has been a complete redesign from the ground up to give much more control (compared to OpenGL) and to support all features of the latest GPU hardware. However, Vulkan has a huge barrier to entry: drawing a simple triangle using the Vulkan API directly involves about a thousand lines of code. In particular, all the logic related to the presentation of images to the screen, using a swapchain for double- or triple-buffering, all the synchronization of the different GPU tasks and CPU-GPU interactions in the main rendering loop must be done manually. To leverage the power of Vulkan for applications such as scientific visualization, there is thus a crucial need for intermediate-level libraries that drastically simplify the access to Vulkan.One potential solution isbe to use existing rendering engines such as Ogre (https://www.ogre3d.org/), Unity (https://unity.com/), or Unreal (https://www.unrealengine.com/). However, although scientific visualization and video games do share many similarities, they are quite different in their ends. Games are generally highly dynamic and interactive, whereas scientific visualization is much more static and less interactive (to some extent) and can be indifferently 1D, 2D, or 3D. Furthermore, scientific visualization involves a number of concepts that are generally not present in game engines, such as colormaps, labeled axes, non-cartesian projections, image interpolations, etc. Scientific visualization must also be faithful to the data and this requires a highly precise rendering to achieve high representational accuracy. Besides, it is not unusual to render millions or even billions of points in a scientific visualization — while a modern GPU has no problem rendering such a large collection, the corresponding API must be aware of such extreme cases to ensure proper rendering. All these limitations disqualify typical video game rendering engines as a general purpose API for scientific visualization.This exposes a crucial need for a rendering engine that is to scientific visualization what game engines are to video games: an intermediate-level library that allows developers of custom scientific visualizations, or developers of high-level plotting libraries, to leverage Vulkan without delving into an incredibly complicated low-level API. We report here progress that we have made in the past couple of years towards a cross-platform, cross-language scientific visualization engine that leverages Vulkan for scalable, low-overhead, high-performance scientific visualization.