Introduction
Virtual reality (VR) is not a new invention in the digital age. The stereoscopy technique has been used to create the illusion of the reality since two hundred years ago \cite{Crary_2002}. During the development of such technology, the VR pioneers have never stopped trying to deliver the more realistic experience. Bringing people to where they will never get the chance to go has become the goal for nearly every VR company -- one can simply tell that by paying attention to the number of appearances of the phrase "being there" in the press about VR. However, the emergence of the digital media undoubtedly makes people’s fantasies about virtual reality into the full circle, as every other medium is witnessed to be integrated into the digital media nowadays, even for the most ubiquitous one -- the reality. The vision of future post-human life regarding VR could be seen in many fictional works, such as The Matrix (1999) or Ghost in the Shell franchise where all human-beings live in inside the electronically virtual world that is a perfect mimic of the reality that we currently live in. Such mainstream envision of VR brings a new question: is re-creating the reality the ultimate capability of digital media? In other words, the idea of "virtual realism" seemingly confines virtual reality solely to the verisimilitude based on our epistemology and limits its possibility of becoming a new medium beyond it. This paper tries to discuss the alternative potentials of VR as a medium, in addition to reconstructing the realism, by retrospectively comparing the state of art of VR to the panorama and motion picture techniques, as well as the philosophical debates about realism behind them.
The Problematic Verisimilitude through Human Eyes
"How to represent the real?" This almost philosophical question has haunted us for centuries. Over the history of media evolution, people have suffered from the inaccurate nature of metaphor and metonymy in language \cite{Jakobson_2002} from the most primitive body language inherited from animals to cinema invented in the last century. Due to the inability of reproducing the real -- representing it in its original form -- the discrepancy between the realism and the reality could be found in any art form. In Cinema and the Code \cite{Youngblood_1989}, Youngblood articulates that the realm of psychological realism originating from the visual language will be abandoned and finally replaced by the reality when the image becomes the object in virtual reality ("a three-dimensional database with stereo vision in a wraparound head-mounted display"). He asserts that skipping the process of interpreting images and perceiving the object as the object itself is the gateway to stepping into the reality. After entering the digital age, as virtual reality is resurrected by the electronics, his vision seemingly has come true. However, although the reality could arguably be reproduced in virtual reality, there is still one step missing. Our reception of such production is limited by human eyes.
Verisimilitude in Panorama: The Reality Confined by Perspective
The catchphrase “being there” in the introduction about contemporary VR technology resonates with another term "the perfect illusion of space" when Dinkla talked about the panorama and the digital cave before the mass adoption of the current VR technology \cite{soke2002art}. In the nineteenth century when the panorama technology was firstly introduced to the public, because of the unprecedented intense telepresence sensation they delivered, a number of panoramic city views were created to save the educated middle class from arduous travel. Artists used panorama as an imaginative representation of the reality that was "intended to resemble as closely as possible the experience of being surrounded by or being inside nature in reality." However, comparing to the real telepresence experience, Dinkla describes the sensation inside of panorama as an illusion. Based on her research, people at that time considered the panorama painting as the non-linear web of multiple worlds "in which it is difficult to achieve one's customary orientation." The technology that was meant to recreate the reality ironically became a constant reminder of human's disability in perceiving it. Such paradoxical realism in the panorama has been further addressed in Crary's article \cite{Crary_2002}, where she introduces Roland Barthes's reality effect \cite{barthes1968reality} to the network (or "web" in Dinkla's words) of the real represented by such technique. She asserts that the world represented in the panorama is composed of referents conjured up by the various image signifiers in the painting, and the limitation of people's perspective results in an awkward relationship between the production and the reception. Because of the small field of view of human's eye, when investigating the "network of the real", the audience has to focus on the specific perspective of the whole picture. The perspective the audience choose predominantly determines the reality they will perceive. However, compared to the traditional "two-dimensional painting", people are more likely to lose their focus in the panorama where every part of the painting is fighting for viewers' attention. As the result, regardless artists' effort in producing the verisimilitude through the network of the signifiers, the inability of receiving it as a whole makes the realism exceed the models of visibility. The mismatch between the production and the reception has limited the spatial storytelling in centuries. The perspective restriction even becomes tougher when it comes to the current VR headset that offers a narrower viewing field than natural human eyes.
Verisimilitude in Motion Picture: The Reality Confined by Time
In addition to the limited field of view that prevents human from perceiving "the real", time is another hindrance to the epistemology of reality. If using the idea of the control variable, it is useful to examine the time confined realism in cinema -- the medium that always stays in one perspective but constantly changes in time. One of the examples would be Tacita Dean's The Green Ray (2001) discussed in A cinema in the gallery, a cinema in ruins\cite{balsom2009cinema}. It is a two-and-a-half-minute film recording a sunrise where one frame captures a glimpse of a green light that is unnoticeable both to its viewers and Dean who was not sure if she saw the light with her own eyes when shooting it. That ephemeral green light acts as an allegory of both film apparatus and human perception regarding the realism. The human eyes could only discern what happens within a very limited range of time (neither too long or too short) while film keeps giving us the illusion that such time range consists of the whole reality until the green light moment. This limitation of our visual perception still exists in the current VR technology that works with the optical lenses.
Verisimilitude in Virtual Reality: Representing Reality through Panorama in Motion
On account of the above discussion about the verisimilitude confined by the epistemology, representing reality in VR seemingly suffers more from the mismatch between production and reception. Looking at current VR practitioners' efforts of delivering the photo-realistic experience, most of them could be seen as working on polishing the content of the panorama painting in motion that gathers both modes of human's perceiving limitations. The realistic content in the current VR works overloads with both objects and the movement, but the audience could not fully grasp neither of them because of our perceiving limitation in both perspective and time. Human has been living in the reality ever since they were born, which has already given them enough familiarity of everything surrounding them based on the epistemology. Quoting McLuhan's hackneyed aphorism "the medium is the message" \cite{McLuhan1994}, the art movement going beyond the realism that liberates people from the sensing restriction and gives people a new perception of the reality could be found in any medium (e.g., the impressionism after the realism in painting) but not yet in VR. Compared to those "more traditional medium", the abundant information preconditioned by VR that is meant to create immersion adds little to our understanding of the reality itself and consequently becomes redundant. Such content wise redundancy, in turn, reorients audience's attention to the imperfection of both the medium and human perception, and therefore, defeats the purpose of immersion (or "being there"). The awkward paradox found here is pivoted on reproducing the real, which confines virtual reality solely to the content that already exists in reality and limits its possibility of becoming a new medium beyond a mere mimic of the real world. In this light, the approach that takes the advantage of VR as a medium away from realism is especially needed.
The State of Art in VR: Realism vs. Formalism
In his essay entitled Basic Concepts \cite{kracauer1960theory}, Siegfried Kracauer particularly mentions a period in early film history in which a clear line was drawn between the two categories -- the realist and the formative. The strict realists such as Lumière brothers followed the idea of seizing upon physical reality in its original movement, while the formalist filmmakers were testing the boundary of the motion picture and gave free reign to their artistic imagination. Although the line was blurred later signaled by those directors who combined the content that is captured from the reality with the unique film forms (e.g., Sergei Eisenstein and Luis Buñuel), the debate between the realism and the formalism was crucial to the development of film as an art form. Interestingly, the current state of art in VR could be seen as the comparable to that time period in film history. Most of VR works could be considered as realism that focuses on the realistic telepresence experience but suffering from the constraints of reality. On the contrary, there are still a number of VR practitioners exploring the medium specificity of VR but having difficulties in delivering the message or telling stories.
One of the examples of the realism in VR is The People's House (2017) made by Felix & Paul Studios. It is a twenty-minute VR documentary that gives an intimate tour of the White House, narrated by the 44th president of USA Barack Obama and his wife Michelle. Meeting the president is undoubtedly an extraordinary experience for most of its audience, not to mention having him as a tour guide to the White House: The People's House perfectly falls into the category of showing the spectacle in reality that is still under the constraint of human visual perception. Based on whether the main characters (Mr. and Mrs. Obama) are presented in the story, the documentary could be divided into two parts. Nearly half of the VR experience is about the character sitting next to the audience and talking to them, while the other half is the mere navigation in the White House with no direction. In the first part where the audience is facing the former president or the first lady, in order to gain audience's attention from the infinite perspective of viewing angles, the rest part of the space is kept static. Such approach simply makes all of the information other than the character represented in the experience meaningless and redundant. However, in the other half of the documentary where no character appears, viewers will easily lose their focus as every perspective contains "useful" information, which makes all of the information "useless." As a result, as discussed in the above section, the audience will be pulled out of the immersion, which counters the initial idea of the immersive documentary.
The example of the formalism in VR could be found in many experimental VR experiences such as Hovering (2017) by 79 Ancestors and LoVR (2016) by Aaron Bradbury. Both of them try to depict elusive and abstract ideas through geometries in space. However, as the audience tends to perceive what appears in the virtual reality as the "real object" \cite{Youngblood1989}, those conceptual VR visuals that never showed in the reality are hard to be understood. Therefore, the narrative and message in these formalism works could hardly convey to their viewers. Just similar to films in that specific time before the convergence of content and form, the unique approach(es) to VR storytelling that uses its medium specificity while keeping the narrative comprehensible is(are) still needed to be found.
Trying A Different Approach to VR Storytelling