An Encounter with Vive:

Some terminology for the purposes of this article:

XR: Extended/Extensible Reality, or possibly just (x)R – an umbrella term covering all forms of simulated and mediated reality. (note: let’s agree to pronounce the “x” as a “z” like xylophone, so XR sounds like the bad guy from The Last Starfighter)
VR: Virtual Reality – a form of XR characterised by blocking out of the “real” world, providing a total immersion in a wholly simulated environment.
AR: Augmented Reality – XR in which the real world remains visible (either directly, or via a camera feed), and computer generated elements are added to mediate reality. (and sounds like a pirate noise)
GPU: Graphics Processing Unit – the card / components that drive the visuals of the VR experience. Usually a dedicated card in desktop computers, but built into the motherboard on many laptops.
eGPU: External GPU – A GPU in an external case, usually connected by Thunderbolt to the main system.

A few weeks ago, here in the sticks of regional Australia, we had a little conference day (immerseconf), with internationally practicing artists from all over the country (including the head of HTC Vive in Australia), demoing how various forms of Extended Reality are being used by artists to create content.

Interestingly, while there was a “serious games” (training & education simulations) discussion, traditional entertainment videogames weren’t covered – this was a conference targeted at makers, and the toolsets available to them for creating. This shouldn’t be taken as indicating the experience was dull – delight and joy are inherent to the experience of doing work in VR.

I’ve been reading about and waiting for this tech since the 1980s. Last time I tried it a couple of years ago, the head-mounted display (goggles) was an Oculus devkit, and interaction was via a playstation controller.

I was ill within a minute.

A theory of why this happens, is that it’s a result of lag between moving your head, and seeing the corresponding movement of the virtual world through the goggles. With the Vive, that problem is solved – the viewpoint is stuck fast to your proprioceptive experience of movement. Lag is gone, you are there.

For an artist, the experience of VR marks a division between everything you have done, learned or experienced in art-making prior, and what you are to do afterwards. It is as redefining an experience as postulated in Crosley Bendix’s discussion of the “discovery” of the new primary colour “Squant”.

In my life, I have been literally moved to tears once by a work I saw in an exhibition – Picasso’s “Head of a Woman”. Why? I had studied this work, part of the canon of historically important constructed sculpture, for years at art school. I’d written essays concerning, and answered slide tests about it. However, every photo I had seen was in black and white. I finally saw it in the flesh at an exhibition, and out of nowhere found myself weeping at the fact that I had never known what colour it was painted. Nothing I had read, or studied, prepared me for the overwhelming emotional impact of meeting it, face to face, and realising that I had not known something as fundamental as its colour.

Of all the great leaps in art making that Picasso was personally involved with, it was his collaboration with Julio González that more or less invented welded steel sculpture. He did this, primarily out of a desire to be able to “take a line for a walk” in three dimensions, to draw with thin metal rod, the only material whose structural strength could span distance without thickness or sagging.

In VR, free-standing, able to walk about with multi-function hand controllers in an entirely simulated, blank environment, I was once again almost in tears at how profound the experience of this tech is for artmaking. One can literally take a line for a walk, twist it, loop it around itself, trace out the topology of knots, zoom out, zoom inside, and see that three dimensional drawing as a physical object, hanging in the air.

The tools I played with were from Google – Blocks, a simple 3d modelling program, and Tilt Brush, a drawing and painting program (which is also a 3d modeller – it just models paint strokes, and so produces flat ribbons of paint, that follow the 3D orientation of the controller when you make them). They’re reasonably primitive compared to traditional 2D painting and modelling apps, but there’s clearly a commercial space for selling tools for VR.

Just watch this. That’s the actual experience of creating and working in Tilt Brush.

Or this:

Why would you want to use a screen-based 3d modeller?

Speculation, based on Observation:

The authoring environment for VR content, is VR.
- After 3D modelling, or drawing in VR, you’ll never want to model or paint on a screen again. The idea of not having a direct 3d experience while creating just becomes nonsensical. As for Tilt Brush, there’s no 2D equivalent, I’m not sure there’s even a way to think about how Tilt Brush would work in 2d.
- Don’t think about VR as a way to preview things you make on screen – making things in VR is so compelling, you will want to change the way you work, or change the sort of work you do, to get as much as possible into the immersion.
360 Video is probably going to end up being a niche or gimmick, like 3d television.
- The very clear sense I have after this encounter, is that 360 Video (which I first saw demonstrated 16 years ago at the QTVR Forum at Macworld New York) is an attempt by an old, established artform (and players within that artform), to annex a new format for itself, regardless of whether it is appropriate for that new format. If all you have is a hammer, you treat everything as if it were a nail.
- Outside of video art – time-lapses of locations, or documentary, 360 video may be a way to make video skyboxes and motion backgrounds, at least until software can make them more effectively than a film crew can shoot a real location, which if you look at any modern film, it can already do.
- Video’s monopoly on “real” will not survive the growth in quality of simulation, which carries with it true interactivity. Why experience a 360 video version of surfing, when you can have a photoreal simulated surfing experience, in which you do more than control the direction you’re looking, and can have it on that water planet in Christopher Nolan’s Interstellar?
- If 360 video fundamentally changes its nature, becoming something in which the narrative progression is reactive to the directed attention of the viewer, perhaps there’s a possibility there, but isn’t that just a video game with the skill tests removed?
- Otherwise, how do you get a jumpscare to work, if the viewer is never looking in the direction of the monster? Interactivity and moving around within a place is VR’s point. 360 video is about being a fixed point. Think of it as similar to the way focus-pulling and depth of field are fundamentally incompatible with 3d cinema – viewers can struggle against the director’s chosen point of focus, trying to see unfocussed objects they physiologically understand they should be able to “grip” with their eyes and pull into focus.
- There are also issues with the physics of optics, revolving around how panoramic images are captured, that make stereo separation with 360 video fundamentally problematic.
- Using VR headsets to screen non-interactive, immersive stereoscopic 3D video (in other words, you only see what the single direction paired cameras are pointed at) would certainly seem to have a future, given the pornography industry has adopted it for the Point Of View genre.
VR is a platform, not a peripheral.
- This is huge – Mac, Windows, Linux – all of these are irrelevant, you’re simply not going to interact with the host OS to any meaningful degree. The operating system of the computer, merely serves as the loader for the VR environment. You’ll have no more cause to interact with macOS or Windows, than you do to interact with your computer’s firmware. Tilt Brush will look like Tilt Brush, regardless of what operating system it is running on. Look at Adobe’s clear strategy to nullify the operating system as a differentiator, and get their users to think of their computers as “Creative Suite Workstations” rather than “Macs or PCs running Creative Suite”. VR will be even moreso.
- Everything is up for grabs as new paradigms for fundamental control schemes are solidified. Think how revolutionary the first pull-down menu was, that’s the sort of world VR is in. From Blocks and Tilt Brush you can see already, UI paradigms that are perhaps overly-literal. Multi-sided, rotatable physical palettes wrapped around the controllers are in vogue, but why? Why not have the equivalent of a 30″ monitor, offset 45 degrees, full of palettes that appears in response to a button press, then goes away again? Or, why not a literal wheeled toolbox, that follows behind you? The physicality of creating in VR is a very different working experience to sitting at a desk.
The GPU is everything.
- VR computers are just a host system for the GPU (Graphics Card). A non-upgradable GPU, or a system that can’t be traded up for the market retail cost of a GPU is a laughable idea, truly laughable. Once you use one of these systems and see how good it is, but more importantly how much better VR is going to get in the near future in terms of graphical fidelity, and consider the soon to arrive retina-scale upgrade to headset display densities, the thought of having to replace a whole computer, just to keep cutting edge, I mean it’s just an unthinkably stupid idea.
  - To put that in perspective, no matter what manufacturers claim, Nvidia’s 1080ti is the minimum graphics card you need to create a simple virtual environment of sufficient fidelity that you’d want to spend all day working within. That is the standard you have to show people, so they can think “this is here and I want to use it“.
  - The 1080ti is the second-fastest GPU Nvidia offers in terms of 3D gaming performance, which is the effective measure of how well the immersive environment will perform.
  - The 1080ti is around 30% faster than the fastest performing GPU AMD makes (Vega 64), for a significantly lower power draw and heat output.
  - Numerous developers, including HTC themselves, were demoing on laptops with Nvidia graphics – none of which required eGPUs. HTC’s laptop was subtly lower fidelity than the desktop machines, but not by a lot.
  - AMD graphics cards were nowhere to be seen. Every tower machine (which were bigger than my cheesegrater Classic Mac Pro, and mostly full of empty space) was team green (Nvidia).
VR has a huge future in healthcare.
- Hospitals here are permanently installing Vive trackers in the children’s wards, so bedridden kids can go participate in networked virtual environments with other kids, and not be bored / confronted with the reality of being in hospital.
- VR is being used for rehabilitation, gamifying physiotherapy rehab exercises for example, to ensure they’re done correctly, and to relieve the monotony of repeat-based therapy.

Food for Thought, AR vs VR:

There is a school of opinion which holds that AR is the “good” version of XR, that VR is a niche for games, that the goggles etc required for immersion makes VR inherently not a thing for the everyperson.

I have a different take on that. I think that AR would seem to be the “good” version of XR, vs full immersion VR, if you’re the sort of person whose socioeconomic status means your life is the sort of life from which you would never want to seek an escape. AR is lovely, if you’re able-bodied, rich, have a nice house, and a job with sufficient seniority that you have your own office and can shut out distraction.

In other words, if you’re employed with any sort of decision-making authority at a large tech company.

If you live in a tiny apartment or room in a sharehouse, or have a disability whose profundity stops you going out to access experiences, or work in a place where you can’t tune out visual distraction, in other words, if your life isn’t already the sort of 1%er fantasy that most people would like to escape to, then perhaps AR isn’t that compelling in comparison.

From that perspective, AR that does not have a “shut out the real world” function isn’t a complete solution – it’s not the whole story.

By the way, saying the goggles are inconvenient – go speak to anyone who does any sort of manual trade work. VR goggles are no more inconvenient than having to wear safety glasses, gloves, steel-capped boots, ear muffs, a respirator, or welding helmet. Just because it’s less convenient than an office worker is used to, doesn’t mean a lot – if I can sketch in 3d before I go out into the welding bay, that’s a huge convenience factor.

So Overall:

My encounter with Vive leaves me with mixed emotions. I am absolutely going to be gearing up for VR. You simply can’t try this tech, and then not move to make art with it. VR is here, and it is now. It is a complete, usable product with both entertainment, and work tools, not an early-access developer preview.

A lot of the coverage I’ve seen of VR, from people who perhaps don’t understand the sheer amount of heavy lifting necessary to drive the experience, centres around ideas like “wait until the PC isn’t required“. That isn’t going to happen, or rather, that’s going to be a sub-standard experience – a better packaging of current smartphone-based VR. The PC to drive VR isn’t going to go away, because the progress to be made in the medium, the complexity and graphical fidelity has so much room for growth that enthusiasts will keep asking for more, and content creators will have to keep up in order to feed that cycle.

Local Australian pricing has the Vive setup for about a thousand dollars, an Nvidia 1080ti for about another thousand, but what to do for a computer to run that rig?

Does Apple have a solution that lets me stay on the Mac, or do I jump to Windows, and begin the inevitable migration of all my Pro software (which is niche enough that it HAS to be cross platform) and production processes across to Windows versions?

Read on in Part 2: Hard Reality

Hey, If you liked reading this, maybe you’d like to donate?