Skip to content

Worlds on a Wire

Virtual reality still searches for a purpose

“Virtual reality” is an oxymoron, of course. That was one of the criticisms of the term back in the 1980s, when the question of what to call the slightly ludicrous assemblages of head-mounted screens and sensors then being developed, and the experience of being in an illusory space they were designed to provide, was still up for debate. “Artificial reality” was more popular for a while—when the New York Times ran a front-page story on the technology in 1989, it was headlined “For Artificial Reality, Wear a Computer”—though that term is no less contradictory. Ted Nelson, who had coined “hypertext” a couple decades earlier, proposed “virtuality”; others preferred “cyberspace,” which was not yet solely associated with the internet. (It would be some time before my favorite alternative, “phantomatics,” coined by Polish author Stanisław Lem in the early 1960s while speculating about the possibility of such technology, made its way into English.) But in the end, which is to say by sometime in the early 1990s, “virtual reality” prevailed.

Incoherence, it turns out, has made the term more apt, not less. VR itself is an oxymoronic technology. Using it, you encounter a new form of contradiction: between presence and appearance. Nothing in VR ever looks real—in fact, most things look pretty terrible. Textures are a little muddier than you’d expect, 3-D models a little blockier, animations a little less smooth, video footage a little less sharp. There’s an implicit Law of VR Inferiority when it comes to graphics: because everything in VR has to be rendered twice, once for each eye, those things will always look worse than they would in non-VR. No matter where you find yourself in the arms race of computer graphics—the decades-long slog to conquer reflections, water, shadows, facial expressions, and all the other problem areas of reality—to move from the screen to the headset is to regress technologically. A Valve Index plugged into a high-end gaming PC will look worse than if you used that same PC with a normal monitor; a PSVR headset plugged into a PlayStation 4 will look worse than just using that same PS4; and so on. And yet the trick of strapping screens and sensors to your face works, all the same. Those unimpressive, clearly fake digital objects feel like they really are there, present in the same space as your actual body: that pixelated, stiffly animated whale really is looming over you; those jagged, blurry rats really are scurrying around you; you really are soaring over that flat, low-res approximation of the ground. They are present, somehow, for all their obvious unreality.

In a way, then, VR is more impressive the less convincing it looks. The older and more primitive iterations make this most apparent. It’s not your eyes that are being fooled, in the usual way of special effects and video games, but something else, some deeper intuition about your surroundings. This can actually be quite irritating. Mediocre or outdated programs retain an effectiveness that feels unearned, overpowered. I remember playing Until Dawn: Rush of Blood a few years ago—a fairly generic horror rail shooter released in 2016 for the PSVR—and becoming increasingly angry at how viscerally unnerving its cheesy evil clowns and lazy jump scares were. None of it was original or impressive or actually unsettling, but those clowns were, nonetheless, behind me.

Just as often, though, it feels like a strange little miracle: to find yourself having an authentic encounter with a poor imitation, truly standing in the presence of the unreal. This is why video recordings of VR are so unappealing: they reproduce the shoddy artificiality but not the miraculous presence. The first time you use, say, Google Earth VR can feel a little transcendent. The earth spins at your whim, landmarks shrink beneath your feet, and you skip over mountains like a god. Recordings of it, on the other hand, look like a screensaver having a breakdown.

VR’s history is similarly contradictory. Most of the time, it pretends not to have one. It is always arriving—all future, no past. Each new headset rides in on a wave of hype and promise, pointing the way to a glorious headsetted world just over the horizon. Palmer Luckey, the cofounder of Oculus, whose successful Kickstarter campaign in 2012 helped start the most recent wave of VR, was, according to a Wired cover, “about to change gaming, movies, TV, music, design, medicine, sex, sports, art, travel, social networking, education—and reality.” After Facebook bought Oculus a few years later, Mark Zuckerberg declared that reality itself was too “limited”: “If you can’t think of any way that your reality can’t be better, then you’re not thinking hard enough.”

Yet, by this point, VR is actually rather old. Howard Rheingold’s Virtual Reality, an elegant, decades-spanning, four-hundred-page history of the medium, was published in 1991. Its dust jacket came blazoned with one of Arthur C. Clarke’s less impressive predictions: “Virtual Reality won’t merely replace TV. It will eat it alive.” Over thirty years later, sales of VR headsets have declined from 2022 to 2023, to about eight million sets sold worldwide. TV sales dipped too—to just under two hundred million.

The Hole Through Which Air Escapes

I’m slumping my way into middle age, but VR was a vivid feature of my childhood. I remember driving down to Boston in the mid-nineties with my dad to go to a fancy new computer café and paying some ungodly amount (a dollar a minute? more?) to play a multiplayer shooter called Dactyl Nightmare on their new VR rig. Even back then, the trick worked. And then as now, the easiest way to showcase it was to spin up a video game. Dactyl Nightmare was slower, clunkier, and far more limited than Doom, Descent, Duke Nukem 3D, and the other first-person shooters I was used to, and strapping on the headset was oppressive and isolating in a way normal video games never were. But its garish checkerboard platforms seemed solid beneath my feet, the blocky forms of other players seemed to really be stuttering past my blocky gun, and when a pterodactyl picked me up and flew me over the level, the height was startling.

Because everything in VR has to be rendered twice, once for each eye, those things will always look worse than they would in non-VR.

That rig was something called a Virtuality 1000. There were hundreds of them back in the 1990s, using magnet-based motion tracking that was cumbersome and expensive, but more accurate and responsive than many much more recent VR systems. Virtuality, the British company behind it, not only spread its machines through the arcades but formed partnerships with Ford and other companies and, with IBM, launched a headset called “Project Elysium” that was meant to “revolutionize diverse fields such as architecture, engineering, medicine and city planning.” It’s archaeological now. Virtuality declared bankruptcy in 1997, and the handful of Virtuality 1000s at the Retro Computer Museum in Leicester seem to be the only ones in the world still in working order.

The first VR bubble burst around 1995: headsets were expensive, arcades were declining, and no other market for them quite panned out. The second bubble, which started with the Oculus Kickstarter in 2012 and accelerated with the release of the Oculus Rift in 2016, has been slowly deflating for the past few years. VR has never quite found a purpose. That’s the hole through which the air escapes.

VR’s backers always claim a revolution is coming—in gaming, of course, but also in surgery and government and just about everything. But so far, it’s turned out to be best at thoroughly impressing people for about an hour, or until they hear the price tag. A few niches have been more lasting. It’s useful for flight simulators (military flight sims are actually one of its main progenitors) and for certain types of medical training. Architects and interior decorators sometimes use it to show clients their designs. Film directors have used headsets while working on especially VFX-heavy films. VR has made a few tentative inroads into porn, that hidden hand behind so much technological progress, but seems too expensive to spread further, especially as big-budget porn has become more or less a thing of the past. (VR porn does have some novel features, not all of them intentional: VR video sometimes struggles with scale, so you could find yourself confronted by a naked woman who seems to be about nine feet tall—surely someone’s dream come true.) All that is just nibbling around the edges, though; it’s nothing that would appeal to millions of ordinary users, and it’s nowhere close to what would sustain the thriving industry so many enthusiasts have wanted and expected for so long.

Want is the operative word. VR is sustained by desire, not utility. Its appeal—to investors, to engineers, to journalists and children and early adopters—is primal, irrational, inchoate. Someday it may find some function real enough to anchor it and finally achieve the status of consumer good, but for now it floats free. By some definitions, that makes it not quite a technology at all—not quite “a means to fulfill a human purpose,” as W. Brian Arthur has it in his book The Nature of Technology—but something else: a premonition, maybe, or a delusion, driven not by what it is but by what it might be when the future finally arrives.

Part of the strangeness of VR comes from this incompleteness, this sense that it is half device, half dream. Every headset and program are a promise of a revolution to come, just a fragment of an unseen whole, a true VR that has not yet seen fit to appear. Its real nature, as a result, is most visible in its failings, not its achievements—in the gaps still left in its construction. VR can sometimes seem most intelligible not while you’re using it but just after, in the disorienting moment when you take the headset off and find the real world not quite as you left it. If you were playing standing up, you might find that you’ve drifted a little and are now in a corner of the room. If you’ve been playing for a while, you might find that the room is now dark, or, as I once did, that a houseguest is now asleep on the couch behind you. And no matter what, you’ll feel the full sensory complexity of the real world come rushing back—reality at its actual resolution, with its full field of view, in its complete spectrum of color.

Patched In

During the early years of the most recent VR boom, it was trendy to talk about VR as an “empathy machine,” as a way of temporarily becoming, and thereby coming to understand, other people. In the anthropologist Lisa Messeri’s recent book In the Land of the Unreal, she follows efforts in Los Angeles in the mid-2010s to use VR in this way, including a group of fledgling VR thought leaders she encounters through workshops and meetups, and the head of a lavishly funded research facility called the Technicolor Experience Center. All of them see LA, with its Hollywood infrastructure and proximity to Silicon Valley, as a possible VR hub, and they envision VR as a vehicle for various types of social progress: a less misogynistic tech industry, less hateful immigration policies, or more compassionate health care. Most of the efforts end in disappointment. (The one tentative exception is a startup that uses VR to help train people in empathetic elder care.) The idea of VR as a cultivator of empathy, Messeri writes, was popularized by a TED Talk the filmmaker and VR entrepreneur Chris Milk gave in 2015: “Through this machine,” he declared, “ultimately we become more human.”

VR is sustained by desire, not utility. Its appeal—to investors, to engineers, to journalists and children and early adopters—is primal, irrational, inchoate.

But to use VR is, much more immediately, to experience what it is to be less human. The artificial existence it offers is so incomplete, so inadequate, it makes you newly aware of your own bodily capacities. You feel them denied and then, miraculously, restored. This is especially clear if you skip between different headsets rather than spending prolonged time with just one. You might start with something especially primitive, like Google’s smartphone-based Cardboard (discontinued in 2021, as the deflation began). Here you can only look around, without otherwise moving. And even “looking around” is patchy: you can turn and tilt your head, but cannot lean forward or crane your neck. Try to, and you are confronted with a kind of reverse-motion sickness, or sensory paralysis: your actions are not resisted, but simply ignored. Writhe all you want, but the world stands still.

Switch to something a little more sophisticated—say, Sony’s old PlayStation VR, launched in 2016, superseded by the PSVR 2 last year—and bodily possibilities come flooding back. Now you can lean however you want, even crouch down or move around a little—though just a little, as the camera-based tracking has very limited range. One of my favorite PSVR demos puts you in a dive cage, and that’s about right: a step or so in each direction, no more. But your hands are back. You can wave them freely, even pick things up, though they are more like binary claws that are open by default, with a button press to close, than human appendages. (And you have to be careful not to move them in front of each other or too far behind you, or the tracking will fail.)

Try something better—perhaps an HTC VIVE Pro 2, plugged into a fancy PC, with a couple external sensors—and the world opens further. Now you can, luxuriously, walk around the whole room. It is still just tracking your head and hands, though, so as you walk you might get to experience, depending on the program, the deep strangeness of an extrapolated body, imperfectly guessing at your actual gait, or maybe the simpler alienation of your body omitted entirely and replaced by a disembodied pair of hands, floating around in front of your face. As controllers have advanced, those hands have regained some capabilities. Some even register whether you’re lifting your fingers, so you can point again, and flip people off, though not quite beckon.

With each new device you can feel yourself disassembled and imperfectly rebuilt, a little differently each time. It is not always a clear-cut advance either. The new Apple Vision Pro uses an array of tiny cameras to track your hands, letting you wiggle your fingers however you want. But with the controllers gone you can’t feel anything, not even the previous crude haptic buzzings, leaving you to interact with the simulated world as a nimble-fingered ghost.

These limitations often have unexpected results. In Joe Hunting’s We Met in Virtual Reality, a 2022 documentary shot entirely within the VRChat social app, he interviews a VR dance instructor who tells him that belly dancing is especially suited for VR because its leg and hip movements are easier for “full body” VR systems to detect. Other parts of the film reveal odd new forms of body language: an occasional indifference to the distinction between sitting and standing, as it is generally more comfortable to stay seated for prolonged VR use, even if your avatar is actively moving around; a tendency toward petting as a simpler, less glitch-prone alternative to hugs, which lose a bit of their point anyway if you can’t feel the other person (that many VRChat users choose anthropomorphic animal avatars presumably adds to petting’s appeal).

This may be why many of the most effective VR experiences involve a further derangement of the senses—darkness, submersion, hallucination. Tilt Brush, Google’s VR drawing program, is still perhaps the most thoroughly pleasing way to spend time in VR: a dream of weightless, bodiless, changeless existence, of a world without physics, a clean Cartesian void to fill with your own frozen gestures. (Even that program was abandoned by Google after a few years, however.)

And of course whole swathes of human capacity remain unsimulated, from the feeling of sprinting to, say, the sense of smell. In Virtual Reality Rheingold spends some time with an early-sixties “prehistoric VR prototype” called the Sensorama. Along with binocular film footage of a motorcycle ride, a “romp on the beach,” and a visit to a belly dancer, it featured a “system for transporting odors to [your] nose and whisking them away”—a frontier to which no subsequent system has returned. That’s probably for the best (what is the olfactory equivalent of a tracking error?), but it remains true that the gap between using VR and actual experience is enormous. This is the hidden, fundamental fantasy of VR: that it could, someday, be an adequate fantasy; that the illusion it offers could be complete and irresistible; that the worlds it presents could seem real and necessary. What each actual system offers instead is its own particular spread of inadequacies, its own unique form of impoverished being. And the technology as a whole is never quite embraced because it never quite manages to exist.

 

A screenshot of a rudimentary videogame depicts a black room with orange stairs, columns, and gateways, with a pterodactyl flying above, and the numbers “2:49” next to it.
Screenshot of 1991’s Dactyl Nightmare taken from the Virtuality 1000 VR hardware.

Second Lives

In his brief, elegant book The Immersive Enclosure, the media theorist Paul Roquet argues that these failings might actually be a part of VR’s attraction, in the right circumstances. His subject is VR in Japan, where it has had a little more sustained popularity than in the United States. There, Roquet argues, VR is more often seen not as a portal to another world but as a kind of souped-up blindfold: “The most important attribute of the headset is its ability to offer a diminished reality, a tool that brackets out all but the task at hand.” Its most important predecessor is not film or video games or nineteenth-century panoramas, but the Walkman, the first piece of technology that let people replace part of the world around them with something else. (Roquet’s history is a surprising reminder of how strange and disruptive, even controversial, the ability to privately listen to music in public once was.) VR, then, would be just a Walkman for a few more senses. In that case, “the appeal of VR as an alternative frontier depends as much on what isn’t brought over from the existing world as what is.”

This appeal is stronger, of course, the more unpleasant the existing world becomes. Roquet focuses mainly on the social history around this virtual privacy. “Record numbers of Japanese relocated to urban environments” in the early 1960s, he notes, as the use of headphones was spreading, “often to live in wooden housing with notoriously thin walls and in close proximity to neighboring homes.” It is hard, however, not to see a vision of VR’s future here as well. It may well be that VR will get more alluring, and more satisfying, the worse things get—as the need for escape begins to outweigh the simulation’s inadequacies and as the real world becomes so degraded it begins to resemble VR. The whales disappear, and a VR dive cage is better than nothing; summers hit 120 degrees, wars spread, and Google Earth VR seems a better idea than going outside. In this sense, Palmer Luckey has been nurturing our virtual future from both sides: first as the founder of Oculus, and now as the head of a company producing military and law-enforcement drones. Perhaps VR will only truly arrive when the apocalypse does.

Certainly it became much more enticing during the pandemic. We Met in Virtual Reality emerges from this period, as does Hunting’s earlier web series Virtually Speaking and the YouTube channel People Make Games’s report on VRChat, “Making Sense of VRChat, the ‘Metaverse’ People Actually Like”—all of which document a vibrant social version of VR, a glitchy, garish, gregarious sanctuary. Users, generally in cartoonish custom avatars, congregate in customized, mildly interactive virtual spaces—from tree houses, living rooms, and bars to classrooms, malls, and theaters—most equipped with lots of mirrors so people can see their avatared selves. They might play a glitchy version of pool or drive around in more or less functional cars, but mostly they just hang out.

One might have expected VRChat to decline as lockdown lifted and Meta’s family-friendly, much better-funded social VR app was launched. But instead it has continued to grow. This year VRChat hit one hundred thousand simultaneous users for the first time. Not quite “the successor of the mobile internet” Zuckerberg predicted of his failed competitor, but impressive for VR’s current reality. And Covid is just one among many reasons people gave for their presence there, even during lockdown. They cited social anxiety, disability (several sections of We Met in Virtual Reality focus on a VR sign-language school), and grief; a desire for a place where they would be judged for what they said and did rather than how they looked; a desire for a place to be without speaking at all.

VRChat is chaotic, confusing, frequently hideous and ridiculous. But its antic, janky energy feels like a return to the old, pre-social media internet—the homegrown, ingrown spirit of message boards, multi-user dungeons, and Second Life. In it, you can see a somewhat less gloomy vision of what VR’s diminished reality might provide: not a revolution but a refuge, a chance to practice being human, or to take a break from its full weight; a place to resocialize, or to socialize more safely, with more distance and control. A place to try out being another version of yourself, or just to try to find a way to be yourself. And yet I can’t help but hope it will become less useful in the years to come, not more.