Valve programmer reveals difficulties of working on virtual reality

Valve programmer Michael Abrash, who is heading the team for the company’s virtual reality headset, has made a post on the Valve blog detailing the difficulties of working on virtual and augmented reality. The main takeaway from the post is the subject of latency. According to the post, latency is the biggest enemy of virtual reality.

Abrash wrote, “When it comes to VR and AR, l__atency is fundamental—if you don’t have low enough latency, it’s impossible to deliver good experiences, by which I mean virtual objects that your eyes and brain accept as real.”

The biggest problem from latency is perceiving virtual objects to be real. Real, in this sense, doesn’t mean that they don’t look virtual, “but rather that your perception of them as part of the world as you move your eyes, head, and body is indistinguishable from your perception of real objects.”

More from News & Analysis

What is the US HIRE Bill and why is India’s $250-billion IT sector worried? Is the internet dead? What's this theory that OpenAI's Sam Altman says might be true?

Sure, Valve. Blame lag for everything…

Abrash states that virtual objects have to stay in the same perceived real-world locations as you move, and being right only 99 percent of the time because latency is no good, “because the occasional mis-registration is precisely the sort of thing your visual system is designed to detect, and will stick out like a sore thumb.”

Back in September, Valve announced its plans to kick off the beta for its augmented reality headsets in 2013. For the beta, Valve will be releasing prototypes of the hardware to a select few gamers on Steam, though how these players will be selected is unknown. It’s not even confirmed how the control scheme on the headset will work. In an interview with Engadget, Valve’s Jeri Ellsworth suggested that no option - “from Phantom Lapboard-esque solutions to hybrid controllers” – is out of the question for the company.

Valve had revealed its plans for a virtual reality gaming headset in September last year, and it seems the company will face greater technical challenges when developing its headset as compared with Google’s Project Glass. While Google’s glasses will display texts and video conferences, Valve has greater technical challenges to overcome with augmented-reality games. It has to figure out how to keep stable an image of a virtual object (say, a billboard) that is meant to be attached to a real-world object (the side of a building) while a player moves around. Otherwise, the illusion would be shattered.

Earlier last year, Valve stated on a job listing for an industrial designer that it is “jumping in” the computer hardware business. The post adds that Valve is “frustrated by the lack of innovation in the computer hardware space” and sees “a real void in the marketplace.” Valve also seems to think that opportunities to create compelling user experiences are “being overlooked”.

“Valve is traditionally a software company. Open platforms like the PC and Mac are important to us, as they enable us and our partners to have a robust and direct relationship with customers. We’re frustrated by the lack of innovation in the computer hardware space though, so we’re jumping in. Even basic input, the keyboard and mouse, haven’t really changed in any meaningful way over the years. There’s a real void in the marketplace, and opportunities to create compelling user experiences are being overlooked,” the post reads.

The post doesn’t include information about any specific computer hardware Valve may be considering, but it does mention that basic input peripherals such as the keyboard and mouse “haven’t really changed in any meaningful way over the years.”