1. Home >
  2. Gaming

Microsoft Research invented video game time travel to fix cloud gaming

Cloud-based gaming hasn't taken off largely because of lag. Delivering those frames takes time, but Microsoft's DeLorean might be able to get around that by rendering the future.
By Ryan Whitwam
Microsoft DeLorean

The promise of cloud-based gaming has yet to be fulfilled in spite of the efforts of companies like OnLive and Nvidia. As low-power smartphone and tablets continue to proliferate, the notion of doing the heavy lifting of game rendering on a remote server makes some sense, but the issue of latency has to be dealt with. Now Microsoft Research thinks it might have devised a workable solution. It's basically video game time traveling, so they're calling it DeLorean. See? Microsoft can have a sense of humor too.

Existing cloud gaming platforms like Nvidia's Grid and OnLive need to make a trade-off between quality and latency. The frames that are being rendered based on user input have to be sent over the internet before they pop up on your screen, which comes with a much longer delay than if they were being rendered on local hardware. There's simply no way around that. When you add up the lag from getting the input into the server and returning a frame, that's called the round trip time (RTT). To keep the RTT reasonable, the cloud gaming systems we have now rely on lower resolutions and harsh compression. DeLorean is much more flexible because it actually renders the future before it happens.

DeLorean does four things to improve responsiveness and lower RTT -- future input prediction, state space subsampling, misprediction compensation, and bandwidth compression. Combined, these techniques can eliminate up to 250ms of lag, which can keep a cloud-based game playable even on mobile connections. Both Doom 3 and Fable 3 were modified successfully to compensate for network lag using DeLorean.

So let's say you're playing a first-person shooter on your tablet via a cloud gaming platform running DeLorean. Before you've even done anything, the DeLorean-equipped server has spawned three "slave" instances of the game (this can be increased depending on network latency). These copies are used to render possible frames depicting what you might do. One frame might be looking slightly to the left, another to the right, and the third could show the beginning of a muzzle flash as you open fire. These are all fed into the master game, which then immediately sends the correct one when your input reaches the server.

To make sure it's as accurate as possible, the future input prediction system uses an algorithm formed by your past behaviors to determine which frames are the most probable. The state space subsampling also plays into narrowing down possible actions with parallel GPU computation.

Speculation

With only a few true instances of the game, DeLorean obviously isn't rendering all possible futures--it's just the most likely ones(Opens in a new window). When the prediction isn't right, the system can actually determine if it was at least close to the right answer -- this is where misprediction compensation comes into play. DeLorean can use view interpolation to transform the pre-rendered frame to make it match the expected output. This is considerably faster than re-rendering the correct frame from scratch. Finally, DeLorean will resort to bandwidth compression when necessary to maintain low latency. However, Microsoft says the video encoding scheme used gets the job done with less artifacting.

Mircrosoft was using some high-end hardware to run DeLorean, but nothing outlandish. The server consisted of a quad-core Intel Core-i7, 16GB of RAM, and an Nvidia GTX680 with 4GB of VRAM. The server was configured to artificially produce delays similar to what a remote game rendering farm might result in, but the games needed a fair bit of tweaking to run on DeLorean. Still, this might be how we play games in the not too distant future.

Tagged In

Microsoft Research Onlive Microsoft Nvidia Games

More from Gaming

Subscribe Today to get the latest ExtremeTech news delivered right to your inbox.
This newsletter may contain advertising, deals, or affiliate links. Subscribing to a newsletter indicates your consent to our Terms of use(Opens in a new window) and Privacy Policy. You may unsubscribe from the newsletter at any time.
Thanks for Signing Up