Apple and Facebook Flash Forward to Computer Memory of the Future

If you hire a plane, you can fly over the massive data center Apple operates in the woodlands of North Carolina, snapping some distant photos of the 500,000-square-foot facility that drives the company's iCloud web services. And if you're on foot, you can get a little closer. You might even sneak a peek at the solar farm or the biogas plant that helps power this internet engine room. But Apple won't let you inside the building -- and it won't tell you what you might find there.

It would be nice to know. Like Google and Amazon, Apple delivers web services to hundreds of millions of people across the globe -- at last count, iCloud served over 250 million souls -- and that requires a whole new breed of hardware and software, stuff that's far more efficient than the gear inside most data centers. You can think of this as the technology of tomorrow. As the web continues to grow, the tech used by the Apples and the Googles will trickle down to the rest of the world. In many cases, it already has.

"The internet giants are the harbinger," says David Floyer, a longtime analyst in the data center world who now runs a tech research house called Wikibon.

What we do know is that Apple is spending mountains of money on a new breed of hardware device from a company called Fusion-io. As a public company, Fusion-io is required to disclose information about customers that account for an usually large portion of its revenue, and with its latest annual report, the Salt Lake City outfit reveals that in 2012, at least 25 percent of its revenue -- $89.8 million -- came from Apple. That's just one figure, from just one company. But it serves as a sign post, showing you where the modern data center is headed.

Inside a data center like the one Apple operates in Maiden, North Carolina, you'll find thousands of computer servers. Fusion-io makes a slim card that slots inside these machines, and it's packed with hundreds of gigabytes of flash memory, the same stuff that holds all the software and the data on your smartphone. You can think of this card as a much-needed replacement for the good old-fashioned hard disk that typically sits inside a server. Much like a hard disk, it stores information. But it doesn't have any moving parts, which means it's generally more reliable. It consumes less power. And it lets you read and write data far more quickly.

But that's only one way to think about it. The same card can also act like a beefed-up version of a server's main memory subsystem -- the place where the central processor temporarily caches data it needs quick access to. You see, today's super-fast processors have outstripped not only the hard disk, but main memory -- the hard disk is too slow, the memory too small -- and with its flash cards, Fusion-io aims to remove both bottlenecks.

"You can make it look like traditional storage if you want to, but it can also give you the appearance of more memory inside a system," says Fusion-io CEO David Flynn, the engineer who founded the company in 2006, alongside a serial entrepreneur named Rick White. "We called it Fusion-io because it was a fusion of memory and storage. They weren't two separate things."

The end result is that an outfit like Apple can more efficiently handle all the requests streaming into its data center from across the internet. It can deliver data faster, and it can do so with fewer servers -- something that's vitally important when you're running such an enormous operation. That's why Apple is spending so much with Fusion-io -- and it's why many others are moving in the same direction. In 2012, Facebook spent even more with the flash outfit than Apple did: $107.79 million. All told, the two internet giants have spent nearly a half billion dollars with Fusion-io over the years, and smaller but growing operations like Salesforce.com are using these cards as well.

Fusion-io has been so successful inside these massive data centers, it has spawned an army of imitators. Just last week, tech giant EMC uncloaked a new set of flash cards along the same lines, as did Silicon Valley startup Violin Memory. Another startup, Virident Systems, is building similar cards, and Seagate will soon start hawking these Virident cards to big-name web operations and other businesses. Yes, Seagate, one of the world's biggest hard drive makers.

Seagate already sells hard disks directly to the big name web players -- including Apple, Facebook, Google, Microsoft, and Amazon -- and it will continue to do so. But it sees where the data center is moving. "There's now a blurring between the storage world and the memory world," says Gary Gentry, who oversees the new flash hardware operation at Seagate. "People have been enlightened by Fusion-io."

The trend is driven by more than just one company. Google also uses flash inside its servers -- as we heard last summer from Urs Hölzle, the man who oversees Google’s worldwide network of data centers -- and according to Silicon Valley scuttlebutt, the search giant builds its own flash cards that look somewhat like Fusion-io devices. Microsoft tells us it's using similar cards in its data centers, and it very much sees flash as the future of server memory.

Kushagra Vaid, Microsoft's general manger of server engineering, points out that flash can further revamp the server memory subsystem because it's non-volatile, meaning that when you power off or reboot the server, it doesn't lose its data. Standard memory does. As researchers explore other options for overhauling server memory, Vaid says "the innovation happening on the non-volatile memory front" is the most interesting of possibilities.

The Bad Idea That Spawned Fusion-io

If you know anything about Fusion-io, you know that its chief scientist is Steve Wozniak, the man who founded Apple alongside high-school friend Steve Jobs. Woz, as he's affectionately known, serves as a kind of walking PR campaign for Fusion-io's new breed of memory, but the idea wasn't his. It sprang from Rick White and David Flynn.

Before they founded Fusion-io, White and Flynn worked together at a company called Realm Systems. Realm sold a computer that fit into your pocket. It was designed to take your online identity anywhere -- you would plug it into another computer and identify yourself with a fingerprint scan or the like, and it would instantly log into all your software applications -- but the thing never really took off, in part because the company's funding dried up.

As they brain-stormed ideas for a new company, White suggested they repurpose the processors inside those tiny Realm machines. If they wired hundreds of them together, he told Flynn, they could create a new kind of low-power supercomputer capable of running massive applications inside the data center. Flynn, an engineer who had once built supercomputers for the likes of the Lawrence Livermore National Labs, told White that was a really bad idea -- "He reminded me that processors on those tiny mobile computers couldn't do floating point," White remembers, referring to the complex calculations typically handled by server chips -- but it sparked another.

Those tiny Realm computers also included flash. They used flash in lieu of hard drives, and as a kind of supplement for the DRAM that made up their memory subsystems. It wouldn't make sense to wire all those tiny processors together, Flynn realized, but you could remake big servers so that they looked more like the Realm machines. You could remake big servers with flash.

Many companies were already offering solid-state drives, or SSDs. These are flash devices designed to mimic hard drives. They look like hard drives, and they plug into the same connectors. But Flynn aimed for something different. He wanted to build a flash device that could also mimic a server's memory subsystem. "Having worked with supercomputers," Flynn says, "I knew that the biggest problem with these big systems was the lack of memory."

Today's server processors, you see, are designed to use only so much DRAM. Theoretically, this limit is 4 terabytes, aka 4,000 gigabytes, but most servers are equipped with far less. Flynn's idea was to build what amounted to a second memory subsystem -- one that could house many more terabytes of data -- and the result was the Fusion-io flash card. It didn't plug into a hard drive controller. It plugged straight into the PCI Express bus, or PCIe bus, that shuttles information across a server. In other words, this card sat closer to the processor. It bypassed all the extra hardware and protocols needed to operate a hard disk.

In hindsight, this seems like an obvious thing to do. But it was very much a departure from the norm. It's not just that companies were accustomed to using just hard drives and DRAM inside the server. They stored most of their data outside the server, on vast arrays of hard disks -- big-beefy systems called storage area networks, or SANs. Packing its card with such enormous amounts of flash, Fusion-io sought to replace these massive appliances as well. It called the card a "SAN in your hand."

Facebook and the New Memory

Fusion-io uncloaked its cards in the fall 2007, and its first big customer was Facebook. David Flynn was friends Paul Saab, a Facebook software developer and a well-known contributor to the FreeBSD open source operating system, and the relationship between the two companies progressed from there. Nowadays, Fusion-io will even design new cards in tandem with Facebook engineers.

Apple came later. The natural assumption is that this relationship was driven by Steve Wozniak -- who joined Fusion-io as chief scientist in late 2008 -- but according to Fusion-io, this wasn't the case. In typical fashion, Apple didn't respond to a request to discuss its use of flash cards -- and Fusion-io demurred -- but Facebook is another matter. Unlike Apple, the social networking giant is willing to open the curtain on the hardware and software used inside its data centers, and its new-age systems give you a pretty good idea of how these flash cards can reshape the server.

Initially, Facebook used the cards in the machines that run its database software. This is where companies typically use flash SSDs as well. Basically, the cards replaced hard drives. Housing its database software on flash rather than hard disks, Facebook can significantly speed the delivery of status updates and comments and other social networking chatter.

"When a request gets to the database tier, we want to be able to serve it up really, really quickly," Facebook's Frank Frankovsky once told us. "It’s a significant improvement on that overall round-trip time to the user." What's more, these flash devices can reduce hassle. As Frankovsky says, a hard disk breaks down more than any other device in the data center. Today, Facebook even boots its database servers from these flash cards.

But Facebook is now moving this sort of flash card into other machines, and it's replacing memory as well as hard drives. Traditionally, Facebook operated another tier of servers that ran something called Memcached. These machines would, well, cache data in memory. The idea was to provide even quicker access to the stuff that users looked at the most. With this data, you wouldn't even have to visit the database tier. This worked well enough, but Facebook has now cranked the dial on these servers as well, creating a new software platform that caches data on flash cards rather than main memory.

With this new platform -- known as McDipper -- the company can cache much larger amounts of data. "Flash provides up to 20 times the capacity per server and still supports tens of thousands of operations per second," the company said in a recent blog post. "It was the obvious answer to this problem."

This sort of setup is still significantly slower than main memory, says Microsoft's Kushagra Vaid. But it's fast enough, and it consumes less power. When you reach the size of a Microsoft or a Facebook or an Apple, keeping power down is just as important as anything else.

The point is that flash gives you more options -- and in the long run, Vaid says, these options will only expand. "It creates this new platform architecture a couple of years down the road," he explains, "where the application can now be more judicious about how to balance the performance and the power -- by choosing the appropriate balance between system memory and non-volatile memory."

Uh, We're a Software Company

But, yes, Facebook built McDipper from scratch. And that too shows you how much these flash cards are changing the data center. Today's software isn't designed to run on flash cards. You're forced to write new software.

That's why Fusion-io has engineers working inside the data centers run its largest customers, including Apple and Facebook. And it's why the company employs several software developers known for their work on the Linux kernel, the software at the heart of the open source operating system that runs most machines inside the big internet data centers. This includes Chris Mason, who oversees a Linux file system called Btrfs (pronounced "butter fs") and Jens Axboe, who built several tools related to data input and output and other tasks inside the Linux kernel.

"The storage devices that are coming out now are almost nothing like the storage devices that people have been using for the past few decades. They break a whole bunch of rules and work in very different ways and do a whole lot of things that just weren't possible before," says Mason. "But you have to change the core software to enable this."

The aim, explains Mason and CEO David Flynn, is to tune Linux and other software to get the most out of these cards, but also to make it easier for companies to build their own software. They company now offers a software development kit, or SDK, for building tools atop the cards. In the end, says Flynn, Fusion-io is a much a software company as a hardware company.

This is borne out by the company's recent contribution to the Open Compute Project, a Facebook-led effort to create new hardware designs for the data center -- and share those designs with the rest of the world. This winter, Fusion-io open sourced the design of its flash card, including everything but the controller that drives the card. The aim is to feed the creation of still more flash devices that can run its new breed of software.

The hope, Flynn says, is that computer makers like Quanta and Wistron will build flash straight into the server motherboard, pushing it even closer to the processor. After all, that's where the data center is headed.

Robert McMillan contributed to this story.