How a Silent Data Center Trend Could Bring Modular Computing to the Masses

I’m about to do something that will shock many of you. I’m going to talk about servers and the data center. If you follow my writing closely you know my main focus is consumer technology. But this interest in a data center trend was driven by my curiosity for anything interesting at a technical level.

Futurists have been talking about modular computing for decades. The idea that the central computing core (CPU) as well as other foundational elements of a computer like GPU, storage, memory, etc., can exist unbundled from the box has been a pipe dream for a long time. Its promise is significant. Completely scalable, near limitless resources, and always upgradeable to the latest and greatest. Not having the CPU, GPU, storage, memory, sensors, etc., stuck forever in the same box thus always having to replace the whole thing to get the latest components is a compelling idea. This is exactly what Intel’s Rack Scale design does and it is fascinating and could signal an interesting trend coming to consumer electronics.

What is new about Rack Scale from Intel is that the storage, memory, and the CPU, are all modular and can be switched, replaced, and upgraded as necessary. It is a brilliant strategic move for Intel because it allows them to sell more components more frequently. Now a server IT architect can put server infrastructure in place and simply replace any and all components as needed. You can also stack many of these components together and essentially have limitless resources by leveraging the power of resources like CPU, memory, storage, etc., that exist in other parts of the data center. It is a move to truly modularize the data center and it is a fascinating technological trend the watch. The challenge for Rack Scale architectures is the software. Most server side software assumes the computing resources (CPU, GPU, Memory, Storage, etc.,) is local and it is written as such. Supporting Rack Scale architecture in the data center takes nimble software that can recognize available computing resources wherever it exists in this modular design. This is a new twist for server software architects but it’s one that was already happening thanks to machine learning. Software to support massively parallel GPU processing stacks has been the norm in the enterprise as software has been evolving to offload as many computationally intense tasks off the CPU and on the GPU. The GPU is capable of much more parallel computing than the CPU given the GPU’s architecture design. To support full modularity in the data center, we only need continue this trend to support not just the stacking of GPUs together potentially infinitely but also other components like CPU, memory, and storage.

At the heart of this trend has been fiber optic cables which are capable of passing massive computing bits over long distances. Allowing a rack of CPUs, GPUs, memory, storage, etc., to sit farther away from the core and be stacked together for scale. While this trend is new, and still evolving, from cost, power efficiency, and computational resource standpoint it seems like something which will inevitably take over data center designs.

Bringing Modularity to Consumers
As any techie will know, the best modular computer available is a huge tower which can support a racking like article I just described but all housed in the tower itself. It looks something like this:

These rigs are typically used by gamers and can house, usually, up to three GPU cards, 4-6 memory card slots, 6 (and sometimes more) storage bays, and leave room for many fans to cool it. These systems can have nearly all the components, except the CPU, frequently upgraded and swapped out. It is the ultimate machine design for expandability and flexibility with components. The problem is roughly only about 20m people in the world buy or build systems like this. So what about the promise of flexibility and expandability for the rest of us? Or are we always going to be stuck with the devices we have which get old and slow over time.

This is where I think an interesting opportunity may emerge. While Apple is not the first to make noise about this, the idea of an external GPU that can connect to your laptop or desktop via a cable has been tried before. AMD and Nvidia have had demos of this and even some past desktop machines like this Sony Vaio Z shipped with an external GPU connected with a fiber optic cable.

The idea seemed novel at the time, but has always made sense to me from a user experience standpoint. Imagine, in the future, that on my desk at home I have my wonderful big screen 8k monitor. Sitting next to that glorious monitor is a rack of three GPUs. Stacked on that is a CPU cluster rack of an additional 10 CPUs. Next to that is an additional 20 terabytes of storage and another two terabytes of RAM. I need to do some heavy lifting design, graphic work, machine learning training, etc., so I bring my laptop upstairs, plug it into my monitor and rack of external components and all of a sudden I have available to me 100 times more computing resources than exist in my notebook.

I overexaggerated the components for the sack of the example, but hopefully you see what I mean. Apple made light of this by talking about external GPU support for macOS. The software needs to understand it can look for GPU resources outside of what is stuck on its internal motherboard. The cable need to support the transferring of computational bits back and forth, but if that comes to fruition, the idea that we can have potentially limitless computing resources starts to become possible. In Apple’s use case they pointed out how you could do high-end VR on a Mac just by using your Mac with an external display. The software looks beyond the limitation of the internal GPU and runs on the external one. Need more GPU power for better gameplay? No problem just add another one, or two, or three, or four! This is the fascinating promise of modularity coming to markets beyond just the data center and high-end gaming.

This type of system can also help consumers hold onto their notebooks even longer. Perhaps not something the OEMs love but there is still great value to the users who need machines with more computational capabilities but don’t necessarily want a huge desktop tower.

Given Apple’s slight pivot with Mac to start to think more deeply about the Pro community, I fully expect whatever they have up their sleeve with Mac Pro to take a much more modular approach, like the one I described than any pro product they have designed before. I can also guarantee this will drive other Windows OEMs in the same direction and we will see more modular designs that support component expandability within their notebook, desktop, and all in one designs.

Lastly, if we carry this further, why couldn’t even things like our smartphones or tablets benefit from modular designs? Perhaps I’ll have a need someday on my iPad to connect to an external GPU, or several, to use for some computationally intense activity that the local GPU on my iPad can’t handle. The bottom line is, moving in this direction allows for more options, and expandability in critical components for our computers than anything that has existed before. If we do go in this direction, it will open new possibilities that did not exist before, not just in the data center but in the home and office as well.