IBM is offering a potentially powerful incentive in its attempts to entice organizations to move supercomputing jobs to the cloud: a high-speed network communications link called InfiniBand.
“Companies are embracing cloud for standard workloads, but they are also looking to move their high-performance workloads to the cloud, or looking to augment capacity they have in-house,” said Marc Jones, IBM SoftLayer vice president of product innovation. “A lot of these companies will have workloads that are extremely demanding of performance.”
InfiniBand may help sway their decision to go to the cloud, Jones said. SoftLayer, the IBM subsidiary running the company’s cloud infrastructure, will offer the option of using InfiniBand to connect bare metal servers within its data centers by the end of the year.
The InfiniBand option may be ideal for large computing jobs that require dozens, hundreds or even thousand of servers, or nodes, working together to tackle a single problem. InfiniBand provides a very high rate of data throughput among nodes, as well as a low rate of latency.
Latency is the time it takes for a node to respond that it has received a bit of data from another node. Low latency is valuable in applications where different nodes must respond to one another as quickly as possible, due to the immense amount of communication that takes place among all of them.
InfiniBand offers the highest throughput for any interconnect. Each InfiniBand can transport up to 56Gbps, more than the 40Gbps that industry-standard Gigabyte ethernet links now offer.
As a result of these features, InfiniBand may make IBM cloud services suitable for jobs that traditionally have been done by supercomputers.
Computer-aided research in life sciences and genomics often runs on supercomputers, as do large jobs in engineering, financial services and electronics design.
Such supercomputers, also called high-performance computing (HPC) systems, are often run by government agencies or academic institutions, and typically have backlogs of jobs waiting in a queue that can span weeks or months.
A cloud service that is able to replicate a supercomputer almost instantly could offer an alternative to the large up-front costs of buying or updating a supercomputer, and eliminate the need for a computer job to wait in a queue, Jones said.
InfiniBand has been a staple in supercomputers.
In the latest twice-yearly compilation of the Top 500 most powerful supercomputers, released in June, 222 systems ran InfiniBand as the primary interconnect, with Gigabit ethernet running a distant second, on only 127 systems.
IBM is not the only cloud computing provider to disrupt the supercomputing market.
Like IBM, Amazon Web Services has also seen an increasing number of HPC jobs being run on AWS, noted AWS product marketing executive Paul Duffy, in an earlier interview. One AWS partner, Cycle Computing, offers software to lash together many AWS computational nodes so they can run together as a single HPC machine.
SoftLayer started to court supercomputer users in 2012 by offering customers the ability to add graphical processing units to their bare metal servers.
GPUs are favored among supercomputer users because they can process large blocks of data in parallel, speeding job completion while cutting power costs.
Beyond supercomputing, IBM also sees InfiniBand as appealing for other workloads, Jones said. Media companies might find value in the service for duties such as large-scale animation jobs or video format transcoding. Oil and gas companies, looking for more sources of fuel, have a lot of data to analyze as well.
“We’re also seeing enterprises that have significant requirements for analyzing large data sets, and they want to do that faster and with lower latency,” Jones said.
IBM SoftLayer has a number of other tools to help manage these jobs. Large amounts of data can be stored in the company’s Elastic Storage service. IBM also has some software for managing complex multinode workloads, IBM Platform LSF or Platform Symphony.
The supercomputing practice of IBM will also use the SoftLayer cloud as an option for its clients.
HPC is one of many specific markets the company hopes to move to the cloud.
IBM has spent considerable resources in ramping up its cloud services this year.
After its US$2 billion acquisition of SoftLayer last year, the company started investing $1.2 billion in building out data centers, working to bring the total to 40 around the globe by 2015. IBM is also investing an additional $1 billion to round out its Bluemix portfolio of services, and now offers over 100 software-based services from its cloud.
Pricing for the InfiniBand option has not been established, though Jones said it will be a paid option on SoftLayer’s service for running bare metal servers, which is billed on a monthly basis.