I’ve worked in the data management and storage business since 1999. One of the things that struck me early on was how supply-side and proprietary the industry was. Storage vendors often pushed extraneous hardware or over-provisioned configurations. Systems were proprietary and didn’t interop. You could not replicate data from vendor A’s system to vendor B’s system and the worst part was that customers often were burdened carrying the tab for years, integrating and making due with supply-side configurations.
For example, one customer I worked with described how a storage vendor dropped the gear off at the loading dock and walked away. The customer was stuck figuring out how to put the pieces together.
In HPC, especially for larger deployments, the complexity of the storage solution often gets overlooked. One-size-fits all storage solutions sold by traditional storage vendors typically do not fit. In the case of parallel file systems (pick your flavor), there’s a ton of technology that needs to come together: networking fabrics, servers, storage, and cables—and software across it all.
This is where Cray Inc. has excelled in storage—delivering open storage solutions for customers. Cray has a track record of going the extra mile for customers. For years, Cray’s custom engineering team has developed these storage systems for supercomputers and today, Cray has an entire storage business delivering prescribed storage architectures and solutions—for HPC and Big Data Linux clusters.
Cray Inc. delivers storage solutions that are:
- Consistent: Consistency becomes important when the customer goes to scale the system. As the storage system is scaled, it should operate in a consistent way—and not require a redesign. Cray’s expertise in system scalability ensures storage systems work consistently.
- Open to any HPC Linux cluster: Through an offering called Cray Cluster Connect™, Cray storage solutions are validated for CentOS, SLES, Red Hat®, and of course Cray Linux Environment.
- Complete: The storage system delivered—all of the components from drives to clients—must be fully qualified and configured in Cray -validated and supported configurations.
What’s the difference between a storage system for a supercomputer and a cluster? This depends on the vendor delivering the storage solution. Cray provides the same storage configurations for cluster systems as we do for supercomputers. In a Cray® supercomputer, there are options for optimizations for fine grained routing selection at scale, and offloading aspects of Lustre™ to nodes in a supercomputer.
Now both clusters and HPC supercomputers benefit from Cray Cluster Connect system, where a common file system can be shared and performance scaled across a diversity of Linux® clients. The interconnect is InfiniBand® and optionally 10/40 GbE for some clusters.
At the end of the day, storage for cluster systems shouldn’t be overlooked. To get the most out of the cluster often requires scalability, interoperability, and trusted knowledge that Cray is well suited to provide.
Jason Goodman, Cray Storage & Data Management