Unlocking High-Performance Storage with Supermicro SuperStorage® Scalable Servers
No-Compromise Storage Solutions to Speed Up HPC Workloads
HPC storage servers must handle massive volumes of data and deliver extreme performance to support demanding workloads, such as large-scale simulations, data-intensive scientific computations, and machine- learning applications. The data generated by these applications can be unstructured, including images, videos, and text, or structured, such as numerical data and databases.
The specific requirements for HPC storage vary depending on the application. For example, scientific simulations may require high-performance storage for large datasets and rapid data transfer. Machine learning applications may require fast access to training data and efficient storage for model checkpoints. In all cases, HPC storage systems must deliver the performance and capacity needed. High-performance computing workloads need multiple storage tiers to balance performance needs and costs.
- An All-Flash Storage tier to cache input for data-hungry application servers and deliver high bandwidth continuously to meet demand.
- A Storage Capacity tier containing all current and historical data, with ample storage capacity.
Supermicro HPC Storage Reference Architecture
Supermicro offers a broad server and storage product line, including high-performance, all-flash storage ideal for active data and high-capacity systems with spinning disks for high-capacity, low-cost capacity storage. Supermicro has developed a general scale-out storage architecture that serves the needs of HPC workloads such as manufacturing, life sciences, energy exploration, research, and financial services.
All-Flash Performance Tier
The core of the solution, the all-flash tier, holds the data actively used by applications. Typically, this tier accounts for 10 to 20 percent of overall data and utilizes all-flash storage servers to store data on a distributed, scale-out file system for various clients. It can also coordinate data tiering to the capacity-storage tier and the cloud. Using servers optimized for storage instead of more costly purpose-built appliances enables linear scaling, increasing bandwidth and the capacity to store more data as additional servers are added to the cluster.
Storage clusters in this tier must be interconnected with the highest bandwidth available networking, either 400 Gbps Ethernet or 400 Gb/s InfiniBand, to maintain the lowest possible latency. Supermicro offers several options for All-Flash performance Tier, including a 1U Petascale server and a 2U High Availability Storage Server.
Capacity Storage Tier
While the all-flash tier acts as a cache for the organization's most actively used data, it can’t just be deleted when it is no longer frequently used. It still has value and needs to be retained in storage optimized for capacity and durability instead of performance, which reduces the overhead for 80 to 90 percent of data in this category. The servers to support this mission have a high density of disk drives and comparatively low bandwidth, as well as IOPS requirements for handling data transfers in and out of the storage cluster.
Similarly, the requirements for networking are relaxed, and 100 Gbps Ethernet is an economical choice that balances cost and performance for capacity storage needs. Supermicro offers several options for Capacity Tier Storage, including High Density 60/90 Hard Disk drive-based Top Loading Storage, a 24 HDD 2U SimplyDouble storage server, and, where needed, a U.2 SSD-based Hyper Rack Mount Server.
Supermicro Reference Architecture for HPC Storage
All-Flash Tier Storage Server
Petascale 1U and 2U Storage Systems
Support high core count latest dual Intel socket and single AMD single socket
Support 1U E1 .S CV1d E3.S and 2U E3.3 TLC, QLC and CXL device and
up to 1 PB all flash in 2UOptimized thermal design with EDSFF Spec
Balanced PCie lanes for front SSO CV1d rear 10 design
Up to 30M 4KB Random Read lOPS and 230+ GB/s 128KB Sequential Read BW
The capacity and performance of the servers’ NVMe storage must be matched with appropriate network connectivity.
Each server supports multiple NVIDIA ConnectX-7 SmartNICs, each with a single NVIDIA InfiniBand NDR 400 Gb/s port. These occupy each server’s PCIe 5.0 x16 expansion slots. A pair of NDR InfiniBand switches (available as part of SSE-MQM9700-NS2F), each with 64 nonblocking ports, supports two independent data fabrics.
Capacity Storage Server
4U Cloud Scale Top Leading Storage Server
Single, Twin and JBOD configurations
Up to 2.7PB capacity in 90bay or
1.8PB capacity in 60 bay server (with 30TB HDD)Enterprise-grade serviceability
Best-in-class performance for the most demanding workloads
Not all data needs to be in the fastest, all-flash tier. Indeed, for a large amount of big data to be processed and analyzed, it can be retained on a cost-effective storage capacity tier until it is needed again.
The Data Platform Software must support migrating data to a second storage tier. It interoperates with several scale-out distributed storage platforms that can be used to support this tier. The large data sets used in HPC workloads will be stored on the capacity tier and migrated to the performance tier as needed.
Conclusion
Choosing the proper storage solution for HPC Workloads is critical and significantly impacts business outcomes for all workloads, especially High Performance Computing requirements. Selecting storage architecture by working with experts from Supermicro is a strategic business decision; it takes advantage of the complete end-to-end Rack Scale solution capabilities, including server, storage, networking, and management tools. Supermicro can customize the solution for sufficient bandwidth and low latency required to meet the application and business requirements, and provide deployment and maintenance services.
SUPERMICRO STORAGE SOLUTIONS FOR HIGH-PERFORMANCE COMPUTING
No-Compromise Storage Solutions to Speed Up HPC Workloads