Home > Products > Interfaces > InfiniBand Controller (IB-381)

InfiniBand Controller (IB-381)

QDR InfiniBand interface for RamSan® products

QDR InfiniBand is a high bandwidth, low latency interface option for Texas Memory Systems RamSan® storage devices.

InfiniBand is a high-speed, low-latency network protocol that provides fast and reliable interconnections between servers, storage, and other devices.  Through the use of Remote Direct Memory Access (RDMA), InfiniBand enables data transfers directly between the system memory of connected devices without interrupting the system processors. This results in lower latencies and lower CPU utilization than other network protocols. The flexibility and high data throughput of InfiniBand allow it to consolidate both traditional Ethernet and SAN infrastructures into one network. InfiniBand is widely used in high-performance computing and Oracle grid computing environments and is making steady inroads into commercial applications.

The IB-381 InfiniBand storage interface from Texas Memory Systems resides within a flash-based RamSan® storage system and enables the direct connection of the system to other InfiniBand devices and networks. The IB-381 is basically a system on a board, built around the Mellanox ConnectX-2 chip. An embedded PowerPC processor on the IB-381 runs a custom real-time OS to handle control and InfiniBand protocols for the interface.  The IB-381 achieves high performance by directing all data to bypass the processor’s memory and go directly to the custom storage hardware. This allows the IB-381 to achieve near line speeds without the requirement for a high-performance (and power-hungry) onboard processor. And because the IB-381 is using the popular ConnectX-2 chip, its interoperability with other popular InfiniBand equipment has been proven in the field.

Features of the IB-381 InfiniBand interface:

  • Two 4x IB channels per interface
  • QSFP connector type, active cables supported
  • Supports QDR (40Gb), DDR (20Gb), and SDR (10Gb) speeds
  • Supports the SCSI RDMA Protocol (SRP)
  • Supports up to 440 simultaneous connections (Queue Pairs) per interface
  • Each interface can supply up to 3GB/s of bandwidth and 200,000 4k IOPs
  • Interoperable with all major InfiniBand Host Channel Adapters, switches, virtualization devices, and operating systems

InfiniBand Performance in RamSan-630

 

IB Ports Direction *Block Size BW (GB/s) IOPs
1 2 Read 8M 3.0 10k
1 2 Read 256k 2.8 55k
1 2 Read 64k 2.5 78k
1 2 Read 8k 1.3 164k
1 2 Read 4k 0.8 200k

 

1 2 Write 8M 2.8 43k
1 2 Write 256k 2.8 50k
1 2 Write 64k 2.6 69k
1 2 Write 8k 1.3 158k
1 2 Write 4k 0.8 194k

 

5 10 Read 8M 10.3 27k
5 10 Read 256k 10.4 224k
5 10 Read 64k 10.6 296k
5 10 Read 8k 6.6 807k
5 10 Read 4k 4.1 993k

 

5 10 Write 8M 9.9 163k
5 10 Write 256k 9.7 212k
5 10 Write 64k 9.6 294k
5 10 Write 8k 6.4 777k
5 10 Write 4k 4.0 965k

* Actual Block Size used by Linux and the IB protocol may be less