I wasn't there, but understood it was intended for general machine-room -- I dislike "datacentres" -- use; Wikipedia implies it was intended for ~everything...
Anyway, IB at least had lower latency than Myrinet, which is what counts in a lot of HPC. I don't remember the numbers, but I got quite close to Myrinet latency with the on-board 1GbE NICs on Sun x2200s, an un-managed switch, and Open-MX, which was interoperable with the Myrinet MX protocol. (I remember MX being slagged off by Mellanox; I don't know how similar MXM later was for IB.)
Anyway, IB at least had lower latency than Myrinet, which is what counts in a lot of HPC. I don't remember the numbers, but I got quite close to Myrinet latency with the on-board 1GbE NICs on Sun x2200s, an un-managed switch, and Open-MX, which was interoperable with the Myrinet MX protocol. (I remember MX being slagged off by Mellanox; I don't know how similar MXM later was for IB.)