On Sunday, August 28, 2005, at 11:42 PM, Joe Landman wrote: > > We haven't had much luck when pushing SMC and other lower end switches > very hard. We have been using (and specing out) HP Procurves for the > last few years for smaller clusters (through about 64 nodes). > > It sounds like your network design is also a tree, not a fat tree, but > a basic tree. This could explain the collisions (limited consumable > resource). We usually spec this design for a maximum of 16 CPUs in > moderate IO situations. Thanks for the reply. The SMC8648T , at least on paper, looks good (trunking, vlans, 88Gbs switch fabric, jumbo frames etc) though it does require rebooting with great frequency which is not something I'm accustomed to seeing. The problem is related to job load which in turn exacerbates underlying problems with NFS. We are pursuing more disks to improve performance and implementing tuning measures for NFS but I'm not convinced that our setup is so far out of band that a near daily reboot of the SMCs should be necessary.