Plan:
640 dual G5 Xserve cluster nodes with 100mbit ethernet and Myrinet high-speed interconnect to each node.
5 head nodes for job submission.
At least 2 file server nodes for shared network file systems to be mounted on all nodes and to act at Netinfo or LDAP system database manager for all nodes.
19 48-port Cisco Catalyst 2950T 10/100mbit switches with 1000mbit uplinks to core switch.
1 Cisco Catalyst 6506 Layer 3 switch at core with 720gb/sec supervisor module, and one 48-port 10/100/1000 line card for gigabit connection to rack switches and head nodes.
2 Xserve RAIDs and fiber channel switch, providing a total of 7TB of raw storage.
22 APC Netshelter VX 42u racks, each with 2 42-outlet, 48Amp PDUs.
Room layout and space usage to allow for possible future addition of 640 additional nodes, to create a 1,280-node Xserve cluster.
Statistics:
1,280 G5 processors in 640 cluster nodes
44 16Amp x 3phase circuits for rack power
6 additional 30Amp circuits for core switch and head rack power
Over 2,100 total Amps of new circuitry in cluster room
Over 550,000 BTU/hr cooling load, over 45 tons of new cooling capacity in cluster room
Current Status:
6/29/04 Xserves, racks, and various other equipment began arriving and are being placed in two rooms on the second floor that we were able to allocate for our equipment.
7/2/04 Physical relocation of old cluster to new home on 4th floor of Siebel Center.
7/7/04 All networking equipment has arrived, except for core switch.
7/16/04 Assembly of most of head node rack, and a cluster node rack with 4 nodes completed for onsite work by Apple with Mike Campbell.
7/21/04 Demolition of cluster room begins.
7/21/04 Mike Bombich (Apple), Mike Campbell, and system managers work together to determine directory services and imaging strategy for cluster nodes. 4 nodes built, connected to network and myrinet, and mounting file systems from head node.
9/20/04 Work begins on temporary installation of 64 node cluster in temporary room
9/23/04 64 nodes unpacked and racked
9/28/04 64 node cluster complete, networked and ready for power beginning 10/4/04
10/4/04 64 node mini-cluster running and file system setup and testing started
11/11/04 Completing tests to use Intel x86 Linux servers to serve NFS instead of Xserves due to poor OSX NFS server performance.
11/22/04 Final server room construction substantially complete. Moved mini-cluster into server room as well as all empty racks for positioning of XDV overhead cooling units.
11/29/04 Begin physical assembly of rest of the cluster