Bluefin Cluster

Bluefin is an Intel-based Linux cluster running Rocks 7.0, which is based on CentOS 7.  It is located in the VUIT Data Center in the Hill building.  The cluster consists of a head node, bluefin.csb.vanderbilt.edu, and 51 compute nodes bf[1-51].csb.vanderbilt.edu.

Bluefin System Hardware

Compute Nodes   Cores  Memory (GB)  CPU                   GPU
bf[1-13,49-51]    20       128      Broadwell (2.2 GHz)   None
bf[14-20]         12        48      Nehalem (2.93 GHz)    None
bf[21-24]          8        32      Broadwell (1.7 GHz)   GeForce GTX 1080 GPUs (4X)
bf[25-48]          8        48      Nehalem (2.93 GHz)    None

Networking

The cluster network consists of private/public GigE networks and Infiniband.  The public network allows access from the University networks while the private network is exclusively used for data transfer between the compute nodes.

The head node, bluefin, is connected to the public cluster network and also has a 10-Gig Ethernet connection to DORS and ACCRE.

All compute nodes have GigE connections to the public and private cluster networks.  Compute nodes bf[1-28] are also connected to an Infiniband fabric using QDR Mellanox ConnectX Infiniband interconnects and a 36-port QDR Mellanox switch.

Job Scheduling

Please do NOT submit jobs to this system interactively. You must use the slurm workload manager.  For more information on slurm, see

http://bluefin.csb.vanderbilt.edu/roll-documentation/slurm/7.0/html/slurm.html

http://www.accre.vanderbilt.edu/?page_id=2154

The slurm queues are:

Queue            Compute Nodes   Cores  Memory (GB)
broadwell-20     bf[1-13,49-51]    20      128
nehalem-12       bf[14-20]         12       48
broadwell-8-gpu  bf[21-24]          8       32
nehalem-8        bf[25-48]          8       48

The bf[21-24] compute nodes each have four GeForce GTX 1080 GPUs.

Getting a Cluster Account

To get an account on bluefin, you first need to obtain a CSB account.  Then request a cluster account by sending an email to support@csb.vander bilt.edu or using Request Tracker.  Login to the headnode ‘bluefin’ using ssh from any CSB system or ‘bluefin.csb.vanderbilt.edu’ from other systems on campus.

Filesystems on Bluefin

For usability and convenience, your CSB home directory is network-mounted on bluefin and the bf compute nodes.  To mount disks from your workstation on the cluster, please send an email to support@csb.vanderbilt.edu or submit a ticket to Request Tracker.

Software

You may run any programs that are available under /sb/apps, in SBGrid, or your own programs as long as the program can be run in non-interactive text mode.