Computing Resources


3Mares

The research activities of the group are supported by an important computing infrastructure, which has been deployed and evolved for almost 20 years by members of the group. Our first steps were carried out “share-memory” cc-NUMA supercomputers, such as SGI Power Challenge, SGI Origin 3000 & SGI Altix series servers mainly. Since then, our computing resources has focus about distributed systems, as cluster HPC/HTC systems. All our actual infrastructure is installed and on running (active status) in 3Mares Data Center, one of the main computing infrastructures centers in the Cantabria University, especially about HP/HT computing. This Data Center was built in 2010 and is fully operative since then.
For the correct and safe running of the computing and networking equipment, that center has the following systems:

  • Efficient cooling system
  • Central and distributed UPS systems
  • Smart power control systems
  • Smart environment control system
  • Fire prevention system
  • Access control system
Our main computing infrastructure is focus in the calderon HPC cluster, a multi-computer nowadays composed by around 80 high performance computers, Infiniband and 1/10G Ethernet networks, shared storage, etc.

3Mares

CLUSTER Docs

Cluster user’s Manuals

OpenGE (SGE EE)

Development


CLUSTER HPC Monitor

Ganglia monitoring system


CLUSTER HPC Sumary

Running nodes

# g3
Model:     HP Proliant DL 145 2º Generación bi-procesador
Processor: 2 AMD Opteron 275 dual core
Memory:    8 GB 
Hard Disk: 80 GB SATA
Network:  2 Gigabit Ethernet	

Nodes: 20
Total Cores: 40 
# g4/g5
Model:     HP Proliant DL 160 5ª Generación
Processor: 2 INTEL Xeon 5472 quad core
Memory:    16 GB
Hard Disk: 160 GB SATA
Network:   2 Gigabit Ethernet

Nodes:       24
Total Cores: 192
# g6
Enclosure: 5 SUPERMICRO SuperServer 6026TT-HT
Model:     4 Super X8DTT-HF+ 
Processor: 2 INTEL Xeon X5650 six-core (HT)
Memory:    24/48 GB
Hard Disk: 250 GB SATA
Network:   2 Gigabit Ethernet

Nodes:       20
Total Cores: 480 (HT)
# g7 (gfast/ghuge)
Enclosure: 5 SUPERMICRO SuperServer 6026TT-HT
Model:     4 Super X8DTT-HF+ 
Processor: 2 INTEL Xeon E5645 six-core (HT)
Memory:    48/54 GB
Hard Disk: 250 GB SAS
Network:   2 Gigabit Ethernet

Nodes:       20
Total Cores: 480 (HT)
# fe-ts
Model:     Supermicro SYS-6016GT-TF-TC2 
Processor: 2 INTEL Xeon 5504 quad core 
GPUs:      2x Nvidia Tesla C1060 GPU Cards
Memory:    10 GB
Hard Disk: 250 GB SAS
Network:   2 Gigabit Ethernet

Nodes:       1
Total Cores: 8 (HT)
# Sun T2
Model:     Sun UltraSPRAC T5120
Processor: 1 Sun Niagara
Memory:    4 GB
Hard Disk: 4x 146 GB SAS
Network:   4 Gigabit Ethernet

Nodes:       1
Total Cores: 4 (8 threads concurrently per core)
# Sun T1
Model:     Sun UltraSPRAC T1
Processor: 1 Sun Niagara
Memory:    4 GB
Hard Disk: 4x 146 GB SAS
Network:   4 Gigabit Ethernet

Nodes:       1
Total Cores: 4 (8 threads concurrently per core)

Compilation nodes

# XEN Container
Model:     HP Proliant DL 160 5ª Generación
Processor: 2 INTEL Xeon 5472 quad core
Memory:    16 GB
Hard Disk: 160 GB SATA
Network:   2 Gigabit Ethernet

Nodes:       2
Total Cores: 16 

Frontend nodes

# XEN Container
Model:    HP Proliant DL 170h 6ª Generación
Processor: 2 INTEL Xeon X5550 quad core
Memory:    24 GB
Hard Disk: 160 GB SATA
Network:   2 Gigabit Ethernet

Nodes:       2
Total Cores: 16 

TOTAL Cores:      ~ 768
TOTAL Cores (HT): ~ 1436
TOTAL Memory:     ~ 8 TB
TOTAL Hard Disk:  ~ 17 TB

TOTAL Nodes:        93

STORE Systems

AFS

# MSA20
Model:       HP MSA20 backplane
Driver:      HP SmartArray 6404 UltraSCSI   
Hard Disk:   12x 500 GB SATA 
Connections: UltraSCSI

Security:    RAID 6
# MSA70
Model:       HP MSA70 backplane
Driver:      HP SmartArray P800 UltraSCSI
Hard Disk:   25x 300 GB SAS
Connections: SAS

Security:    RAID 6
# SUPERMICRO 1
Enclosure:   SuperMicro Chassis SC847E backplane
Processor:   2x Inter SANDY BRIDGE 6C E5-2620 2.0G
Memory:	     32 GB
Driver:      Expander supports SAS2 - 6Gb/s
Hard Disk:   36x 1TB SAS 
Connections: PCI
Network:     10G PCI-E 

Security (Store):    RAID Z (ZFS software)

Lustre

# SUPERMICRO 2
Enclosure:   SuperMicro Chassis SC847E backplane
Processor:   2x Inter SANDY BRIDGE 6C E5-2620 2.0G
Memory:      32 GB
Driver:      Expander supports SAS2 - 6Gb/s
Hard Disk:   36x 1TB SAS
Connections: PCI
Network:     10G PCI-E

Security (Store):    RAID Z (ZFS software)

SERVICES nodes

XEN domains mainly

1x “enclosure” HP DL1000 
  --> 4x nodos HP Proliant DL 170h 6ª Generación biprocesador con INTEL Xeon X5550 quad core.
2x nodos HP Proliant DL 160 5ª Generación bi-procesador con INTEL Xeon 5472 quad core.
8x Xen domains in charge of managing several service roles:
  --> Web service 
  --> Mail services
  --> Monitor services (Ganglia/Nagios3) 
  --> Imgenes (servers) and cloning services
  --> GitLab repository
  --> Docker repository
  --> Software floated licenses
  --> Information services (Kerberos/LDAP)
  --> Frontends (load balancers) for the HPC cluster calderon

NETWORKING

Ethetnet network based in star topology and dual 10Gb port
Infiniband network for only HPC jobs, implamented by Mellanox switch with 24 dual 10 Gb ports