HPC with Beoqulf

[Patrick Schmid]

Morning everybody!

I'm trying to build a Linux HPC based on Debian with Beowulf.
But I didn't find any information about if Beowulf is a solution for
High Availability or High Performance.

Can somebody help?


regards

[Carsten Aulbert]

Hi

If I understood it correct, the Beowulf project is just a bundle of
software to build a cluster.
So I'm referring to some software which allows me to build a Linux HPC.

Yes and no :)

You first need to identify what you want/need:

* what is the (typical) problem you want to solve?
* what software do you need for that, do you need a batch scheduler or do you
have very few users which work at the same place and share the cluster without
technical measures?
* think about the OS (Debian is a good choice here ;))
* Think about the compute hardware, you probably need a login node, execute
nodes and a file server, do you need many local cores or are the problems too
large to fit into a few nodes? Then you need to look into networking
(Infiniband or high performance Ethernet), is the software susceptible to
latency and/or bandwidth available......

You see there are MANY questions to look at first, before you even want to
start installing the machines :)

Try describing a "typical" problem, the sizes involved, that may help a lot.

Cheers

Carsten

[Joan Marc Riera Duocastella]

Hi,

Check this out :
http://svn.oscar.openclustergroup.org/trac/oscar

OSCAR allows users, regardless of their experience level with a *nix
environment, to install a Beowulf type
<http://beowulf.org/overview/faq.html> high performance computing
cluster. It also contains everything needed to administer and program
this type of HPC cluster. OSCAR's flexible package management system has
a rich set of pre-packaged applications and utilities which means you
can get up and running without laboriously installing and configuring
complex cluster administration and communication packages. It also lets
administrators create customized packages for any kind of distributed
application or utility, and to distribute those packages from an online
package repository, either on or off site.


Fundació Barcelona Media
*


Joan Marc Riera Duocastella

*


Barcelona Media - Centre d'Innovació
Av. Diagonal, 177, planta 9 08018 - BARCELONA
Telèfon [phone] Fax [phone]
www.barcelonamedia.org <http://www.barcelonamedia.org>

Post by Carsten Aulbert
Hi

If I understood it correct, the Beowulf project is just a bundle of
software to build a cluster.
So I'm referring to some software which allows me to build a Linux HPC.

Yes and no :)
* what is the (typical) problem you want to solve?
* what software do you need for that, do you need a batch scheduler or do you
have very few users which work at the same place and share the cluster without
technical measures?
* think about the OS (Debian is a good choice here ;))
* Think about the compute hardware, you probably need a login node, execute
nodes and a file server, do you need many local cores or are the problems too
large to fit into a few nodes? Then you need to look into networking
(Infiniband or high performance Ethernet), is the software susceptible to
latency and/or bandwidth available......
You see there are MANY questions to look at first, before you even want to
start installing the machines :)
Try describing a "typical" problem, the sizes involved, that may help a lot.
Cheers
Carsten

[SpiduS Okami]

Hi,

There is a good book for you to follow, it gives lots of informations on how
to build the beowulf cluster. I don't know if there is a translation to it,
because it is from a brazilian researcher. The title is "Construindo
supercomputadores com Linux", author is "Marcos Pitanga".

Also there is a quick configuration cluster, I could say it can be beowulf,
the name is Pelican. You don't need to install it to run, but there is an
option for that.

Att,
Victor.

Post by Joan Marc Riera Duocastella
Hi,
http://svn.oscar.openclustergroup.org/trac/oscar
OSCAR allows users, regardless of their experience level with a *nix
environment, to install a Beowulf type<http://beowulf.org/overview/faq.html>high performance computing cluster. It also contains everything needed to
administer and program this type of HPC cluster. OSCAR's flexible package
management system has a rich set of pre-packaged applications and utilities
which means you can get up and running without laboriously installing and
configuring complex cluster administration and communication packages. It
also lets administrators create customized packages for any kind of
distributed application or utility, and to distribute those packages from an
online package repository, either on or off site.
[image: Fundació Barcelona Media]
* Joan Marc Riera Duocastella * Barcelona Media - Centre d'Innovació
Av. Diagonal, 177, planta 9 08018 - BARCELONA
Telèfon [phone] Fax [phone]
www.barcelonamedia.org
Hi
If I understood it correct, the Beowulf project is just a bundle of
software to build a cluster.
So I'm referring to some software which allows me to build a Linux HPC.
Yes and no :)
* what is the (typical) problem you want to solve?
* what software do you need for that, do you need a batch scheduler or do you
have very few users which work at the same place and share the cluster without
technical measures?
* think about the OS (Debian is a good choice here ;))
* Think about the compute hardware, you probably need a login node, execute
nodes and a file server, do you need many local cores or are the problems too
large to fit into a few nodes? Then you need to look into networking
(Infiniband or high performance Ethernet), is the software susceptible to
latency and/or bandwidth available......
You see there are MANY questions to look at first, before you even want to
start installing the machines :)
Try describing a "typical" problem, the sizes involved, that may help a lot.
Cheers
Carsten

[Fabricio Cannini]

Hi Patrick, Hi Carsten.

What you are trying to accomplish is certainly doable, Patrick. I have
done it myself, and it sure is a great experience.
I'll quote Carsten so that you can see what i have done.

Post by Carsten Aulbert

If I understood it correct, the Beowulf project is just a bundle of
software to build a cluster.
So I'm referring to some software which allows me to build a Linux HPC.

Yes and no :)
* what is the (typical) problem you want to solve?

Molecular dynamics, DNA analysis... Most of them using MPI to run in
more than one node in parallel.

Post by Carsten Aulbert
* what software do you need for that,
do you need a batch scheduler or do you have very few users which work at the same place
and share the cluster without technical measures?

Here we are using the TORQUE scheduler
[http://www.clusterresources.com/products/torque-resource-manager.php
], which is avaiable in squeeze repos. It is an essential part of the
system, as there 100+ users to it.

Post by Carsten Aulbert
* think about the OS (Debian is a good choice here ;))

Agreed. :)
I've been using squeeze since i began researching about it (
april/2010 ), and it's been great.
All, really, all software i needed to setup a scientific computing
cluster was already packaged, just an 'aptitude install' away.
Scientific software properly you may need to compile, as we did with
all the ones used here.

Post by Carsten Aulbert
* Think about the compute hardware, you probably need a login node, execute nodes and a file server,
do you need many local cores or are the problems too large to fit into a few nodes?

A file server is nice, but if the cluster will be small you can get by
with a beefed-up login node.
Of course, as the machine grows, you will feel the need to separate
tasks. A stand-alone installation server, backup||redundant storage
server, round-robin login nodes, you name it. The cluster i setup is
very homogeneous, and it's composed by 1 login node, 1 storage node,
and 22 processing nodes, with ethernet and infiniband DDR connections.
All nodes have 2 Xeon 5400 quad-core processors each.
As Carsten said, the kind of processing node will depend on the kind
of problem that you are trying to solve. Some tasks are more naturally
parallel, then you can use a higher number of lower frequency cores ,
like AMD 6000 series. Or your problem is not so easily
parallelism-friendly, so you will need higher frequency processors, or
faster memory access if your problem is memory-bound, then you will go
for an Intel 5600 or 7500 series. There's a lot to think about only in
choosing the processors of the nodes.

Post by Carsten Aulbert
Then you need to look into networking (Infiniband or high performance Ethernet),
is the software susceptible to latency and/or bandwidth available......

My rule of thumb is:
Gigabit Ethernet for single/multi-threaded programs ran in a single
machine, with low or none communication between $output_dir and
processing node, and with output file(s) size at most in the low GB.
If your software can use more than one node ( e.g. through MPI ) for
inter-node parallelization of the task(s) , is latency-sensitive (
like CFD, FEM ) and/or the output files are really big ( tens of GB
and upwards ) , i'd go with 10Gigabit Ethernet or Infiniband.
Currenntly, infiniband is the best choice by bandwidth, latency, and
perhaps, price ( as in lower price than 10G ethernet ) , but it
demands modifications in the software to use it. 10Gigabit Ethernet,
on the other hand, doesn't, has reportedly decent latency and the
price is falling quick.

Post by Carsten Aulbert
You see there are MANY questions to look at first, before you even want to
start installing the machines :)

I went with FAI to the automated installation, which is a superb piece
of software.
You will have to spend some time learning it and tuning it to do
exactly what you want, write some scripts and such, because it is not
a "Next, Next, Finish" installation system like, say, Rocks Linux. I
liked it, it is IMHO a much more powerful solution than
redhat/centos/rocks/fedora anaconda, but YMMV.

Post by Carsten Aulbert
Try describing a "typical" problem, the sizes involved, that may help a lot.
Cheers
Carsten

[Carsten Aulbert]

Hi Patrick,

It has to be a HPC cause it's a given challenge: "Build a Linux HPC
with ethernet"

Good, and I can confirm it works and scales quite well (at least the beast we
manage here ;))

The goal is to split up the load of computing a job to multiple nodes.
So you have a script / job you need to compute, and need a fast way to
to that. That's typically it ;)
My imagination is, that you upload this script to a master node, which
splits it up and distributes the parts to its nodes.
hope you can understand :)
What do you think? Still possible to do that with Beowulf?

Sure, but it will depend how "social" your users will be, i.e. if they will
partition the resources without fighting each other.

You can use "dsh" (distributed shell) to start jobs manually on a set of
cluster nodes (or loop over ssh connections), or you can go the way to use a
"real" job scheduler (PBS, OGE, Torque, Condor, ...) depending of what you
really want/need...

You just need to grow with the expectations I guess, start small and become
bigger of time if need be.

Cheers

Carsten

[Stéphan Gorget]

On Mon, Jan 17, 2011 at 9:42 AM, Carsten Aulbert

Post by Fabricio Cannini
Hi Patrick,

It has to be a HPC cause it's a given challenge: "Build a Linux HPC
with ethernet"

Good, and I can confirm it works and scales quite well (at least the beast we
manage here ;))

The goal is to split up the load of computing a job to multiple nodes.
So you have a script / job you need to compute, and need a fast way to
to that. That's typically it ;)
My imagination is, that you upload this script to a master node, which
splits it up and distributes the parts to its nodes.
hope you can understand :)
What do you think? Still possible to do that with Beowulf?

Sure, but it will depend how "social" your users will be, i.e. if they will
partition the resources without fighting each other.
You can use "dsh" (distributed shell) to start jobs manually on a set of

clustershell (in debian sid) provide many features like file copy,
nice agregated output and it scales much more better.

Post by Fabricio Cannini
cluster nodes (or loop over ssh connections), or you can go the way to use a
"real" job scheduler (PBS, OGE, Torque, Condor, ...) depending of what you
really want/need...

Slurm is a really good choice too and it is easy to configure. The
community is active and open. It is provided in Debian too.

Post by Fabricio Cannini
You just need to grow with the expectations I guess, start small and become
bigger of time if need be.
Cheers
Carsten
--

cheers,

[Patrick Schmid]

Hi Carsten

It has to be a HPC cause it's a given challenge: "Build a Linux HPC
with ethernet"

Post by Carsten Aulbert
* what is the (typical) problem you want to solve?

The goal is to split up the load of computing a job to multiple nodes.

Post by Carsten Aulbert
* what software do you need for that, do you need a batch scheduler or do you
have very few users which work at the same place and share the cluster without
technical measures?

second one.

Post by Carsten Aulbert
* think about the OS (Debian is a good choice here ;))

OS isn't chosen yet.
But i can handle nearly every Linux distribution, so I'm free in that
point. Depends on which distribution the needed software is running.

Post by Carsten Aulbert
* Think about the compute hardware, you probably need a login node, execute
nodes and a file server, do you need many local cores or are the problems too
large to fit into a few nodes? Then you need to look into networking
(Infiniband or high performance Ethernet), is the software susceptible to
latency and/or bandwidth available......

there are 7 sun fire x4000 with ethernet (infiniband would get to expensiv).
I don't need much storage, but I could use NAS if needed.

Post by Carsten Aulbert
You see there are MANY questions to look at first, before you even want to
start installing the machines :)

I see ;)

Post by Carsten Aulbert
Try describing a "typical" problem, the sizes involved, that may help a lot.

So you have a script / job you need to compute, and need a fast way to
to that. That's typically it ;)
My imagination is, that you upload this script to a master node, which
splits it up and distributes the parts to its nodes.

hope you can understand :)

What do you think? Still possible to do that with Beowulf?

Cheers
Patrick

[Geoffrey Jacobs]

On Mon, Jan 17, 2011 at 2:24 AM, Patrick Schmid

Post by Patrick Schmid
Hi Carsten
It has to be a HPC cause it's a given challenge: "Build a Linux HPC
with ethernet"

Post by Carsten Aulbert
* what is the (typical) problem you want to solve?

The goal is to split up the load of computing a job to multiple nodes.

Post by Carsten Aulbert
* what software do you need for that, do you need a batch scheduler or do

you

Post by Carsten Aulbert
have very few users which work at the same place and share the cluster

without

Post by Carsten Aulbert
technical measures?

second one.

Post by Carsten Aulbert
* think about the OS (Debian is a good choice here ;))

OS isn't chosen yet.
But i can handle nearly every Linux distribution, so I'm free in that
point. Depends on which distribution the needed software is running.

Post by Carsten Aulbert
* Think about the compute hardware, you probably need a login node,

execute

Post by Carsten Aulbert
nodes and a file server, do you need many local cores or are the problems

too

Post by Carsten Aulbert
large to fit into a few nodes? Then you need to look into networking
(Infiniband or high performance Ethernet), is the software susceptible to
latency and/or bandwidth available......

there are 7 sun fire x4000 with ethernet (infiniband would get to expensiv).
I don't need much storage, but I could use NAS if needed.

Post by Carsten Aulbert
You see there are MANY questions to look at first, before you even want

to

Post by Carsten Aulbert
start installing the machines :)

I see ;)

Post by Carsten Aulbert
Try describing a "typical" problem, the sizes involved, that may help a

lot.
So you have a script / job you need to compute, and need a fast way to
to that. That's typically it ;)
My imagination is, that you upload this script to a master node, which
splits it up and distributes the parts to its nodes.
hope you can understand :)
What do you think? Still possible to do that with Beowulf?
Cheers
Patrick

Well, auto partitioning like that isn't the way things are done, even with
vector machines. The computation needs to be split manually. There are some
dandy tools to help and some fine prepackaged applications you might be able
to use. What is your experience in computing, and what kind of expertise do
you have access to?

[Patrick Schmid]

Hi Carsten

Thanks for your reply.

I'm not sure if you refer to some "Beowulf software" or to the original
concept described here: http://www.beowulf.org/overview/index.html

If I understood it correct, the Beowulf project is just a bundle of
software to build a cluster.
So I'm referring to some software which allows me to build a Linux HPC.

cheers

[Carsten Aulbert]

H Patrick

Post by Patrick Schmid
I'm trying to build a Linux HPC based on Debian with Beowulf.
But I didn't find any information about if Beowulf is a solution for
High Availability or High Performance.
Can somebody help?

I'm not sure if you refer to some "Beowulf software" or to the original
concept described here: http://www.beowulf.org/overview/index.html

Can you elaborate what you mean?

Cheers

Carsten

[a***@libero.it]

I suppose you refer beowulf means you refer a trasparent distributed processes
managment system.

Any Linux distribution supporting beowulf can be the point of a good start to
build a Linux based HPC cluster.

In order to build a performante architecture solution to HPC cluster, the best
strategy is to elaborate a combined

solution using high fast interconnection system and a reliable cluster
resource management systems: beowulf is

only a distributed processes managment system.

Beowulf is not a High Availability HPC system, is not a high network
interconnect system, is not a is not a Cluster

Management system, is not a File system managment, is not a scheduling of
parallel programs, is not a inter

processor communication.

You can start from beowulf software and add code to reach the above
functionalities.

Antonio Dima
www.clusterarch.it

----Messaggio originale----
Data: 14/01/2011 9.04
Ogg: HPC with Beoqulf
Morning everybody!
I'm trying to build a Linux HPC based on Debian with Beowulf.
But I didn't find any information about if Beowulf is a solution for
High Availability or High Performance.
Can somebody help?
regards
--
Patrick Schmid
--

org

Archive: http://lists.debian.

org/***@mail.gmail.com