Loosely Coupled Parallelism: Clusters. Context We have studied older archictures for loosely coupled parallelism, such as mesh’s, hypercubes etc, which.

  • Published on
    01-Jan-2016

  • View
    212

  • Download
    0

DESCRIPTION

Loosely Coupled Parallelism: Clusters Context We have studied older archictures for loosely coupled parallelism, such as meshâs, hypercubes etc, which require that special…

Transcript

Loosely Coupled Parallelism: Clusters Context We have studied older archictures for loosely coupled parallelism, such as meshâs, hypercubes etc, which require that special systems are built. This is also true for the switched closely coupled architectures such as Crossbar systems, Omega switched networks etc. Special builds implies cost (sometimes massive cost) An obvious idea is to take standard computers (e.g. Pentium PCâs) and standard connections between them (e.g. Ethernet links or the Internet), connect them up and devise some suitable software to deliver parallel computing at much lower cost and potentially on a very big scale. Such systems were first set up in the 1990âs (at NASA) and are now among the most important ways of achieving parallel processing power. They are called Clusters, or Clusters of Workstations (COWS) Cluster basics Clusters can be 2 or more (even 1000âs) of individual machines Can be used for true parallel processing i.e. to apply more computing power to hard problems. This is difficult â devising distributed parallel algorithms to attack a problem at the many nodes of a cluster simultaneously. Solutions will be largely problem specific, although groups of similar problems will, of course, have similar solutions Can be used for âload balancingâ, i.e. where a computer system has to service many largely independent jobs, use the cluster to âcontract out' and divide up the jobs among the nodes of the cluster. Obviously, this is a way to set up powerful servers for large numbers of clients. This kind of load balancing is easier to do than the âsingle problemâ application of the previous paragraph and it can be done with generic software tools â see later slides about this. Clusters provide very good scaling and fault tolerance. Since there is no special hardware and since all the computers at each node are the exactly the same, nodes can be added to or lost without problems â the only effect will be on the aggregate power available Small Clusters The most common way to set these up is to use Linux, along with open source software such as Mosix and OSCAR, which allows PC clusters for load balancing to be set up in quite a straightforward manner. Can be used for true parallel processing i.e. to apply more computing power to hard problems. This is difficult â devising distributed parallel algorithms to attack a problem at the many nodes of a cluster simultaneously. Solutions will be largely problem specific, although groups of similar problems will, of course, have similar solutions Can be used for âload balancingâ, i.e. where a computer system has to service many largely independent jobs, using the cluster to âcontract out' and divide up the jobs among the nodes of the cluster. Obviously, this is a way to set up powerful servers for large numbers of clients. This kind of load balancing is easier to do than the âsingle problemâ application of the previous paragraph and it can be done with generic software tools â see later slides about this. Provides very good scaling and fault tolerance. Since there is no special hardware and since all the computers at each node are the exactly the same, nodes can be added to or lost without problems â the only effect will be on the aggregate power available Big Clusters Again, these are often based on Linux. The early NASA software was named âBeowulfâ and Beowulf systems are still being developed. The top 500 supercomputer web site (www.top500.org) from the University of Mannheim now lists Linux based clusters among the worldâs biggest beasts, alongside the specially built giants from Cray, Silicon Graphics, IBM and Fujitsu (who built the Earth Simulator). In the latest Nov 2004 list, Linux clusters are at noâs 4 and 5. Top computer in this list is a special IBM 3D torus with 32,768 nodes, each of which is a complete computer+memory on a chip. No 4 is âMareNostrumâ a 2268 node Linux cluster in Barcelona, Spain. The next list is due in June 2005. The SETI@Home (search for Extra terrestrial Intelligence) project is in effect a cluster of up to 3.5 million personal computers in 226 countries. A recent estimate of the aggregate PC time devoted in this way is 800,000 years! Some details: Mosix Mosix â Multiple Computer Operating System for Unix. Extends the Linux kernal so that any standard Linux process can migrated to another node to take advantage of better resources or execute faster. The migrated process doesnât know where it is. As far as its home node is concerned, the process is running locally. Thatâs transparency for you! â no special programming is required. Migration can be automatic, each node is a master for locally created processes and a server for remote process that migrated from other nodes. There are monitoring algorithms showing what is going on where and how efficient everything is. But Mosix canât run a single process on two or more nodes at the same time, so it comes into the âload balancingâ category. Some details: OSCAR and Beolwulf OSCAR (Open Source Cluster Application Resource) clusters consist of one server node and lots of clients, all of which must have the same hardware setup. OSCAR provides an installation framework to install the necessary files and software resources among the clients. Mostly aimed at High Performance Computing (HPC). Beowulf. The best known Linux based type of cluster. Basically provides multiple machine libraries of clustering tools. The most interesting allowing PVM (Parallel Virtual Machines) and MPI (Message Passing Interface). Both these allow for true parallel attack on difficult problems, but both need users who know how to write special software to hook into PVM or MPI to take advantage of the cluster. MPI in particular is a vendor independent standard for message passing, with a large user based applications forum