History and Motivation
Whats parallel computing?
is a multiple-processor computer system suppmting
parallel programming.
multicomputer
centralized mUltiprocessor
Reduce the time needed
to solve a single computational problem.
Why should I program using MPI and OpenMP?
MPI (Message Passing Interface) is a standard specification for messagepassing
lihmries
MPI is a perfectly satisfactory way for processors in different SMPs to cornrnullicate
with each other, OpenMP is a better way for processors within a single SMP to interact
MODERN SCIENTIFIC METHOD
Explain the phenomenon and designs an expellment to test that theory
Quantum chemistry, statistical mechanics, and relativistic physics
Cosmology and astropbysics
Computational fluid dynamics and turbulence
Materials design and supercond.uctivity
Biology, pharmacology, genome sequencing, genetic engineering, protein
folding, enzyme activity, and cell modeling
Medicine, and modeling of human organs and bones
Global weather and environmental modeling
EVOLUTION OF SUPERCOMPUTING
Supercomputers are the most powerful computers that can be built.
Today's supercomputers
are more than a billion times faster, able to petfonn trillions of floating point
operations per second
In 1976
supercomputer meant a Cray-l, a single-CPU computer with a high ·petformance
pipelined Vector processor wnnected to a lIigh-performance memory system.
Supercomputers have typically cost $10 million or more.
Today, supercomput£r means a parallel computer with thousand~ of CPUs.
Even mainframe computers
are being constructed out of microprocessors.
PROGRAMM1NG PARALLEL COMPUTERS
Extend a Compiler
Extend a Sequential Programming Language
Add a Parallel Programming Layer
Create a Parallel Language
DATA CLUSTERING
Data Clostering:
1. input N documents
2. For each of the N documenLI .generate a D-dirnensional vector indicating how well il cover the
D different topics
yellow = facts, examples
3. Choose the K initial cluster cenlers using II rnndom sample
1. Repeat the following steps for I iteratiolis or until he performance function converges. whichver
come firts
1. OUlPUT K centers
Subtema
Multidimensional data clustering is an important tool for data mining.
SEEKING CONCURRENCY
Data Parallelism
for i <- 0 to 99 do
a[i] <- b[i] + e[i]
endfor
Functional Parallelism
a<-2
17 <- 3
m <- (a +b)/2
s <- (a2 + 172
)/2
v<- s ~ m2
MODERN PARALLEL COMPUTERS
The Cosmic Cube
, a parallel computer constructed out of 64 Intel 8086 microprocessors
[34]
the potential for microprocessor-based parallel computing
Intel Corporation had donated much of the hardware for the Cosmic Cube.
Commercial Parallel_Computers
Some commercial parallel computers had support for higher-level parallel
programming languages and debuggers
Experimenal parallel computers were less expensive
Beowulf
The dynamic PC marketplace set the
stage for the next breakthrough in parallel computing
Their system, named Beowulf, contained
16 Intel DX4 processors connected by multiple to Mbitfsec Ethernet links
Beowulf is all example of a system wnstlUcted out of commodity, off-theshelf
(COTS) componenls.
Advanced Strategic Computing Initiative
The first of these supercomputers, ASCI Red, was delivered to Sandia National
Laboratories in 1997