Introduction to GPU Computing

Abstract:

At the beginning everything was simple:
The computing Industry using Cenetral Purpose Units (CPU) and the gaming industry acted like two parallel lines which never meet.
Gamers were using dedicated Gaming Consoles (Atari, Sega, Megason – Rings a bell?).
Slowly, with the development of computers and television screens, the graphic processors (GPU) field became powerful now dominanting in the gaming industry and overcomeing the video consoles mainly at “professional gamers” use.

What is the GPU (Graphics Processing Unit)?

Graphics processor designed for graphics applications, so the structure of the processor architecture is dedicated to treat matrices of pixels.
GPU is a massively parallel processor support thousands of active threads, the parallelism is possible thanks to large number of calculation cores designed to perform quickly specific functions

In order to really understand how GPU computing can save us hours, we have to look back in history of Alegbra:

Linear algebra is the branch of mathematics concerning vector spaces and linear mappings between such spaces. It includes the study of lines, planes, and subspaces, but is also concerned with properties common to all vector spaces. (WikipediA).
The first systematic link between Euclidean geometry and algebra has done by René Descartes in 1637, by the invention of Cartesian coordinates.About 10 years later Isaac Newton used the Linear algebra to formulate his famous Newton’s Second law of motion. In 1857 developed Arthur Cayley the matrix, At this point the relationship between mathematics & Physics changed. Up to this point the physics in fact was the motivation for the mathematics development; Researchers sought out mathematical patterns to explain physical phenomena. But the 18th century is a turning point because at this point mathematics develops as separate path without physical essence. For example to develop the theory of general relativity, Einstein is required to learn Tensors mathematics which was already mature mathematics filed.Picture is also a matrix (Coincidence) and also electric fields, magnetic fields, discrete Fourier transform and more..
Computers always helped in solving mathematical problems, but it is the 20th century which was a breaking point:
The interaction between computing and mathematics blurs the boundaries.
A weeks calculation found their solution in a few hours Using graphics computation engine
In this way the graphics processor became mathematics and physics researchers best friend.

This “friendly relationship” is actually based on what we call : Task parallelism :

What is Task Parallelism:

Task parallelism (also known as function parallelism and control parallelism) is a form of parallelization of computer code across multiple processors in parallel computing environments. Task parallelism focuses on distributing tasks—concretely performed by processes or threads—across different parallel computing nodes. It contrasts to data parallelism as another form of parallelism. (Wikipedia)

GPU computing and Task Parallelism:

• GPU is a massively parallel processor Support thousands of active threads
• GPU Computing requires a programming model that can efficiently express that kind of parallelism
• GPU designed as a numeric computing engine
• Will not perform well on some tasks as CPUs
• Most applications will use both CPUs and GPUs
• CUDA implements such a programming model

The CUDA platform is a software layer that gives direct access to the GPU’s virtual instruction set and parallel computational elements

Typical software Applications of GPU Computing:

Within the typical software application of GPU computing we can find:

• Evolved Machines
• Matlab
• Hanweck Associates
• CST
• Manifold® System

For applications examples:

http://www.eimesc.co.il/introduction-to-gpu-computing-101/