Collective operation
Encyclopedia
A collective operation is a concept in parallel computing
in which data is simultaneously sent to or received from many nodes.
Common examples of collective operations are gather (in which data is collected from all nodes), scatter (in which a set of data is broken up into pieces, and a different piece is sent to all nodes), and broadcast (in which the same data is sent to all nodes).
Parallel computing
Parallel computing is a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently . There are several different forms of parallel computing: bit-level,...
in which data is simultaneously sent to or received from many nodes.
Common examples of collective operations are gather (in which data is collected from all nodes), scatter (in which a set of data is broken up into pieces, and a different piece is sent to all nodes), and broadcast (in which the same data is sent to all nodes).
Parallel computing frameworks with collective operations
Collective operations are present in the following parallel programming frameworks:- Google's MapReduceMapReduceMapReduce is a software framework introduced by Google in 2004 to support distributed computing on large data sets on clusters of computers. Parts of the framework are patented in some countries....
framework - MPIMessage Passing InterfaceMessage Passing Interface is a standardized and portable message-passing system designed by a group of researchers from academia and industry to function on a wide variety of parallel computers...