Have fun with MPI in C

spagnuolocarmine

135.8K views

GitHub

Open Source Your Knowledge, Become a Contributor

Technology knowledge has to be shared and made accessible for free. Join the movement.

Create Content

Previous: Communicate noncontiguous data Next: Chapter Questions

Derived Datatypes

MPI allows the programmer to create your own data types, analogous to defining structures in C. MPI data types are mostly of use if you want to send multiple items in one message. Derived types allows to create new contiguous data type consisting of an array of elements of another data type; a vector data type consisting of regularly spaced blocks of elements of a component type; not regularly spaced data, there is the indexed data type , where you specify an array of index locations for blocks of elements of a component type; and struct data type can accomodate multiple data types.

A general datatype is an opaque object that specifies two things:

A sequence of basic datatypes.
A sequence of integer (byte) displacements.

The displacements are not required to be positive, distinct, or in increasing order.

Process to create and use a new Datatype

Create your own datatype using Datatype Constructors
Commit your datatype
Use it in your communications routines
Free your datatype

Commit and Free your Datatype

Before to describe the construction of data type, we present how to commit and free a new data type, because these functions will be used in the examples.

A datatype object has to be committed before it can be used in a communication. As an argument in datatype constructors, uncommitted and also committed datatypes can be used. There is no need to commit basic datatypes (they are "pre-committed").

MPI_TYPE_COMMIT(datatype)

INOUT datatype, datatype that is committed (handle)

C version

int MPI_Type_commit(MPI_Datatype *datatype)

A datatype that will be not be used again must be freed.

MPI_TYPE_FREE(datatype)

INOUT datatype, datatype that is freed (handle)

C version

int MPI_Type_free(MPI_Datatype *datatype

Address and Size Functions useful for Datatype construction

The displacements in a general datatype are relative to some initial buffer address. Absolute addresses can be substituted for these displacements: we treat them as displacements relative to "address zero" the start of the address space. This initial address zero is in-dicated by the constant MPI_BOTTOM.

The address of a location in memory can be found by invoking the function MPI_GET_ADDRESS, which uses the MPI_Aint value, that is a signed integer, or more in details a C type that holds any valid address. Moreover, the relative displacement between two absolute addresses can be calculated with the function MPI_AINT_DIFF. A new absolute address as sum of an absolute base address and a relative displacement can be calculated with the function MPI_AINT_ADD.

MPI_GET_ADDRESS(location, address)

IN location, location in caller memory (choice)
OUT address, address of location (integer)

C version

int MPI_Get_address(const void *location, MPI_Aint *address)

The following auxiliary functions provide useful information on derived datatypes.

Size of a datatype: return the number of bytes occupied by entries in the datatype.

MPI_TYPE_SIZE(datatype, size)

IN datatype, datatype (handle)
OUT size, datatype size (integer)

C version

int MPI_Type_size(MPI_Datatype datatype, int *size)

Datatype extend, get the lower bound and extent for a Datatype.

MPI_TYPE_GET_EXTENT(datatype, lb, extent)

IN datatype, datatype to get information on (handle)
OUT lblower bound of datatype (integer)
OUT extent, extent of datatype (integer)

C version

int MPI_Type_get_extent(MPI_Datatype datatype, MPI_Aint *lb,MPI_Aint *extent)

Datatype Constructors

Contiguous

The simplest datatype constructor is MPI_TYPE_CONTIGUOUS which allows replication of a datatype into contiguous locations.

MPI_TYPE_CONTIGUOUS

MPI_TYPE_CONTIGUOUS(count, oldtype, newtype)

IN count, replication count (non-negative integer)
IN oldtype, old datatype (handle)
OUT newtype, new datatype (handle)

C version

int MPI_Type_contiguous(int count, MPI_Datatype oldtype,MPI_Datatype *newtype)

The following example uses 2 processes.

MPI_Type_contiguous

#include "mpi.h"
#include <stdio.h>
 
int main(int argc, char *argv[])
{
    int myrank;
    MPI_Status status;
    MPI_Datatype type;
    int buffer[100];
 
    MPI_Init(&argc, &argv);
 
    MPI_Type_contiguous( 100, MPI_CHAR, &type );
    MPI_Type_commit(&type);
 
    MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
 
    if (myrank == 0)
    {
        MPI_Send(buffer, 1, type, 1, 123, MPI_COMM_WORLD);
    }
    else if (myrank == 1)
    {
        MPI_Recv(buffer, 1, type, 0, 123, MPI_COMM_WORLD, &status);
    }
 
    MPI_Finalize();
    return 0;
}

Vector

The function MPI_TYPE_VECTOR is a more general constructor that allows replication of a datatype into locations that consist of equally spaced blocks. Each block is obtained by concatenating the same number of copies of the old datatype. The spacing between blocks is a multiple of the extent of the old datatype.

MPI_TYPE_VECTOR

MPI_TYPE_VECTOR(count, blocklength, stride, oldtype, newtype)

IN count, number of blocks (non-negative integer)
IN blocklength, number of elements in each block (non-negative integer)
IN stride, number of elements between start of each block (integer)
IN oldtype, old datatype (handle)
OUT newtype, new datatype (handle)

C version

int MPI_Type_vector(int count, int blocklength, int stride,MPI_Datatype oldtype, MPI_Datatype *newtype)

A call to MPI_TYPE_CONTIGUOUS(count, oldtype, newtype) is equivalent to a call to MPI_TYPE_VECTOR(count, 1, 1, oldtype, newtype), or to a call to MPI_TYPE_VECTOR(1,count, n, oldtype, newtype), n arbitrary.

The following example uses 4 processes.

MPI_Type_vector

#include "mpi.h"
#include <stdio.h>
#include <stdlib.h>
#define SIZE 4
/*Sendind each colum to a processor*/
int main (int argc, char *argv[])
{
    int numtasks, rank, source=0, dest, tag=1, i;
    float a[SIZE][SIZE] =
    {1.0, 2.0, 3.0, 4.0,
    5.0, 6.0, 7.0, 8.0,
    9.0, 10.0, 11.0, 12.0,
    13.0, 14.0, 15.0, 16.0};
    float b[SIZE];
    MPI_Status stat;
    MPI_Datatype columntype;
    MPI_Init(&argc,&argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &numtasks);
    MPI_Type_vector(SIZE/*num of element in a column*/,
                     1 /*one element for row*/, 
                    SIZE /*take an element each 4*/, MPI_FLOAT, &columntype);
    MPI_Type_commit(&columntype);
    if (numtasks == SIZE) {
    if (rank == 0) {
        for (i=0; i<numtasks; i++)
        MPI_Send(&a[0][i], 1, columntype, i, tag, MPI_COMM_WORLD);
            }
    MPI_Recv(b, SIZE, MPI_FLOAT, source, tag, MPI_COMM_WORLD, &stat);
    printf("rank= %d  b= %3.1f %3.1f %3.1f %3.1f\n",
            rank,b[0],b[1],b[2],b[3]);
    }
    else
    printf("Must specify %d processors. Terminating.\n",SIZE);
    MPI_Type_free(&columntype);
    MPI_Finalize();
}

Indexed

The function MPI_TYPE_INDEXED allows replication of an old datatype into a sequence of blocks (each block is a concatenation of the old datatype), where each blockcan contain a different number of copies and have a different displacement. All block displacements are multiples of the old type extent.

MPI_TYPE_INDEXED

MPI_TYPE_INDEXED(count, array_of_blocklengths, array_of_displacements, oldtype,newtype)

IN count, number of blocks — also number of entries inarray_of_displacements and array_of_blocklengths(non-negative integer)
IN array_of_blocklengths, number of elements per block (array of non-negative integers)
IN array_of_displacements, displacement for each block, in multiples of oldtype extent (array of integer)
IN oldtype, old datatype (handle)
OUT newtype, new datatype (handle)

C version

int MPI_Type_indexed(int count, const int array_of_blocklengths[],const int array_of_displacements[], MPI_Datatype oldtype,MPI_Datatype *newtype)

The following example uses 4 processes.

MPI_Type_indexed

 #include "mpi.h"
   #include <stdio.h>
   #define NELEMENTS 6
   main(int argc, char *argv[])  {
   int numtasks, rank, source=0, dest, tag=1, i;
   int blocklengths[2], displacements[2];
   float a[16] = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0};
   float b[NELEMENTS]; 
   MPI_Status stat;
   MPI_Datatype indextype;   // required variable
   MPI_Init(&argc,&argv);
   MPI_Comm_rank(MPI_COMM_WORLD, &rank);
   MPI_Comm_size(MPI_COMM_WORLD, &numtasks);
   blocklengths[0] = 4; /*take 4 elements from the array*/
   blocklengths[1] = 2; /*take 2 elemnets from the array*/
   displacements[0] = 5;/*start from the element index 5 the first block that is 6.0 */
   displacements[1] = 12;/*start from the element index 12  the first block that is 13.0 */
   
   // create indexed derived data type
   MPI_Type_indexed(2, blocklengths, displacements, MPI_FLOAT, &indextype);
   MPI_Type_commit(&indextype);
   if (rank == 0) {
     for (i=0; i<numtasks; i++) 
      // task 0 sends one element of indextype to all tasks
        MPI_Send(a, 1, indextype, i, tag, MPI_COMM_WORLD);
     }
   // all tasks receive indextype data from task 0
   MPI_Recv(b, NELEMENTS, MPI_FLOAT, source, tag, MPI_COMM_WORLD, &stat);
   printf("rank= %d  b= %3.1f %3.1f %3.1f %3.1f %3.1f %3.1f\n",
          rank,b[0],b[1],b[2],b[3],b[4],b[5]);
   
   // free datatype when done using it
   MPI_Type_free(&indextype);
   MPI_Finalize();
   }

Struct

MPI_TYPE_CREATE_STRUCT is the most general type constructor. Allows the programmer to define a newdata type formed by completely defined map of the component data types.

MPI_TYPE_STRCUT

MPI_TYPE_CREATE_STRUCT(count, array_of_blocklengths, array_of_displacements, array_of_types, newtype)

IN count, number of blocks (non-negative integer) — also number of entries in arrays array_of_types,array_of_displacement sandarray_of_blocklengths
IN array_of_blocklength, number of elements in each block (array of non-negative integer)
IN array_of_displacements, byte displacement of each block (array of integer)
IN array_of_types, type of elements in each block (array of handles to datatype objects)
OUT newtype, new datatype (handle)

C version

int MPI_Type_create_struct(int count, const int array_of_blocklengths[],const MPI_Aint array_of_displacements[],const MPI_Datatype array_of_types[], MPI_Datatype *newtype

The following example uses 4 processes.

MPI_Type_struct

#include "mpi.h"
   #include <stdio.h>
   #define NELEM 25
   main(int argc, char *argv[])  {
   int numtasks, rank, source=0, dest, tag=1, i;
   typedef struct {
     float x, y, z;
     float velocity;
     int  n, type;
     }          Particle;
   Particle     p[NELEM], particles[NELEM];
   MPI_Datatype particletype, oldtypes[2];   // required variables
   int          blockcounts[2];
   // MPI_Aint type used to be consistent with syntax of
   // MPI_Type_extent routine
   MPI_Aint    offsets[2], lb, extent;
   MPI_Status stat;
   MPI_Init(&argc,&argv);
   MPI_Comm_rank(MPI_COMM_WORLD, &rank);
   MPI_Comm_size(MPI_COMM_WORLD, &numtasks);
 
   // setup description of the 4 MPI_FLOAT fields x, y, z, velocity
   offsets[0] = 0;
   oldtypes[0] = MPI_FLOAT;
   blockcounts[0] = 4;
   // setup description of the 2 MPI_INT fields n, type
   // need to first figure offset by getting size of MPI_FLOAT
   MPI_Type_get_extent(MPI_FLOAT, &lb, &extent);
   offsets[1] = 4 * extent;
   oldtypes[1] = MPI_INT;
   blockcounts[1] = 2;
   // define structured type and commit it
   MPI_Type_create_struct(2, blockcounts, offsets, oldtypes, &particletype);
   MPI_Type_commit(&particletype);
   // task 0 initializes the particle array and then sends it to each task
   if (rank == 0) {
     for (i=0; i<NELEM; i++) {
        particles[i].x = i * 1.0;
        particles[i].y = i * -1.0;
        particles[i].z = i * 1.0; 
        particles[i].velocity = 0.25;
        particles[i].n = i;
        particles[i].type = i % 2; 

Open Source Your Knowledge: become a Contributor and help others learn. Create New Content

Open Source Your Knowledge, Become a Contributor

12/22 Derived Datatypes

Derived Datatypes

Process to create and use a new Datatype

Commit and Free your Datatype

Address and Size Functions useful for Datatype construction

Datatype Constructors

Contiguous

Vector

Indexed

Struct

La courbe du Dragon - S5

HEPL - Bloc 1, Q1 - Labos C - P Worontzoff

Programmation appliquée à l'électronique - Niveau II

Programação C