3.9 Persistent communication requests

Often a communication with the same argument list is repeatedly executed within the inner loop of a parallel computation. In such a situation, it may be possible to optimize the communication by binding the list of communication arguments to a persistent communication request once and, then, repeatedly using the request to initiate and complete messages. The persistent request thus created can be thought of as a communication port or a ``half-channel.'' It does not provide the full functionality of a conventional channel, since there is no binding of the send port to the receive port. This construct allows reduction of the overhead for communication between the process and communication controller, but not of the overhead for communication between one communication controller and another. It is not necessary that messages sent with a persistent request be received by a receive operation using a persistent request, or vice versa.

A persistent communication request is created using one of the four following calls. These calls involve no communication.



MPI_SEND_INIT(buf, count, datatype, dest, tag, comm, request)

IN
buf initial address of send buffer (choice)
IN
count number of elements sent (integer)
IN
datatype type of each element (handle)
IN
dest rank of destination (integer)
IN
tag message tag (integer)
IN
comm communicator (handle)
OUT
request communication request (handle)

int MPI_Send_init(void* buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm, MPI_Request *request)



MPI_SEND_INIT(BUF, COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR)
<type> BUF(*)
INTEGER REQUEST, COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR



Creates a persistent communication request for a standard mode send operation, and binds to it all the arguments of a send operation.



MPI_BSEND_INIT(buf, count, datatype, dest, tag, comm, request)

IN
buf initial address of send buffer (choice)
IN
count number of elements sent (integer)
IN
datatype type of each element (handle)
IN
dest rank of destination (integer)
IN
tag message tag (integer)
IN
comm communicator (handle)
OUT
request communication request (handle)

int MPI_Bsend_init(void* buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm, MPI_Request *request)



MPI_BSEND_INIT(BUF, COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR)
<type> BUF(*)
INTEGER REQUEST, COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR



Creates a persistent communication request for a buffered mode send.



MPI_SSEND_INIT(buf, count, datatype, dest, tag, comm, request)

IN
buf initial address of send buffer (choice)
IN
count number of elements sent (integer)
IN
datatype type of each element (handle)
IN
dest rank of destination (integer)
IN
tag message tag (integer)
IN
comm communicator (handle)
OUT
request communication request (handle)

int MPI_Ssend_init(void* buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm, MPI_Request *request)



MPI_SSEND_INIT(BUF, COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR)
<type> BUF(*)
INTEGER COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR



Creates a persistent communication object for a synchronous mode send operation.



MPI_RSEND_INIT(buf, count, datatype, dest, tag, comm, request)

IN
buf initial address of send buffer (choice)
IN
count number of elements sent (integer)
IN
datatype type of each element (handle)
IN
dest rank of destination (integer)
IN
tag message tag (integer)
IN
comm communicator (handle)
OUT
request communication request (handle)

int MPI_Rsend_init(void* buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm, MPI_Request *request)



MPI_RSEND_INIT(BUF, COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR)
<type> BUF(*)
INTEGER COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR



Creates a persistent communication object for a ready mode send operation.



MPI_RECV_INIT(buf, count, datatype, source, tag, comm, request)

OUT
buf initial address of receive buffer (choice)
IN
count number of elements received (integer)
IN
datatype type of each element (handle)
IN
source rank of source or MPI_ANY_SOURCE (integer)
IN
tag message tag or MPI_ANY_TAG (integer)
IN
comm communicator (handle)
OUT
request communication request (handle)

int MPI_Recv_init(void* buf, int count, MPI_Datatype datatype, int source, int tag, MPI_Comm comm, MPI_Request *request)



MPI_RECV_INIT(BUF, COUNT, DATATYPE, SOURCE, TAG, COMM, REQUEST, IERROR)
<type> BUF(*)
INTEGER COUNT, DATATYPE, SOURCE, TAG, COMM, REQUEST, IERROR



Creates a persistent communication request for a receive operation. The argument buf is marked as OUT because the user gives permission to write on the receive buffer by passing the argument to MPI_RECV_INIT.

A persistent communication request is inactive after it was created -- no active communication is attached to the request.

A communication (send or receive) that uses a persistent request is initiated by the function MPI_START.



MPI_START(request)

INOUT
request communication request (handle)

int MPI_Start(MPI_Request *request)



MPI_START(REQUEST, IERROR)
INTEGER REQUEST, IERROR



The argument, request, is a handle returned by one of the previous five calls. The associated request should be inactive. The request becomes active once the call is made.

If the request is for a send with ready mode, then a matching receive should be posted before the call is made. The communication buffer should not be accessed after the call, and until the operation completes.

The call is local, with similar semantics to the nonblocking communication operations described in section 3.7. That is, a call to MPI_START with a request created by MPI_SEND_INIT starts a communication in the same manner as a call to MPI_ISEND; a call to MPI_START with a request created by MPI_BSEND_INIT starts a communication in the same manner as a call to MPI_IBSEND; and so on.



MPI_STARTALL(count, array_of_requests)

IN
count list length (integer)
INOUT
array_of_requests array of requests (array of handle)

int MPI_Startall(int count, MPI_Request *array_of_requests)



MPI_STARTALL(COUNT, ARRAY_OF_REQUESTS, IERROR)
INTEGER COUNT, ARRAY_OF_REQUESTS(*), IERROR



Start all communications associated with requests in array_of_requests. A call to
MPI_STARTALL(count, array_of_requests) has the same effect as calls to MPI_START (&array_of_requests[i]), executed for i=0 ,..., count-1, in some arbitrary order.

A communication started with a call to MPI_START or MPI_STARTALL is completed by a call to MPI_WAIT, MPI_TEST, or one of the derived functions described in section 3.7.5. The request becomes inactive after successful completion of such call. The request is not deallocated and it can be activated anew by an MPI_START or MPI_STARTALL call.

A persistent request is deallocated by a call to MPI_REQUEST_FREE (Section 3.7.3).

The call to MPI_REQUEST_FREE can occur at any point in the program after the persistent request was created. However, the request will be deallocated only after it becomes inactive. Active receive requests should not be freed. Otherwise, it will not be possible to check that the receive has completed. It is preferable, in general, to free requests when they are inactive. If this rule is followed, then the functions described in this section will be invoked in a sequence of the form,



\( \bf
Create \ (Start \ Complete)^* \ Free \ ,
\)

where $*$ indicates zero or more repetitions. If the same communication object is used in several concurrent threads, it is the user's responsibility to coordinate calls so that the correct sequence is obeyed.

A send operation initiated with MPI_START can be matched with any receive operation and, likewise, a receive operation initiated with MPI_START can receive messages generated by any send operation.

Enhancement/Correction in the MPI-1.2 Standard: [*]

MPI-Standard for MARMOT