Template Numerical Library version\ main:c173bea
Loading...
Searching...
No Matches
TNL::Containers::DistributedNDArraySynchronizer< DistributedNDArray > Class Template Reference

Synchronizer for DistributedNDArray. More...

#include <TNL/Containers/DistributedNDArraySynchronizer.h>

Collaboration diagram for TNL::Containers::DistributedNDArraySynchronizer< DistributedNDArray >:

Classes

struct  CopyKernel
 

Public Types

enum class  AsyncPolicy { synchronous , deferred , threadpool , async }
 
using RequestsVector = std::vector< MPI_Request >
 

Public Member Functions

 DistributedNDArraySynchronizer (DistributedNDArraySynchronizer &&other) noexcept
 
 DistributedNDArraySynchronizer (int max_threads=1)
 
void setBufferOffsets (int shift=0)
 Sets the send and receive offsets for all buffer objects.
 
void setCudaStream (SyncDirection direction, Backend::stream_t stream_id)
 
void setNeighbor (SyncDirection direction, int neighbor)
 
template<std::size_t Q>
void setSynchronizationPattern (const std::array< SyncDirection, Q > &pattern)
 Set the communication pattern between neighbors during data synchronization.
 
void setTagOffset (int offset)
 
void setTags (SyncDirection direction, int tag_recv, int tag_send)
 
void stage_0 (DistributedNDArray &array, SyncDirection mask)
 
void stage_1 ()
 
void stage_2 ()
 
void stage_3 ()
 
void stage_4 ()
 
void synchronize (AsyncPolicy policy, DistributedNDArray &array, SyncDirection mask=SyncDirection::All)
 Synchronizes data in array distributed among MPI ranks.
 
void synchronize (DistributedNDArray &array, SyncDirection mask=SyncDirection::All)
 Synchronizes data in array distributed among MPI ranks.
 
void wait ()
 

Public Attributes

std::future< void > async_op
 Can be used for checking if a synchronization started asynchronously has been finished.
 
std::size_t async_ops_count = 0
 
Timer async_start_timer
 
Timer async_wait_timer
 
std::size_t recv_bytes = 0
 
std::size_t recv_messages = 0
 
RequestsVector requests
 
std::size_t sent_bytes = 0
 
std::size_t sent_messages = 0
 

Protected Member Functions

void allocateHelper ()
 
void copyHelper (bool to_buffer, SyncDirection mask)
 

Static Protected Member Functions

static int countDimensionsWithOverlap (const DistributedNDArrayView &array_view)
 
template<std::size_t order>
static int getDimensionWithOverlap (const DistributedNDArrayView &array_view)
 
static void setCudaStream (Backend::LaunchConfiguration &launch_config, Backend::stream_t stream)
 
template<typename LaunchConfiguration >
static void setCudaStream (LaunchConfiguration &launch_config, Backend::stream_t stream)
 

Detailed Description

template<typename DistributedNDArray>
class TNL::Containers::DistributedNDArraySynchronizer< DistributedNDArray >

Synchronizer for DistributedNDArray.

Member Function Documentation

◆ setBufferOffsets()

template<typename DistributedNDArray >
void TNL::Containers::DistributedNDArraySynchronizer< DistributedNDArray >::setBufferOffsets ( int shift = 0)
inline

Sets the send and receive offsets for all buffer objects.

This is primarily an internal function, but applications that require special communication patterns (e.g. the A-A pattern in LBM) can use it to adjust the behavior.

Parameters
shiftdetermines by how many cells the offsets are shifted in the direction configured for the buffer.

◆ setSynchronizationPattern()

template<typename DistributedNDArray >
template<std::size_t Q>
void TNL::Containers::DistributedNDArraySynchronizer< DistributedNDArray >::setSynchronizationPattern ( const std::array< SyncDirection, Q > & pattern)
inline

Set the communication pattern between neighbors during data synchronization.

Template Parameters
Qis the number of elements in pattern.
Parameters
patternis the synchronization pattern (array of directions in which the data will be sent). It must be consistent with the partitioning of the distributed array.

◆ synchronize() [1/2]

template<typename DistributedNDArray >
void TNL::Containers::DistributedNDArraySynchronizer< DistributedNDArray >::synchronize ( AsyncPolicy policy,
DistributedNDArray & array,
SyncDirection mask = SyncDirection::All )
inline

Synchronizes data in array distributed among MPI ranks.

This method is not thread-safe - only the thread which created and "owns" the instance of this object can call this method.

Also note that this method must not be called again until the previous asynchronous operation has finished.

Parameters
policydetermines the async policy used by the synchronizer.
arrayis the distributed array to be synchronized.
maskcan be used to suppress specific directions from the pattern set with setSynchronizationPattern (useful e.g. for the lattice Boltzmann method).

◆ synchronize() [2/2]

Synchronizes data in array distributed among MPI ranks.

Parameters
arrayis the distributed array to be synchronized.
maskcan be used to suppress specific directions from the pattern set with setSynchronizationPattern (useful e.g. for the lattice Boltzmann method).

The documentation for this class was generated from the following file: