TNL::Containers::DistributedNDArraySynchronizer< DistributedNDArray > Class Template Reference

Synchronizer for DistributedNDArray. More...

#include <TNL/Containers/DistributedNDArraySynchronizer.h>

Collaboration diagram for TNL::Containers::DistributedNDArraySynchronizer< DistributedNDArray >:

Classes
struct	CopyKernel

Public Types
enum class	AsyncPolicy { synchronous , deferred , threadpool , async }
using	RequestsVector = std::vector< MPI_Request >

Public Member Functions
	DistributedNDArraySynchronizer (DistributedNDArraySynchronizer &&other) noexcept
	DistributedNDArraySynchronizer (int max_threads=1)
void	setBufferOffsets (int shift=0)
	Sets the send and receive offsets for all buffer objects.
void	setCudaStream (SyncDirection direction, Backend::stream_t stream_id)
void	setNeighbor (SyncDirection direction, int neighbor)
template<std::size_t Q>
void	setSynchronizationPattern (const std::array< SyncDirection, Q > &pattern)
	Set the communication pattern between neighbors during data synchronization.
void	setTagOffset (int offset)
void	setTags (SyncDirection direction, int tag_recv, int tag_send)
void	stage_0 (DistributedNDArray &array, SyncDirection mask)
void	stage_1 ()
void	stage_2 ()
void	stage_3 ()
void	stage_4 ()
void	synchronize (AsyncPolicy policy, DistributedNDArray &array, SyncDirection mask=SyncDirection::All)
	Synchronizes data in array distributed among MPI ranks.
void	synchronize (DistributedNDArray &array, SyncDirection mask=SyncDirection::All)
	Synchronizes data in array distributed among MPI ranks.
void	wait ()

Public Attributes
std::future< void >	async_op
	Can be used for checking if a synchronization started asynchronously has been finished.
std::size_t	async_ops_count = 0
Timer	async_start_timer
Timer	async_wait_timer
std::size_t	recv_bytes = 0
std::size_t	recv_messages = 0
RequestsVector	requests
std::size_t	sent_bytes = 0
std::size_t	sent_messages = 0

Protected Member Functions
void	allocateHelper ()
void	copyHelper (bool to_buffer, SyncDirection mask)

Static Protected Member Functions
static constexpr int	countDimensionsWithOverlap ()
template<std::size_t order>
static constexpr int	getDimensionWithOverlap ()
static void	setCudaStream (Backend::LaunchConfiguration &launch_config, Backend::stream_t stream)
template<typename LaunchConfiguration >
static void	setCudaStream (LaunchConfiguration &launch_config, Backend::stream_t stream)

Detailed Description

template<typename DistributedNDArray>
class TNL::Containers::DistributedNDArraySynchronizer< DistributedNDArray >

Synchronizer for DistributedNDArray.

Member Function Documentation

setBufferOffsets()

template<typename DistributedNDArray >

void TNL::Containers::DistributedNDArraySynchronizer< DistributedNDArray >::setBufferOffsets ( int  shift = 0 )

inline

Sets the send and receive offsets for all buffer objects.

This is primarily an internal function, but applications that require special communication patterns (e.g. the A-A pattern in LBM) can use it to adjust the behavior.

Parameters

shift

determines by how many cells the offsets are shifted in the direction configured for the buffer.

setSynchronizationPattern()

template<typename DistributedNDArray >

template<std::size_t Q>

void TNL::Containers::DistributedNDArraySynchronizer< DistributedNDArray >::setSynchronizationPattern ( const std::array< SyncDirection, Q > &  pattern )

inline

Set the communication pattern between neighbors during data synchronization.

Template Parameters

Q	is the number of elements in pattern.

Parameters

pattern

is the synchronization pattern (array of directions in which the data will be sent). It must be consistent with the partitioning of the distributed array.

synchronize() [1/2]

template<typename DistributedNDArray >

void TNL::Containers::DistributedNDArraySynchronizer< DistributedNDArray >::synchronize ( AsyncPolicy  policy, DistributedNDArray &  array, SyncDirection  mask = SyncDirection::All  )

inline

Synchronizes data in array distributed among MPI ranks.

This method is not thread-safe - only the thread which created and "owns" the instance of this object can call this method.

Also note that this method must not be called again until the previous asynchronous operation has finished.

Parameters

policy	determines the async policy used by the synchronizer.
array	is the distributed array to be synchronized.
mask	can be used to suppress specific directions from the pattern set with setSynchronizationPattern (useful e.g. for the lattice Boltzmann method).

synchronize() [2/2]

template<typename DistributedNDArray >

void TNL::Containers::DistributedNDArraySynchronizer< DistributedNDArray >::synchronize ( DistributedNDArray &  array, SyncDirection  mask = SyncDirection::All  )

inline

Synchronizes data in array distributed among MPI ranks.

Parameters

array	is the distributed array to be synchronized.
mask	can be used to suppress specific directions from the pattern set with setSynchronizationPattern (useful e.g. for the lattice Boltzmann method).

---

The documentation for this class was generated from the following file:

• src/TNL/Containers/DistributedNDArraySynchronizer.h