TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension > Class Template Reference

Inheritance diagram for TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >:

Collaboration diagram for TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >:

Public Types
using	ByteArrayView = typename Base::ByteArrayView
using	DeviceType = typename DistributedMesh::DeviceType
using	GlobalIndexType = typename DistributedMesh::GlobalIndexType
using	RequestsVector = typename Base::RequestsVector
Public Types inherited from TNL::Containers::ByteArraySynchronizer< DistributedMesh::DeviceType, DistributedMesh::GlobalIndexType >
enum	AsyncPolicy
using	ByteArrayView
using	RequestsVector

Public Member Functions
int	getEntityOwner (GlobalIndexType global_idx) const
const auto &	getGhostEntitiesCounts () const
const auto &	getGhostNeighborOffsets () const
const auto &	getGhostNeighbors () const
const auto &	getGhostOffsets () const
const auto &	getGlobalOffsets () const
void	initialize (const DistributedMesh &mesh)
template<typename Array , std::enable_if_t< ! HasMeshType< Array >::value, bool > = true>
void	synchronize (Array &array)
template<typename MeshFunction , std::enable_if_t< HasMeshType< MeshFunction >::value, bool > = true>
void	synchronize (MeshFunction &function)
template<typename Array >
void	synchronizeArray (Array &array, int valuesPerElement=1)
void	synchronizeByteArray (ByteArrayView array, int bytesPerValue) override
	Main synchronization function.
RequestsVector	synchronizeByteArrayAsyncWorker (ByteArrayView array, int bytesPerValue) override
template<typename SparsePattern >
auto	synchronizeSparse (const SparsePattern &pattern, bool assumeConsistentRowCapacities=false)
Public Member Functions inherited from TNL::Containers::ByteArraySynchronizer< DistributedMesh::DeviceType, DistributedMesh::GlobalIndexType >
void	synchronizeByteArrayAsync (ByteArrayView array, int bytesPerValue, AsyncPolicy policy=AsyncPolicy::synchronous)
	An asynchronous version of synchronizeByteArray.

Protected Attributes
MPI::Comm	communicator
Matrices::DenseMatrix< GlobalIndexType, Devices::Host, int >	ghostEntitiesCounts
Containers::Array< GlobalIndexType, Devices::Host, int >	ghostNeighborOffsets
Containers::Vector< GlobalIndexType, DeviceType, GlobalIndexType >	ghostNeighbors
Containers::Array< GlobalIndexType, Devices::Host, int >	ghostOffsets
Containers::Array< GlobalIndexType, Devices::Host, int >	globalOffsets
Containers::Array< std::uint8_t, DeviceType, GlobalIndexType >	sendBuffers

Additional Inherited Members
Public Attributes inherited from TNL::Containers::ByteArraySynchronizer< DistributedMesh::DeviceType, DistributedMesh::GlobalIndexType >
std::future< void >	async_op
	Can be used for checking if a synchronization started asynchronously has been finished.
std::size_t	async_ops_count
Timer	async_start_timer
Timer	async_wait_before_start_timer
Timer	async_wait_timer

Member Function Documentation

synchronizeByteArray()

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>

void TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::synchronizeByteArray ( ByteArrayView array, int bytesPerValue )

inlineoverridevirtual

Main synchronization function.

This is only a pure virtual function – the functionality must be implemented in a subclass.

Implements TNL::Containers::ByteArraySynchronizer< DistributedMesh::DeviceType, DistributedMesh::GlobalIndexType >.

synchronizeByteArrayAsyncWorker()

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>

RequestsVector TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::synchronizeByteArrayAsyncWorker ( ByteArrayView array, int bytesPerValue )

inlinenodiscardoverridevirtual

Implements TNL::Containers::ByteArraySynchronizer< DistributedMesh::DeviceType, DistributedMesh::GlobalIndexType >.

Member Data Documentation

ghostEntitiesCounts

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>

Matrices::DenseMatrix< GlobalIndexType, Devices::Host, int > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::ghostEntitiesCounts

protected

Communication pattern:

• an unsymmetric nproc x nproc matrix G such that G_ij represents the number of ghost entities on rank i that are owned by rank j
• assembly of the i-th row involves traversal of the ghost entities on the local mesh and determining its owner based on the global index
• assembly of the full matrix needs all-to-all communication
• for the i-th rank, the i-th row determines the receive buffer sizes and the i-th column determines the send buffer sizes

ghostNeighborOffsets

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>

Containers::Array< GlobalIndexType, Devices::Host, int > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::ghostNeighborOffsets

protected

Ghost neighbor offsets: array of size nproc + 1 where the i-th value is the offset of ghost neighbor indices requested by the i-th rank. The last value is the size of the ghostNeighbors and sendBuffers arrays (see below).

ghostNeighbors

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>

Containers::Vector< GlobalIndexType, DeviceType, GlobalIndexType > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::ghostNeighbors

protected

Ghost neighbor indices: array containing local indices of the entities which are ghosts on other ranks. The indices requested by the i-th rank are in the range starting at ghostNeighborOffsets[i] (inclusive) and ending at ghostNeighborOffsets[i+1] (exclusive). These indices are used for copying the mesh function values into the sendBuffers array. Note that ghost neighbor indices cannot be made contiguous in general so we need the send buffers.

ghostOffsets

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>

Containers::Array< GlobalIndexType, Devices::Host, int > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::ghostOffsets

protected

Ghost offsets: array of size nproc + 1 where the i-th value is the local index of the first ghost entity owned by the i-th rank. The last value is equal to the entities count on the local mesh. All ghost entities owned by the i-th rank are assumed to be indexed contiguously in the local mesh.

globalOffsets

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>

Containers::Array< GlobalIndexType, Devices::Host, int > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::globalOffsets

protected

Global offsets: array of size nproc where the i-th value is the lowest global index of the entities owned by the i-th rank. This can be used to determine the owner of every entity based on its global index.

sendBuffers

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>

Containers::Array< std::uint8_t, DeviceType, GlobalIndexType > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::sendBuffers

protected

Send buffers: array for buffering the mesh function values which will be sent to other ranks. We use std::uint8_t as the value type to make this class independent of the mesh function's real type. When cast to the real type values, the send buffer for the i-th rank is the part of the array starting at index ghostNeighborOffsets[i] (inclusive) and ending at index ghostNeighborOffsets[i+1] (exclusive).

---

The documentation for this class was generated from the following file:

• src/TNL/Meshes/DistributedMeshes/DistributedMeshSynchronizer.h