TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh< Grid< MeshDimension, GridReal, Device, Index > >, MeshDimension > Class Template Reference

Inheritance diagram for TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh< Grid< MeshDimension, GridReal, Device, Index > >, MeshDimension >:

Collaboration diagram for TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh< Grid< MeshDimension, GridReal, Device, Index > >, MeshDimension >:

Public Types
using	ByteArrayView
using	Cell = typename Grid< MeshDimension, GridReal, Device, Index >::Cell
using	CoordinatesType = typename DistributedGridType::CoordinatesType
using	DeviceType
using	DistributedGridType = DistributedMesh< Grid< MeshDimension, GridReal, Device, Index > >
using	GlobalIndexType
enum	PeriodicBoundariesCopyDirection { BoundaryToOverlap , OverlapToBoundary }
using	RequestsVector
using	SubdomainOverlapsType = typename DistributedGridType::SubdomainOverlapsType
Public Types inherited from TNL::Containers::ByteArraySynchronizer< DistributedMesh::DeviceType, DistributedMesh::GlobalIndexType >
enum	AsyncPolicy
using	ByteArrayView
using	RequestsVector

Public Member Functions
	DistributedMeshSynchronizer (const DistributedGridType *distributedGrid)
int	getEntityOwner (GlobalIndexType global_idx) const
const auto &	getGhostEntitiesCounts () const
const auto &	getGhostNeighborOffsets () const
const auto &	getGhostNeighbors () const
const auto &	getGhostOffsets () const
const auto &	getGlobalOffsets () const
void	initialize (const DistributedMesh &mesh)
void	setDistributedGrid (const DistributedGridType *distributedGrid)
void	setPeriodicBoundariesCopyDirection (const PeriodicBoundariesCopyDirection dir)
void	synchronize (Array &array)
void	synchronize (MeshFunction &function)
template<typename MeshFunctionType, typename PeriodicBoundariesMaskPointer = Pointers::SharedPointer< MeshFunctionType >>
void	synchronize (MeshFunctionType &meshFunction, bool periodicBoundaries=false, const PeriodicBoundariesMaskPointer &mask=PeriodicBoundariesMaskPointer(nullptr))
void	synchronizeArray (Array &array, int valuesPerElement=1)
void	synchronizeByteArray (ByteArrayView array, int bytesPerValue) override
	Main synchronization function.
RequestsVector	synchronizeByteArrayAsyncWorker (ByteArrayView array, int bytesPerValue) override
auto	synchronizeSparse (const SparsePattern &pattern, bool assumeConsistentRowCapacities=false)
Public Member Functions inherited from TNL::Containers::ByteArraySynchronizer< DistributedMesh::DeviceType, DistributedMesh::GlobalIndexType >
void	synchronizeByteArrayAsync (ByteArrayView array, int bytesPerValue, AsyncPolicy policy=AsyncPolicy::synchronous)
	An asynchronous version of synchronizeByteArray.

Static Public Member Functions
static constexpr int	getMeshDimension ()
static constexpr int	getNeighborsCount ()

Protected Attributes
MPI::Comm	communicator
Matrices::DenseMatrix< GlobalIndexType, Devices::Host, int >	ghostEntitiesCounts
Containers::Array< GlobalIndexType, Devices::Host, int >	ghostNeighborOffsets
Containers::Vector< GlobalIndexType, DeviceType, GlobalIndexType >	ghostNeighbors
Containers::Array< GlobalIndexType, Devices::Host, int >	ghostOffsets
Containers::Array< GlobalIndexType, Devices::Host, int >	globalOffsets
Containers::Array< std::uint8_t, DeviceType, GlobalIndexType >	sendBuffers

Additional Inherited Members
Public Attributes inherited from TNL::Containers::ByteArraySynchronizer< DistributedMesh::DeviceType, DistributedMesh::GlobalIndexType >
std::future< void >	async_op
	Can be used for checking if a synchronization started asynchronously has been finished.
std::size_t	async_ops_count
Timer	async_start_timer
Timer	async_wait_before_start_timer
Timer	async_wait_timer

Member Function Documentation

synchronizeByteArray()

void TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::synchronizeByteArray ( ByteArrayView array, int bytesPerValue )

inlineoverridevirtual

Main synchronization function.

This is only a pure virtual function – the functionality must be implemented in a subclass.

Implements TNL::Containers::ByteArraySynchronizer< DistributedMesh::DeviceType, DistributedMesh::GlobalIndexType >.

synchronizeByteArrayAsyncWorker()

RequestsVector TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::synchronizeByteArrayAsyncWorker ( ByteArrayView array, int bytesPerValue )

inlinenodiscardoverridevirtual

Implements TNL::Containers::ByteArraySynchronizer< DistributedMesh::DeviceType, DistributedMesh::GlobalIndexType >.

Member Data Documentation

ghostEntitiesCounts

Matrices::DenseMatrix< GlobalIndexType, Devices::Host, int > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::ghostEntitiesCounts

protected

Communication pattern:

• an unsymmetric nproc x nproc matrix G such that G_ij represents the number of ghost entities on rank i that are owned by rank j
• assembly of the i-th row involves traversal of the ghost entities on the local mesh and determining its owner based on the global index
• assembly of the full matrix needs all-to-all communication
• for the i-th rank, the i-th row determines the receive buffer sizes and the i-th column determines the send buffer sizes

ghostNeighborOffsets

Containers::Array< GlobalIndexType, Devices::Host, int > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::ghostNeighborOffsets

protected

Ghost neighbor offsets: array of size nproc + 1 where the i-th value is the offset of ghost neighbor indices requested by the i-th rank. The last value is the size of the ghostNeighbors and sendBuffers arrays (see below).

ghostNeighbors

Containers::Vector< GlobalIndexType, DeviceType, GlobalIndexType > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::ghostNeighbors

protected

Ghost neighbor indices: array containing local indices of the entities which are ghosts on other ranks. The indices requested by the i-th rank are in the range starting at ghostNeighborOffsets[i] (inclusive) and ending at ghostNeighborOffsets[i+1] (exclusive). These indices are used for copying the mesh function values into the sendBuffers array. Note that ghost neighbor indices cannot be made contiguous in general so we need the send buffers.

ghostOffsets

Containers::Array< GlobalIndexType, Devices::Host, int > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::ghostOffsets

protected

Ghost offsets: array of size nproc + 1 where the i-th value is the local index of the first ghost entity owned by the i-th rank. The last value is equal to the entities count on the local mesh. All ghost entities owned by the i-th rank are assumed to be indexed contiguously in the local mesh.

globalOffsets

Containers::Array< GlobalIndexType, Devices::Host, int > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::globalOffsets

protected

Global offsets: array of size nproc where the i-th value is the lowest global index of the entities owned by the i-th rank. This can be used to determine the owner of every entity based on its global index.

sendBuffers

Containers::Array< std::uint8_t, DeviceType, GlobalIndexType > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::sendBuffers

protected

Send buffers: array for buffering the mesh function values which will be sent to other ranks. We use std::uint8_t as the value type to make this class independent of the mesh function's real type. When cast to the real type values, the send buffer for the i-th rank is the part of the array starting at index ghostNeighborOffsets[i] (inclusive) and ending at index ghostNeighborOffsets[i+1] (exclusive).

---

The documentation for this class was generated from the following file:

• src/TNL/Meshes/DistributedMeshes/DistributedGridSynchronizer.h