Template Numerical Library version\ main:f17d0c8
Loading...
Searching...
No Matches
Public Types | Public Member Functions | Protected Attributes | List of all members
TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension > Class Template Reference
Inheritance diagram for TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >:
Inheritance graph
[legend]
Collaboration diagram for TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >:
Collaboration graph
[legend]

Public Types

using ByteArrayView = typename Base::ByteArrayView
 
using DeviceType = typename DistributedMesh::DeviceType
 
using GlobalIndexType = typename DistributedMesh::GlobalIndexType
 
using RequestsVector = typename Base::RequestsVector
 
- Public Types inherited from TNL::Containers::ByteArraySynchronizer< DistributedMesh::DeviceType, DistributedMesh::GlobalIndexType >
enum  AsyncPolicy
 
using ByteArrayView
 
using RequestsVector
 

Public Member Functions

int getEntityOwner (GlobalIndexType global_idx) const
 
const auto & getGhostEntitiesCounts () const
 
const auto & getGhostNeighborOffsets () const
 
const auto & getGhostNeighbors () const
 
const auto & getGhostOffsets () const
 
const auto & getGlobalOffsets () const
 
void initialize (const DistributedMesh &mesh)
 
template<typename Array , std::enable_if_t< ! HasMeshType< Array >::value, bool > = true>
void synchronize (Array &array)
 
template<typename MeshFunction , std::enable_if_t< HasMeshType< MeshFunction >::value, bool > = true>
void synchronize (MeshFunction &function)
 
template<typename Array >
void synchronizeArray (Array &array, int valuesPerElement=1)
 
void synchronizeByteArray (ByteArrayView array, int bytesPerValue) override
 Main synchronization function.
 
RequestsVector synchronizeByteArrayAsyncWorker (ByteArrayView array, int bytesPerValue) override
 
template<typename SparsePattern >
auto synchronizeSparse (const SparsePattern &pattern, bool assumeConsistentRowCapacities=false)
 
- Public Member Functions inherited from TNL::Containers::ByteArraySynchronizer< DistributedMesh::DeviceType, DistributedMesh::GlobalIndexType >
void synchronizeByteArrayAsync (ByteArrayView array, int bytesPerValue, AsyncPolicy policy=AsyncPolicy::synchronous)
 An asynchronous version of synchronizeByteArray.
 

Protected Attributes

MPI::Comm communicator
 
Matrices::DenseMatrix< GlobalIndexType, Devices::Host, int > ghostEntitiesCounts
 
Containers::Array< GlobalIndexType, Devices::Host, int > ghostNeighborOffsets
 
Containers::Vector< GlobalIndexType, DeviceType, GlobalIndexType > ghostNeighbors
 
Containers::Array< GlobalIndexType, Devices::Host, int > ghostOffsets
 
Containers::Array< GlobalIndexType, Devices::Host, int > globalOffsets
 
Containers::Array< std::uint8_t, DeviceType, GlobalIndexType > sendBuffers
 

Additional Inherited Members

- Public Attributes inherited from TNL::Containers::ByteArraySynchronizer< DistributedMesh::DeviceType, DistributedMesh::GlobalIndexType >
std::future< void > async_op
 Can be used for checking if a synchronization started asynchronously has been finished.
 
std::size_t async_ops_count
 
Timer async_start_timer
 
Timer async_wait_before_start_timer
 
Timer async_wait_timer
 

Member Function Documentation

◆ synchronizeByteArray()

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>
void TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::synchronizeByteArray ( ByteArrayView array,
int bytesPerValue )
inlineoverridevirtual

Main synchronization function.

This is only a pure virtual function – the functionality must be implemented in a subclass.

Implements TNL::Containers::ByteArraySynchronizer< DistributedMesh::DeviceType, DistributedMesh::GlobalIndexType >.

◆ synchronizeByteArrayAsyncWorker()

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>
RequestsVector TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::synchronizeByteArrayAsyncWorker ( ByteArrayView array,
int bytesPerValue )
inlineoverridevirtual

Member Data Documentation

◆ ghostEntitiesCounts

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>
Matrices::DenseMatrix< GlobalIndexType, Devices::Host, int > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::ghostEntitiesCounts
protected

Communication pattern:

  • an unsymmetric nproc x nproc matrix G such that G_ij represents the number of ghost entities on rank i that are owned by rank j
  • assembly of the i-th row involves traversal of the ghost entities on the local mesh and determining its owner based on the global index
  • assembly of the full matrix needs all-to-all communication
  • for the i-th rank, the i-th row determines the receive buffer sizes and the i-th column determines the send buffer sizes

◆ ghostNeighborOffsets

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>
Containers::Array< GlobalIndexType, Devices::Host, int > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::ghostNeighborOffsets
protected

Ghost neighbor offsets: array of size nproc + 1 where the i-th value is the offset of ghost neighbor indices requested by the i-th rank. The last value is the size of the ghostNeighbors and sendBuffers arrays (see below).

◆ ghostNeighbors

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>
Containers::Vector< GlobalIndexType, DeviceType, GlobalIndexType > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::ghostNeighbors
protected

Ghost neighbor indices: array containing local indices of the entities which are ghosts on other ranks. The indices requested by the i-th rank are in the range starting at ghostNeighborOffsets[i] (inclusive) and ending at ghostNeighborOffsets[i+1] (exclusive). These indices are used for copying the mesh function values into the sendBuffers array. Note that ghost neighbor indices cannot be made contiguous in general so we need the send buffers.

◆ ghostOffsets

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>
Containers::Array< GlobalIndexType, Devices::Host, int > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::ghostOffsets
protected

Ghost offsets: array of size nproc + 1 where the i-th value is the local index of the first ghost entity owned by the i-th rank. The last value is equal to the entities count on the local mesh. All ghost entities owned by the i-th rank are assumed to be indexed contiguously in the local mesh.

◆ globalOffsets

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>
Containers::Array< GlobalIndexType, Devices::Host, int > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::globalOffsets
protected

Global offsets: array of size nproc where the i-th value is the lowest global index of the entities owned by the i-th rank. This can be used to determine the owner of every entity based on its global index.

◆ sendBuffers

template<typename DistributedMesh , int EntityDimension = DistributedMesh::getMeshDimension()>
Containers::Array< std::uint8_t, DeviceType, GlobalIndexType > TNL::Meshes::DistributedMeshes::DistributedMeshSynchronizer< DistributedMesh, EntityDimension >::sendBuffers
protected

Send buffers: array for buffering the mesh function values which will be sent to other ranks. We use std::uint8_t as the value type to make this class independent of the mesh function's real type. When cast to the real type values, the send buffer for the i-th rank is the part of the array starting at index ghostNeighborOffsets[i] (inclusive) and ending at index ghostNeighborOffsets[i+1] (exclusive).


The documentation for this class was generated from the following file: