#include <IndexMap.h>

Public Member Functions
	IndexMap (MPI_Comm comm, std::int32_t local_size)
	Create an non-overlapping index map.
	IndexMap (MPI_Comm comm, std::int32_t local_size, std::span< const std::int64_t > ghosts, std::span< const int > owners, int tag=static_cast< int >(dolfinx::MPI::tag::consensus_nbx))
	Create an overlapping (ghosted) index map.
	IndexMap (MPI_Comm comm, std::int32_t local_size, const std::array< std::vector< int >, 2 > &src_dest, std::span< const std::int64_t > ghosts, std::span< const int > owners)
	Create an overlapping (ghosted) index map.
	IndexMap (const IndexMap &map)=delete
	IndexMap (IndexMap &&map)=default
	Move constructor.
	~IndexMap ()=default
	Destructor.
IndexMap &	operator= (IndexMap &&map)=default
	Move assignment.
IndexMap &	operator= (const IndexMap &map)=delete
std::array< std::int64_t, 2 >	local_range () const noexcept
	Range of indices (global) owned by this process.
std::int32_t	num_ghosts () const noexcept
	Number of ghost indices on this process.
std::int32_t	size_local () const noexcept
	Number of indices owned by this process.
std::int64_t	size_global () const noexcept
	Number indices across communicator.
std::span< const std::int64_t >	ghosts () const noexcept
MPI_Comm	comm () const
	Return the MPI communicator that the map is defined on.
void	local_to_global (std::span< const std::int32_t > local, std::span< std::int64_t > global) const
	Compute global indices for array of local indices.
void	global_to_local (std::span< const std::int64_t > global, std::span< std::int32_t > local) const
	Compute local indices for array of global indices.
std::vector< std::int64_t >	global_indices () const
	Build list of indices with global indexing.
std::span< const int >	owners () const
	The ranks that own each ghost index.
graph::AdjacencyList< int >	index_to_dest_ranks (int tag=static_cast< int >(dolfinx::MPI::tag::consensus_nbx)) const
	Compute map from each local (owned) index to the set of ranks that have the index as a ghost.
std::vector< std::int32_t >	shared_indices () const
	Build a list of owned indices that are ghosted by another rank.
std::span< const int >	src () const noexcept
	Ordered set of MPI ranks that own caller's ghost indices.
std::span< const int >	dest () const noexcept
	Ordered set of MPI ranks that ghost indices owned by caller.
std::vector< std::int32_t >	weights_src () const
	Compute the number of ghost indices owned by each rank in IndexMap::src.
std::vector< std::int32_t >	weights_dest () const
	Compute the number of ghost indices owned by each rank in IndexMap::dest.
std::array< std::vector< int >, 2 >	rank_type (int split_type) const
	Destination and source ranks by type, e.g, ranks that are destination/source ranks for the caller and are in a common shared memory region.

Detailed Description

This class represents the distribution index arrays across processes. An index array is a contiguous collection of N+1 indices [0, 1, . . ., N] that are distributed across M processes. On a given process, the IndexMap stores a portion of the index set using local indices [0, 1, . . . , n], and a map from the local indices to a unique global index.

Constructor & Destructor Documentation

◆ IndexMap() [1/3]

IndexMap	(	MPI_Comm	comm,
		std::int32_t	local_size )

Create an non-overlapping index map.

Note: Collective

Parameters

[in]	comm	MPI communicator that the index map is distributed across.
[in]	local_size	Local size of the index map, i.e. the number of owned entries.

◆ IndexMap() [2/3]

IndexMap	(	MPI_Comm	comm,
		std::int32_t	local_size,
		std::span< const std::int64_t >	ghosts,
		std::span< const int >	owners,
		int	tag = static_cast<int>(dolfinx::MPI::tag::consensus_nbx) )

Create an overlapping (ghosted) index map.

This constructor uses a 'consensus' algorithm to determine the ranks that ghost indices that are owned by the caller. This requires non-trivial MPI communication. If the ranks that ghost indices owned by the caller are known, it more efficient to use the constructor that takes these ranks as an argument.

Note: Collective

Parameters

[in]	comm	MPI communicator that the index map is distributed across.
[in]	local_size	Local size of the index map, i.e. the number of owned entries
[in]	ghosts	The global indices of ghost entries
[in]	owners	Owner rank (on comm) of each entry in ghosts
[in]	tag	Tag used in non-blocking MPI calls in the consensus algorithm.

Note: A tag can sometimes be required when there are a series of calls to this constructor, or other functions that call the consensus algorithm, that are close together. In cases where this constructor is called a second time on rank and another rank has not completed its first consensus algorithm call, communications can be corrupted if each collective call of this constructor does not have its own tag value. Each collective call to this constructor must use the same tag value. An alternative to passing a tag is to have an implicit or explicit MPI barrier before and after the call to this constructor.

◆ IndexMap() [3/3]

IndexMap	(	MPI_Comm	comm,
		std::int32_t	local_size,
		const std::array< std::vector< int >, 2 > &	src_dest,
		std::span< const std::int64_t >	ghosts,
		std::span< const int >	owners )

Create an overlapping (ghosted) index map.

This constructor is optimised for the case where the 'source' (ranks that own indices ghosted by the caller) and 'destination' ranks (ranks that ghost indices owned by the caller) are already available. It allows the complex computation of the destination ranks from owners.

Note: Collective

Parameters

[in]	comm	MPI communicator that the index map is distributed across.
[in]	local_size	Local size of the index map, i.e. the number
[in]	src_dest	Lists of [0] src and [1] dest ranks. The list in each must be sorted and not contain duplicates. src ranks are owners of the indices in ghosts. dest ranks are the rank that ghost indices owned by the caller.
[in]	ghosts	The global indices of ghost entries
[in]	owners	Owner rank (on comm) of each entry in ghosts

Member Function Documentation

◆ comm()

MPI_Comm comm ( ) const

Return the MPI communicator that the map is defined on.

Returns: Communicator

◆ dest()

std::span< const int > dest ( ) const

noexcept

Ordered set of MPI ranks that ghost indices owned by caller.

Typically used when creating neighbourhood communicators.

Returns: MPI ranks than own ghost indices. The ranks are unique and sorted.

◆ ghosts()

std::span< const std::int64_t > ghosts ( ) const

noexcept

Local-to-global map for ghosts (local indexing beyond end of local range)

◆ global_indices()

std::vector< std::int64_t > global_indices ( ) const

Build list of indices with global indexing.

Returns: The global index for all local indices (0, 1, 2, ...) on this process, including ghosts

◆ global_to_local()

void global_to_local	(	std::span< const std::int64_t >	global,
		std::span< std::int32_t >	local ) const

Compute local indices for array of global indices.

Parameters

[in]	global	Global indices
[out]	local	The local of the corresponding global index in 'global'. Returns -1 if the local index does not exist on this process.

◆ index_to_dest_ranks()

graph::AdjacencyList< int > index_to_dest_ranks ( int tag = static_cast<int>(dolfinx::MPI::tag::consensus_nbx) ) const

Compute map from each local (owned) index to the set of ranks that have the index as a ghost.

Todo: Aim to remove this function?

Note: Collective

Parameters

[in] tag Tag to pass to MPI calls.

Note: See IndexMap(MPI_Comm,std::int32_t,std::span<const std::int64_t>,std::span<const int>,int) for an explanation of when tag is required.

Returns: Shared indices.

◆ local_to_global()

void local_to_global	(	std::span< const std::int32_t >	local,
		std::span< std::int64_t >	global ) const

Compute global indices for array of local indices.

Parameters

[in]	local	Local indices
[out]	global	The global indices

◆ owners()

std::span< const int > owners ( ) const

inline

The ranks that own each ghost index.

Returns: List of ghost owners. The owning rank of the ith ghost index is owners()[i].

◆ rank_type()

std::array< std::vector< int >, 2 > rank_type ( int split_type ) const

Destination and source ranks by type, e.g, ranks that are destination/source ranks for the caller and are in a common shared memory region.

This function is used to group destination and source ranks by 'type'. The type is defined by the MPI split_type. Split types include ranks from a common shared memory region (MPI_COMM_TYPE_SHARED) or a common NUMA region. Splits types are listed at https://docs.open-mpi.org/en/main/man-openmpi/man3/MPI_Comm_split_type.3.html#split-types.

Note: Collective operation on comm().

Parameters

[in] split_type MPI split type, as used in the function MPI_Comm_split_type. See https://docs.open-mpi.org/en/main/man-openmpi/man3/MPI_Comm_split_type.3.html#split-types.

Returns: (0) Intersection of ranks in split_type and in dest(), and (1) intersection of ranks in split_type and in src(). Returned ranks are on the comm() communicator.

◆ shared_indices()

std::vector< std::int32_t > shared_indices ( ) const

Build a list of owned indices that are ghosted by another rank.

Returns: The local index of owned indices that are ghosts on other rank(s). The indices are unique and sorted.

◆ src()

std::span< const int > src ( ) const

noexcept

Ordered set of MPI ranks that own caller's ghost indices.

Typically used when creating neighbourhood communicators.

Returns: MPI ranks than own ghost indices. The ranks are unique and sorted.

◆ weights_dest()

std::vector< std::int32_t > weights_dest ( ) const

Compute the number of ghost indices owned by each rank in IndexMap::dest.

This is a measure of the amount of data:

Sent from this rank to other ranks when performing a forward (owner -> ghost) scatter.
Received by this rank from other ranks when performing a reverse forward (owner <- ghost) scatter.

Returns: A weight vector, where weight[i] the the number of ghost indices owned by rank IndexMap::dest()[i].

◆ weights_src()

std::vector< std::int32_t > weights_src ( ) const

Compute the number of ghost indices owned by each rank in IndexMap::src.

This is a measure of the amount of data:

Sent from this rank to other ranks when performing a reverse (owner <- ghost) scatter.
Received by this rank from other ranks when performing a forward (owner -> ghost) scatter.

Returns: A weight vector, where weight[i] the the number of ghost indices owned by rank IndexMap::src()[i].

The documentation for this class was generated from the following files:

/__w/dolfinx/dolfinx/cpp/dolfinx/common/IndexMap.h
/__w/dolfinx/dolfinx/cpp/dolfinx/common/IndexMap.cpp

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ IndexMap() [1/3]

◆ IndexMap() [2/3]

◆ IndexMap() [3/3]

Member Function Documentation

◆ comm()

◆ dest()

◆ ghosts()

◆ global_indices()

◆ global_to_local()

◆ index_to_dest_ranks()

◆ local_to_global()

◆ owners()

◆ rank_type()

◆ shared_indices()

◆ src()

◆ weights_dest()

◆ weights_src()