dolfinx.common#

General tools for timing and configuration.

Functions

`create_sub_index_map`(imap, indices)	Create an index map for a subset of the indices of an index map.
`index_map`(comm, local_size[, ghosts, ...])	Create an index map.
`list_timings`(comm[, reduction])	Print out a summary of all Timer measurements.
`scatterer`(index_map)	Create a scatterer for data with a layout described by an index map.
`timed`(task)	Decorator for timing functions.
`timing`(task)	Return the logged elapsed time.

Classes

`IndexMap`(imap)	Map indices across processes.
`Reduction`(*values)
`Scatterer`(s)	Scatter and gather data with a layout described by an `IndexMap`.
`Timer`([name])	A timer for timing section of code.

class dolfinx.common.IndexMap(imap: IndexMap)[source]#

Bases: object

Map indices across processes.

An index map describes the parallel distribution of a range of indices. Each index is owned by exactly one process. A process holds the indices it owns, numbered [0, size_local) locally, followed by the ‘ghost’ indices it holds but does not own, numbered [size_local, size_local + num_ghosts).

Create an index map.

Note

This initialiser is intended for internal library use only. User code should call index_map() to create an index map.

Parameters:: imap – C++ IndexMap object.

property comm: Comm#: MPI communicator that the index map is distributed over.

property ghosts: ndarray[tuple[Any, ...], dtype[int64]]#: Global index of each ghost index.

Note

The returned array is a read-only view.

global_to_local(global_index: ndarray[tuple[Any, ...], dtype[int64]]) → ndarray[tuple[Any, ...], dtype[int32]][source]#

Map global indices to local indices.

Parameters:: global_index – Global indices.
Returns:: Local index of each entry of global_index, with -1 for indices that are not owned or ghosted by the caller.

index_to_dest_ranks(tag: int) → tuple[ndarray[tuple[Any, ...], dtype[int32]], ndarray[tuple[Any, ...], dtype[int32]]][source]#

Ranks that ghost each owned index, as an adjacency list.

Parameters:: tag – MPI tag used by the consensus exchange. Must be the same on all ranks, and must not clash with another in-flight exchange.
Returns:: Ghosting ranks of each owned index, as a (data, offsets) pair.

property local_range: tuple[int, int]#: Range of global indices owned by the calling process.

local_to_global(local: ndarray[tuple[Any, ...], dtype[int32]]) → ndarray[tuple[Any, ...], dtype[int64]][source]#

Map local indices to global indices.

Parameters:: local – Local indices.
Returns:: Global index of each entry of local.

property num_ghosts: int#: Number of ghost indices on the calling process.

property owners: ndarray[tuple[Any, ...], dtype[int32]]#: Owning rank of each ghost index.

Note

The returned array is a read-only view.

property size_global: int#: Number of indices across all processes.

property size_local: int#: Number of indices owned by the calling process.

class dolfinx.common.Reduction(*values)#

Bases: Enum

average = 0#

max = 1#

min = 2#

class dolfinx.common.Scatterer(s: Scatterer)[source]#

Bases: object

Scatter and gather data with a layout described by an IndexMap.

A scatterer is stateless: it holds only the communication pattern derived from an IndexMap, and does not track buffers or the status of in-flight MPI requests. Callers of scatter_fwd_begin/ scatter_rev_begin are responsible for managing the send/receive buffers and the returned request, and can share one scatterer between multiple objects (e.g. dolfinx.la.Vector) that use the same index map.

A forward scatter sends data associated with owned/local indices to the ranks that ghost them; a reverse scatter sends ghost data back to the owning ranks, to be accumulated into the owned data. Both use the same two-step begin/end pattern, splitting the non-blocking exchange from its completion so that unrelated work can be done while communication is in flight. A round trip for a forward scatter with block size 1, where x holds the owned data and x_ghost the ghost data:

local_idx = sc.local_indices_block
remote_idx = sc.remote_indices_block

send_buffer = x[local_idx]
recv_buffer = np.empty(remote_idx.size, dtype=x.dtype)
request = sc.scatter_fwd_begin(send_buffer, recv_buffer, 1)
# ... unrelated work can be done here while communication is
# in flight, but send_buffer/recv_buffer must not be touched ...
sc.scatter_fwd_end(request)
x_ghost[remote_idx] = recv_buffer

A reverse scatter follows the same pattern with the roles of local_indices_block/remote_indices_block and of send_buffer/recv_buffer swapped, and accumulating (rather than assigning) into the destination array; see scatter_rev_begin() and scatter_rev_end().

Create a scatterer.

Note

This initialiser is intended for internal library use only. User code should call scatterer() to create a scatterer object.

Parameters:: s – C++ Scatterer object.

property local_indices_block: ndarray[tuple[Any, ...], dtype[int32]]#

Indices for packing/unpacking owned data in a send/recv buffer.

For a forward scatter, used to copy owned entries into a send buffer. For a reverse scatter, used to accumulate received values into the owned entries. Blocked: for block size bs a buffer holds bs values per index and must be bs * local_indices_block.size long.

property remote_indices_block: ndarray[tuple[Any, ...], dtype[int32]]#

Indices for packing/unpacking ghost data in a send/recv buffer.

For a forward scatter, used to unpack received values into ghost entries. For a reverse scatter, used to pack ghost entries into a send buffer. Blocked: for block size bs a buffer holds bs values per index and must be bs * remote_indices_block.size long.

Start a non-blocking exchange of owned data with ghosting ranks.

The communication is completed by calling scatter_fwd_end(). See local_indices_block for how to pack send_buffer and remote_indices_block for how to unpack recv_buffer.

Note

Collective. Every rank in the communicator must call this, including ranks without neighbours.

Note

send_buffer/recv_buffer must not be changed or accessed until after a call to scatter_fwd_end().

Parameters:

send_buffer – Packed owned data, blocked by bs, sized bs * local_indices_block.size.
recv_buffer – Buffer for storing received data, blocked by bs, sized bs * remote_indices_block.size.
bs – Number of values per index map index.

Returns:

Request to pass to scatter_fwd_end().

scatter_fwd_end(request: Request) → None[source]#

Complete the exchange started by scatter_fwd_begin().

Note

Local completion of the caller’s own request, not itself collective. Every rank that called scatter_fwd_begin() must still call this before reusing the buffers.

Parameters:: request – Request returned by scatter_fwd_begin().

Start a non-blocking exchange of ghost data with owning ranks.

The communication is completed by calling scatter_rev_end(). See remote_indices_block for how to pack send_buffer and local_indices_block for how to unpack (accumulate into) recv_buffer.

Note

Collective. Every rank in the communicator must call this, including ranks without neighbours.

Note

send_buffer/recv_buffer must not be changed or accessed until after a call to scatter_rev_end().

Parameters:

send_buffer – Data associated with each ghost index, blocked by bs, sized bs * remote_indices_block.size.
recv_buffer – Buffer for storing received data, blocked by bs, sized bs * local_indices_block.size.
bs – Number of values per index map index.

Returns:

Request to pass to scatter_rev_end().

scatter_rev_end(request: Request) → None[source]#

Complete the exchange started by scatter_rev_begin().

Note

Local completion of the caller’s own request, not itself collective. Every rank that called scatter_rev_begin() must still call this before reusing the buffers.

Parameters:: request – Request returned by scatter_rev_begin().

class dolfinx.common.Timer(name: str | None = None)[source]#

Bases: object

A timer for timing section of code.

The recommended usage is with a context manager.

Example

With a context manager, the timer is started when entering and stopped at exit. With a named Timer:

with Timer("Some costly operation"):
    costly_call_1()
    costly_call_2()

delta = timing("Some costly operation")
print(delta)

or with an un-named Timer:

with Timer() as t:
    costly_call_1()
    costly_call_2()
    print(f"Elapsed time: {t.elapsed()}")

Example

It is possible to start and stop a timer explicitly:

t = Timer("Some costly operation")
costly_call()
delta = t.stop()

and retrieve timing data using:

delta = t.elapsed()

To flush the timing data for a named Timer to the logger, the timer should be stopped and flushed:

t.stop()
t.flush()

Timings are stored globally (if task name is given) and once flushed (if used without a context manager) may be printed using functions timing() and list_timings(), e.g.:

list_timings(comm)

Create timer.

Parameters:: name – Identifier to use when storing elapsed time in logger.

elapsed() → timedelta[source]#

Return elapsed time.

Returns:: Elapsed time.

flush() → None[source]#: Flush timer duration to the logger.

Note

Timer must have been stopped before flushing.

Timer can be flushed only once. Subsequent calls will have no effect.

resume() → None[source]#: Resume timer.

start() → None[source]#: Reset elapsed time and (re-)start timer.

stop() → timedelta[source]#

Stop timer and return elapsed time.

Returns:: Elapsed time.

dolfinx.common.create_sub_index_map(imap: IndexMap, indices: ndarray[tuple[Any, ...], dtype[int32]]) → tuple[IndexMap, ndarray[tuple[Any, ...], dtype[int32]], bool][source]#

Create an index map for a subset of the indices of an index map.

An index that is included by a process that ghosts it, but not by its owner, is re-assigned to one of the including processes.

Note

Collective.

Parameters:

imap – Index map to build a sub-map of.
indices – Local indices of imap, unique and in range, to include in the sub-map.

Returns:

The sub-map, the index in imap of each of its indices, and whether any index acquired a new owner.

Note

The owner-change flag is rank-local and is not reduced, so it can differ across ranks. Reduce it (e.g. comm.allreduce(..., op=MPI.LOR)) before using it in a collective decision; branching on the unreduced value can leave some ranks in a collective that others have skipped.

dolfinx.common.index_map(comm: Comm, local_size: int, ghosts: tuple[ndarray[tuple[Any, ...], dtype[int64]], ndarray[tuple[Any, ...], dtype[int32]]] | None = None, *, dest_src: Sequence[ndarray[tuple[Any, ...], dtype[int32]]] | None = None, tag: int = 1202) → IndexMap[source]#

Create an index map.

Note

Collective. ghosts must be None on every process or given on every process, and likewise for dest_src. This is a precondition and is not checked, since checking it would require communication on every call.

Parameters:

comm – MPI communicator to distribute the indices over.
local_size – Number of indices owned by the calling process.
ghosts – Tuple (ghost_indices, owners) of global ghost indices and their owning ranks. If None, the index map is non-overlapping and ghosts must be None on every process. For an overlapping map, a process with no ghosts must pass empty arrays.
dest_src – Pair (dest, src) of destination and source rank arrays. dest lists ranks that ghost caller-owned indices; src lists ranks that own the caller’s ghosts and must equal the unique values in owners. Both arrays must be sorted, unique, and contain valid ranks. Supplying them avoids the consensus exchange that otherwise discovers which ranks ghost the caller’s owned indices.
tag – MPI tag for the consensus exchange. Ignored if dest_src is given. Must be the same on all ranks, and must not clash with another in-flight exchange.

Returns:

A new index map.

dolfinx.common.list_timings(comm: Comm, reduction: Reduction = Reduction.max) → None[source]#

Print out a summary of all Timer measurements.

When used in parallel, a reduction is applied across all processes. By default, the maximum time is shown.

dolfinx.common.scatterer(index_map: IndexMap) → Scatterer[source]#

Create a scatterer for data with a layout described by an index map.

Parameters:: index_map – Index map that describes the parallel layout of the data.
Returns:: A new scatterer.

dolfinx.common.timed(task: str) → Callable[source]#: Decorator for timing functions.

dolfinx.common.timing(task: str) → tuple[int, timedelta][source]#

Return the logged elapsed time.

Timing data is for the calling process.

Parameters:: task – The task name using when logging the time.
Returns:: (number of times logged, total wall time)