dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc > Class Template Reference

Global memory region with dynamic size. More...

#include <GlobHeapMem.h>

Public Types
using	value_type = typename allocator_traits::value_type
using	allocator_type = typename allocator_traits::allocator_type
using	local_memory_space = typename memory_traits::memory_space_type
typedef allocator_type::size_type	size_type
typedef allocator_type::difference_type	difference_type
typedef allocator_type::difference_type	index_type
typedef allocator_type::pointer	raw_pointer
typedef GlobHeapLocalPtr< value_type, index_type >	local_pointer
typedef GlobHeapLocalPtr< value_type, index_type >	const_local_pointer
typedef GlobHeapPtr< value_type, GlobHeapMem >	pointer
typedef GlobHeapPtr< const value_type, GlobHeapMem >	const_pointer
typedef value_type &	local_reference
typedef const value_type &	const_local_reference
typedef local_pointer::bucket_type	bucket_type
template<typename U >
using	rebind = GlobHeapMem< U, LocalMemorySpace, AllocationPolicy, LocalAlloc >
using	local_void_pointer = void *

Public Member Functions
	GlobHeapMem (size_type n_local_elem=0)
	Constructor, collectively allocates the given number of elements in local memory of every unit in dash::Team::All(). More...
	GlobHeapMem (size_type n_local_elem, dash::Team &team)
	Constructor, collectively allocates the given number of elements in local memory of every unit in a team. More...
	GlobHeapMem (size_type n_local_elem, LocalMemorySpace *r)
	Constructor, collectively allocates the given number of elements in local memory of every unit in dash::Team::All(). More...
	GlobHeapMem (size_type n_local_elem, LocalMemorySpace *r, Team &team)
	~GlobHeapMem ()
	Destructor, collectively frees underlying global memory. More...
	GlobHeapMem (const self_t &other)=default
	Copy constructor. More...
self_t &	operator= (const self_t &rhs)=default
	Assignment operator. More...
constexpr bool	operator== (const self_t &rhs) const noexcept
	Equality comparison operator. More...
constexpr bool	operator!= (const self_t &rhs) const noexcept
	Inequality comparison operator. More...
constexpr size_type	size () const noexcept
	Total number of elements in attached memory space, including size of local unattached memory segments. More...
constexpr size_type	local_size () const noexcept
	Number of elements in local memory space. More...
size_type	local_size (team_unit_t unit) const
	Number of elements in local memory space of given unit. More...
constexpr dash::Team &	team () const noexcept
	The team containing all units accessing the global memory space. More...
local_pointer	grow (size_type num_elements)
	Increase capacity of local segment of global memory region by the given number of elements. More...
void	shrink (size_type num_elements)
	Decrease capacity of local segment of global memory region by the given number of elements. More...
void	commit ()
	Commit changes of local memory region to global memory space. More...
void	resize (size_type num_elements)
	Resize capacity of local segment of global memory region to the given number of elements. More...
pointer	begin () noexcept
	Global pointer of the initial address of the global memory. More...
constexpr const_pointer	begin () const noexcept
	Global pointer of the initial address of the global memory. More...
pointer	end () noexcept
	Global pointer of the initial address of the global memory. More...
const_pointer	end () const noexcept
	Global pointer of the initial address of the global memory. More...
local_pointer &	lbegin () noexcept
	Native pointer of the initial address of the local memory of the unit that initialized this GlobHeapMem instance. More...
const_local_pointer	lbegin () const noexcept
	Native pointer of the initial address of the local memory of the unit that initialized this GlobHeapMem instance. More...
local_pointer &	lend () noexcept
	Native pointer of the initial address of the local memory of the unit that initialized this GlobHeapMem instance. More...
const_local_pointer &	lend () const noexcept
	Native pointer of the initial address of the local memory of the unit that initialized this GlobHeapMem instance. More...
template<typename ValueType = value_type>
void	put_value (const ValueType &newval, index_type global_index)
	Write value to global memory at given offset. More...
template<typename ValueType = value_type>
void	get_value (ValueType *ptr, index_type global_index) const
	Read value from global memory at given offset. More...
void	barrier () const
	Synchronize all units associated with this global memory instance. More...
template<typename IndexT >
pointer	at (team_unit_t unit, IndexT local_index)
	Resolve the global iterator referencing an element position in a unit's local memory. More...
template<typename IndexT >
const_pointer	at (team_unit_t unit, IndexT local_index) const
	Resolve the global iterator referencing an element position in a unit's local memory. More...
const bucket_list &	local_buckets () const

Detailed Description

template<typename ElementType, class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>
class dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >

Global memory region with dynamic size.

For global memory spaces with static size, see dash::GlobStaticMem.

See also

dash::GlobStaticMem

Conventional global memory (see dash::GlobStaticMem) allocates a single contiguous range of fixed size in local memory at every unit. Iterating static memory space is trivial as native pointer arithmetics can be used to traverse elements in canonical storage order.

In global dynamic memory, units allocate multiple heap-allocated buckets in local memory. The number of local buckets and their sizes may differ between units. In effect, elements in local memory are distributed in non-contiguous address ranges and a custom iterator is used to access elements in logical storage order.

Usage examples:

size_t initial_local_capacity  = 1024;
GlobHeapMem<double> gdmem(initial_local_capacity);

size_t initial_global_capacity = dash::size() * initial_local_capacity;

if (dash::myid() == 0) {
  // Allocate another 512 elements in local memory space.
  // This is a local operation, the additionally allocated memory
  // space is only accessible by the local unit, however:
  gdmem.grow(512);
}
if (dash::myid() == 1) {
  // Decrease capacity of local memory space by 128 units.
  // This is a local operation. New size of logical memory space is
  // effective for the local unit immediately but memory is not
  // physically freed yet and is still accessible by other units.
  gdmem.shrink(128);
}

// Global memory space has not been updated yet, changes are only
// visible locally:
if (dash::myid() == 0) {
  assert(gdmem.size() == initial_global_capacity + 512);
} else if (dash::myid() == 1) {
  assert(gdmem.size() == initial_global_capacity - 128);
} else {
  assert(gdmem.size() == initial_global_capacity);
}

auto unit_0_local_size = gdmem.lend(0) - gdmem.lbegin(0);
auto unit_1_local_size = gdmem.lend(1) - gdmem.lbegin(1);

if (dash::myid() == 0) {
  assert(unit_0_local_size == 1024 + 512);
  assert(unit_0_local_size == gdmem.local_size());
} else {
  assert(unit_0_local_size == initial_local_capacity);
}
if (dash::myid() == 1) {
  assert(unit_1_local_size == 1024 - 128);
  assert(unit_1_local_size == gdmem.local_size());
} else {
  assert(unit_1_local_size == initial_local_capacity);
}

// Memory marked for deallocation is still accessible by other units:
if (dash::myid() != 1) {
  auto unit_1_last = gdmem.at(dash::myid(), initial_local_capacity-1);
  double * value;
  gdmem.get_value(value, unit_1_last);
}

// Collectively commit changes of local memory allocation to global
// memory space:
// register newly allocated local memory and remove local memory marked
// for deallocation.
gdmem.commit();

// Changes are globally visible now:
assert(gdmem.size() == initial_global_capacity + 512 - 128);

unit_0_local_size = gdmem.lend(0) - gdmem.lbegin(0);
unit_1_local_size = gdmem.lend(1) - gdmem.lbegin(1);
assert(unit_0_local_size == 1024 + 512);
assert(unit_1_local_size == 1024 - 128);

Implemented concept:

DashMemorySpaceConcept

Implemented concept:

Global Dynamic Memory Concept

Definition at line 204 of file GlobHeapMem.h.

Constructor & Destructor Documentation

~GlobHeapMem()

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::~GlobHeapMem ( )

inline

Destructor, collectively frees underlying global memory.

Definition at line 380 of file GlobHeapMem.h.

References dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::operator=().

  {
    DASH_LOG_TRACE("GlobHeapMem.~GlobHeapMem()");
  }

GlobHeapMem()

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::GlobHeapMem ( const self_t &  other )

default

Copy constructor.

Member Function Documentation

at() [1/2]

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

template<typename IndexT >

pointer dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::at ( team_unit_t  unit, IndexT  local_index  )

inline

Resolve the global iterator referencing an element position in a unit's local memory.

Parameters

unit	The unit id
local_index	The unit's local address offset

Definition at line 913 of file GlobHeapMem.h.

Referenced by dash::UnorderedMapLocalIter< Key, Mapped, Hash, Pred, LMemSpace >::dart_gptr().

  {
    DASH_LOG_DEBUG("GlobHeapMem.at()",
                   "unit:", unit, "lidx:", local_index);
    if (_nunits == 0) {
      DASH_THROW(dash::exception::RuntimeError, "No units in team");
    }
    pointer git(this, unit, local_index);
    DASH_LOG_DEBUG_VAR("GlobHeapMem.at >", git);
    return git;
  }

at() [2/2]

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

template<typename IndexT >

const_pointer dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::at ( team_unit_t  unit, IndexT  local_index  ) const

inline

Resolve the global iterator referencing an element position in a unit's local memory.

Parameters

unit	The unit id
local_index	The unit's local address offset

Definition at line 934 of file GlobHeapMem.h.

  {
    DASH_LOG_DEBUG("GlobHeapMem.at() const",
                   "unit:", unit, "lidx:", local_index);
    if (_nunits == 0) {
      DASH_THROW(dash::exception::RuntimeError, "No units in team");
    }
    const_pointer git(this, unit, local_index);
    DASH_LOG_DEBUG_VAR("GlobHeapMem.at const >", git);
    return git;
  }

barrier()

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

void dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::barrier ( ) const

inline

Synchronize all units associated with this global memory instance.

Does not commit changes of local memory space.

See also

commit

Definition at line 901 of file GlobHeapMem.h.

References dart_barrier(), and DART_OK.

  {
    DASH_ASSERT_RETURNS(
      dart_barrier(_teamid),
      DART_OK);
  }

begin() [1/2]

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

pointer dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::begin ( )

inlinenoexcept

Global pointer of the initial address of the global memory.

Definition at line 798 of file GlobHeapMem.h.

  {
    return pointer(this, _begin_idx);
  }

begin() [2/2]

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

constexpr const_pointer dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::begin ( ) const

inlinenoexcept

Global pointer of the initial address of the global memory.

Definition at line 806 of file GlobHeapMem.h.

  {
    return const_pointer(this, _begin_idx);
  }

commit()

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

void dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::commit ( )

inline

Commit changes of local memory region to global memory space.

Applies calls of grow(), shrink() and resize() to global memory.

Collective operation.

Attaches local memory allocated since the last call of commit() to global memory space and thus makes it accessible to other units. Frees local memory marked for deallocation and detaches it from global memory.

See also

resize

grow

shrink

Definition at line 739 of file GlobHeapMem.h.

References dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::size().

Referenced by dash::List< ElementType, LocalMemorySpace >::barrier(), and dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::GlobHeapMem().

  {
    DASH_LOG_DEBUG("GlobHeapMem.commit()");
    DASH_LOG_TRACE_VAR("GlobHeapMem.commit", _buckets.size());

    // First detach, then attach to minimize number of elements allocated
    // at the same time:
    size_type num_detached_elem = commit_detach();
    size_type num_attached_elem = commit_attach();

    if (num_detached_elem > 0 || num_attached_elem > 0) {
      // Update _begin iterator:
      DASH_LOG_TRACE("GlobHeapMem.commit", "updating _begin");
      _begin_idx = 0;
      DASH_LOG_TRACE("GlobHeapMem.commit", "updating _end");
      _end_idx   = size();
    }
    // Update local iterators as bucket iterators might have changed:
    DASH_LOG_TRACE("GlobHeapMem.commit", "updating _lbegin");
    update_lbegin();
    DASH_LOG_TRACE("GlobHeapMem.commit", "updating _lend");
    update_lend();
    DASH_LOG_DEBUG("GlobHeapMem.commit >", "finished");
  }

end() [1/2]

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

pointer dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::end ( )

inlinenoexcept

Global pointer of the initial address of the global memory.

Definition at line 814 of file GlobHeapMem.h.

  {
    return pointer(this, _end_idx);
  }

end() [2/2]

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

const_pointer dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::end ( ) const

inlinenoexcept

Global pointer of the initial address of the global memory.

Definition at line 822 of file GlobHeapMem.h.

  {
    return const_pointer(this, _end_idx);
  }

get_value()

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

template<typename ValueType = value_type>

void dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::get_value ( ValueType *  ptr, index_type  global_index  ) const

inline

Read value from global memory at given offset.

See also

dash::get_value

Definition at line 885 of file GlobHeapMem.h.

References dash::get_value().

  {
    DASH_LOG_TRACE("GlobHeapMem.get_value(newval, gidx = %d)",
                   global_index);
    auto git = pointer(this, global_index);
    dash::get_value(ptr, git);
  }

grow()

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

local_pointer dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::grow ( size_type  num_elements )

inline

Increase capacity of local segment of global memory region by the given number of elements.

Same as resize(size() + num_elements).

Local operation. Newly allocated memory is attached to global memory space by calling the collective operation attach().

Returns

Native pointer to beginning of new allocated memory.

See also

resize

shrink

commit

Definition at line 485 of file GlobHeapMem.h.

References DART_GPTR_NULL, and dash::Array< ElementType, IndexType, PatternType, LocalMemSpaceT >::local.

Referenced by dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::GlobHeapMem(), dash::LocalListRef< ElementType, LocalMemorySpace >::push_back(), and dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::resize().

  {
    DASH_LOG_DEBUG_VAR("GlobHeapMem.grow()", num_elements);
    size_type local_size_old = _local_sizes.local[0];
    DASH_LOG_TRACE("GlobHeapMem.grow",
                   "current local size:", local_size_old);
    if (num_elements == 0) {
      DASH_LOG_DEBUG("GlobHeapMem.grow >", "no grow");
      return _lend;
    }
    // Update size of local memory space:
    _local_sizes.local[0]        += num_elements;
    // Update number of local buckets marked for attach:
    _num_attach_buckets.local[0] += 1;

    // Create new unattached bucket:
    DASH_LOG_TRACE("GlobHeapMem.grow", "creating new unattached bucket:",
                   "size:", num_elements);
    bucket_type bucket;
    bucket.size               = num_elements;
    bucket.allocated_size     = num_elements;
    bucket.lptr               = _allocator.allocate_local(bucket.size);
    bucket.gptr               = DART_GPTR_NULL;
    bucket.attached           = false;
    // Add bucket to local memory space:
    _buckets.push_back(bucket);
    if (_attach_buckets_first == _buckets.end()) {
      // Move iterator to first unattached bucket to position of new bucket:
      _attach_buckets_first = _buckets.begin();
      std::advance(_attach_buckets_first,  _buckets.size() - 1);
    }
    _bucket_cumul_sizes[_myid].push_back(_local_sizes.local[0]);
    DASH_LOG_TRACE("GlobHeapMem.grow", "added unattached bucket:",
                   "allocated size:", bucket.size,
                   "lptr:", bucket.lptr);
    // Update local iteration space:
    update_lbegin();
    update_lend();
    DASH_ASSERT_EQ(_local_sizes.local[0], _lend - _lbegin,
                   "local size differs from local iteration space size");
    DASH_LOG_TRACE("GlobHeapMem.grow",
                   "new local size:",     _local_sizes.local[0]);
    DASH_LOG_TRACE("GlobHeapMem.grow",
                   "local buckets:",      _buckets.size(),
                   "unattached buckets:", _num_attach_buckets.local[0]);
    DASH_LOG_TRACE("GlobHeapMem.grow >");
    // Return local iterator to start of allocated memory:
    return _lbegin + local_size_old;
  }

lbegin() [1/2]

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

local_pointer& dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::lbegin ( )

inlinenoexcept

Native pointer of the initial address of the local memory of the unit that initialized this GlobHeapMem instance.

Definition at line 831 of file GlobHeapMem.h.

Referenced by dash::List< ElementType, LocalMemorySpace >::allocate(), dash::UnorderedMapLocalIter< Key, Mapped, Hash, Pred, LMemSpace >::operator pointer(), dash::UnorderedMapLocalIter< Key, Mapped, Hash, Pred, LMemSpace >::operator*(), and dash::LocalListRef< ElementType, LocalMemorySpace >::push_back().

  {
    return _lbegin;
  }

lbegin() [2/2]

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

const_local_pointer dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::lbegin ( ) const

inlinenoexcept

Native pointer of the initial address of the local memory of the unit that initialized this GlobHeapMem instance.

Definition at line 840 of file GlobHeapMem.h.

  {
    return _lbegin;
  }

lend() [1/2]

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

local_pointer& dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::lend ( )

inlinenoexcept

Native pointer of the initial address of the local memory of the unit that initialized this GlobHeapMem instance.

Definition at line 849 of file GlobHeapMem.h.

  {
    return _lend;
  }

lend() [2/2]

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

const_local_pointer& dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::lend ( ) const

inlinenoexcept

Native pointer of the initial address of the local memory of the unit that initialized this GlobHeapMem instance.

Definition at line 858 of file GlobHeapMem.h.

  {
    return _lend;
  }

local_size() [1/2]

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

constexpr size_type dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::local_size ( ) const

inlinenoexcept

Number of elements in local memory space.

Definition at line 430 of file GlobHeapMem.h.

References dash::Array< ElementType, IndexType, PatternType, LocalMemSpaceT >::local.

Referenced by dash::List< ElementType, LocalMemorySpace >::lcapacity(), dash::LocalListRef< ElementType, LocalMemorySpace >::push_back(), dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::shrink(), and dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::size().

  {
    return _local_sizes.local[0];
  }

local_size() [2/2]

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

size_type dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::local_size ( team_unit_t  unit ) const

inline

Number of elements in local memory space of given unit.

Returns

Local capacity as published by the specified unit in last commit.

Definition at line 441 of file GlobHeapMem.h.

  {
    DASH_LOG_TRACE("GlobHeapMem.local_size(u)", "unit:", unit);
    DASH_ASSERT_RANGE(0, unit, _nunits-1, "unit id out of range");
    DASH_LOG_TRACE_VAR("GlobHeapMem.local_size",
                       _bucket_cumul_sizes[unit]);
    size_type unit_local_size;
    if (unit == _myid) {
      // Value of _local_sizes[u] is the local size as visible by the unit,
      // i.e. including size of unattached buckets.
      unit_local_size = _local_sizes.local[0];
    } else {
      unit_local_size = _bucket_cumul_sizes[unit].back();
    }
    DASH_LOG_TRACE("GlobHeapMem.local_size >", unit_local_size);
    return unit_local_size;
  }

operator!=()

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

constexpr bool dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::operator!= ( const self_t &  rhs ) const

inlinenoexcept

Inequality comparison operator.

Definition at line 413 of file GlobHeapMem.h.

  {
    return !(*this == rhs);
  }

operator=()

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

self_t& dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::operator= ( const self_t &  rhs )

default

Assignment operator.

Referenced by dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::~GlobHeapMem().

operator==()

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

constexpr bool dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::operator== ( const self_t &  rhs ) const

inlinenoexcept

Equality comparison operator.

Definition at line 400 of file GlobHeapMem.h.

  {
    return (_teamid         == rhs._teamid &&
            _nunits         == rhs._nunits &&
            _lbegin         == rhs._lbegin &&
            _lend           == rhs._lend &&
            _buckets        == rhs._buckets &&
            _detach_buckets == rhs._detach_buckets);
  }

put_value()

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

template<typename ValueType = value_type>

void dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::put_value ( const ValueType &  newval, index_type  global_index  )

inline

Write value to global memory at given offset.

See also

dash::put_value

Definition at line 869 of file GlobHeapMem.h.

References dash::put_value().

  {
    DASH_LOG_TRACE("GlobHeapMem.put_value(newval, gidx = %d)",
                   global_index);
    auto git = pointer(this, global_index);
    dash::put_value(newval, git);
  }

resize()

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

void dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::resize ( size_type  num_elements )

inline

Resize capacity of local segment of global memory region to the given number of elements.

Local operation.

If capacity is increased, newly allocated memory is only attached to global memory space and thus made accessible to other units by calling the collective operation commit(). If capacity is decreased, resizes logical local memory space but does not deallocate memory. Local memory is accessible by other units until deallocated and detached from global memory space by calling the collective operation commit().

See also

grow

shrink

commit

Definition at line 783 of file GlobHeapMem.h.

References dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::grow(), dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::shrink(), and dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::size().

  {
    DASH_LOG_DEBUG("GlobHeapMem.resize()", "new size:", num_elements);
    index_type diff_capacity = num_elements - size();
    if (diff_capacity > 0) {
      grow(diff_capacity);
    } else if (diff_capacity < 0) {
      shrink(-diff_capacity);
    }
    DASH_LOG_DEBUG("GlobHeapMem.resize >");
  }

shrink()

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

void dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::shrink ( size_type  num_elements )

inline

Decrease capacity of local segment of global memory region by the given number of elements.

Buckets are not deallocated until next commit as other units might still reference them. Same as resize(size() - num_elements).

Local operation.

Resizes logical local memory space but does not deallocate memory. Local memory is accessible by other units until deallocated and detached from global memory space by calling the collective operation commit().

See also

resize

grow

commit

Definition at line 553 of file GlobHeapMem.h.

References dash::distance(), dash::Array< ElementType, IndexType, PatternType, LocalMemSpaceT >::local, dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::local_size(), and dash::LocalArrayRef< T, IndexType, PatternType, LocalMemSpaceT >::size().

Referenced by dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::resize().

  {
    // This function updates the size of the local memory space of the
    // calling unit u.
    // The following members are updated:
    //
    // _local_sizes[u]:
    //   Size of local memory space as visible to unit u.
    //
    // _bucket_cumul_sizes:
    //    An array mapping units to a list of their cumulative bucket sizes
    //    (i.e. postfix sum) which is required to iterate over the
    //    non-contigous global dynamic memory space.
    //    For example, if unit 2 allocated buckets with sizes 1, 3 and 5,
    //    _bucket_cumul_sizes[2] is a list { 1, 4, 9 }.
    //
    // _buckets:
    //    List of local buckets that provide the underlying storage of the
    //    active unit's local memory space.
    //
    // Notes:
    //
    // It must be ensured that the updated cumulative bucket sizes of a
    // remote unit can be resolved in \c update_local_size() after any
    // possible combination of grow- and shrink-operations at the remote unit
    // from the following information:
    //
    // - the cumulative bucket sizes of the remote unit at the time of the
    //   last commit
    // - the remote unit's local size at the time of the last commit
    // - the remote unit's current local size (including unattached buckets)
    // - the number of the remote unit's unattached buckets and their size

    DASH_LOG_DEBUG_VAR("GlobHeapMem.shrink()", num_elements);
    DASH_ASSERT_LT(num_elements, local_size() + 1,
                   "cannot shrink size " << local_size() << " "
                   "by " << num_elements << " elements");
    if (num_elements == 0) {
      DASH_LOG_DEBUG("GlobHeapMem.shrink >", "no shrink");
      return;
    }
    DASH_LOG_TRACE("GlobHeapMem.shrink",
                   "current local size:", _local_sizes.local[0]);
    DASH_LOG_TRACE("GlobHeapMem.shrink",
                   "current local buckets:", _buckets.size());
    // Position of iterator to first unattached bucket:
    auto attach_buckets_first_pos = std::distance(_buckets.begin(),
                                                  _attach_buckets_first);
    DASH_LOG_TRACE_VAR("GlobHeapMem.shrink", attach_buckets_first_pos);
    // Number of elements left to deallocate:
    auto num_dealloc = num_elements;
    // Try to reduce local capacity by deallocating un-attached local buckets
    // as they do not have to be detached collectively.
    // Unattached buckets can be removed from memory space immediately as
    // remote units cannot have a pending reference on them.
    while (!_buckets.back().attached && num_dealloc > 0) {
      bucket_type & bucket_last = _buckets.back();
      // Shrink / remove unattached buckets starting at newest bucket:
      if (bucket_last.size <= num_dealloc) {
        DASH_LOG_TRACE("GlobHeapMem.shrink", "remove unattached bucket:",
                       "size:", bucket_last.size);
        // Mark entire bucket for deallocation below:
        num_dealloc           -= bucket_last.size;
        _local_sizes.local[0] -= bucket_last.size;
        _bucket_cumul_sizes[_myid].pop_back();
        // End iterator of _buckets about to change, update iterator to first
        // unattached bucket if it references the removed bucket:
        auto it_last_bucket = _buckets.end();
        std::advance(it_last_bucket, -1);
        if (_attach_buckets_first == it_last_bucket) {
          // Iterator to first unattached bucket references last bucket:
          DASH_LOG_TRACE("GlobHeapMem.shrink",
                         "updating iterator to first unattached bucket");
          std::advance(_attach_buckets_first, -1);
        }
        _allocator.deallocate_local(bucket_last.lptr, bucket_last.allocated_size);
        _buckets.pop_back();
        if (_attach_buckets_first->attached) {
          // Updated iterator to first unattached bucket references attached
          // bucket:
          _attach_buckets_first = _buckets.end();
        }
        // Update number of local buckets marked for attach:
        DASH_ASSERT_GT(_num_attach_buckets.local[0], 0,
                       "Last bucket unattached but number of buckets marked "
                       "for attach is 0");
        _num_attach_buckets.local[0] -= 1;
      } else if (bucket_last.size > num_dealloc) {
        auto const size_new = bucket_last.size - num_dealloc;

        DASH_LOG_TRACE("GlobHeapMem.shrink", "shrink unattached bucket:",
                       "old size:", bucket_last.size,
                       "new size:", size_new);
        _local_sizes.local[0]             -= num_dealloc;
        _bucket_cumul_sizes[_myid].back() -= num_dealloc;
        num_dealloc = 0;

        //Deallocate old local bucket
        _allocator.deallocate_local(bucket_last.lptr, bucket_last.allocated_size);
        //Allocate new bucket with specified size
        bucket_last.lptr = _allocator.allocate_local(size_new);
        if (bucket_last.lptr == nullptr) {
          DASH_THROW(dash::exception::RuntimeError,
                     "GlobHeapMem.shrink: Allocating bucket of size failed");
        }
        bucket_last.size = size_new;
        bucket_last.allocated_size = size_new;
      }
    }
    // Number of elements to deallocate exceeds capacity of un-attached
    // buckets, deallocate attached buckets:
    auto num_dealloc_gbuckets = 0;
    // Shrink attached buckets starting at newest bucket:
    for (auto bucket_it = _buckets.rbegin();
         bucket_it != _buckets.rend();
         ++bucket_it) {
      if (!bucket_it->attached) {
        continue;
      }
      if (num_dealloc == 0) {
        break;
      }
      if (bucket_it->size <= num_dealloc) {
        // mark entire bucket for deallocation:
        num_dealloc_gbuckets++;
        _num_detach_buckets.local[0]      += 1;
        _local_sizes.local[0]             -= bucket_it->size;
        _bucket_cumul_sizes[_myid].back() -= bucket_it->size;
        num_dealloc                       -= bucket_it->size;
      } else if (bucket_it->size > num_dealloc) {
        DASH_LOG_TRACE("GlobHeapMem.shrink", "shrink attached bucket:",
                       "old size:", bucket_it->size,
                       "new size:", bucket_it->size - num_dealloc);
        bucket_it->size                   -= num_dealloc;
        _local_sizes.local[0]             -= num_dealloc;
        _bucket_cumul_sizes[_myid].back() -= num_dealloc;
        num_dealloc = 0;
      }
    }
    // Mark attached buckets for deallocation.
    // Requires separate loop as iterators on _buckets could be invalidated.
    DASH_LOG_DEBUG_VAR("GlobHeapMem.shrink", num_dealloc_gbuckets);
    while (num_dealloc_gbuckets-- > 0) {
      auto dealloc_bucket = _buckets.back();
      DASH_LOG_TRACE("GlobHeapMem.shrink", "deallocate attached bucket:"
                     "size:", dealloc_bucket.size,
                     "lptr:", dealloc_bucket.lptr);
      // Mark bucket to be detached in next call of commit():
      _detach_buckets.push_back(dealloc_bucket);
      // Unregister bucket:
      _buckets.pop_back();
    }
    // Update local iterators as bucket iterators might have changed:
    update_lbegin();
    update_lend();

    DASH_LOG_TRACE("GlobHeapMem.shrink",
                   "cumulative bucket sizes:",  _bucket_cumul_sizes[_myid]);
    DASH_LOG_TRACE("GlobHeapMem.shrink",
                   "new local size:",           _local_sizes.local[0],
                   "new iteration space size:", std::distance(
                                                  _lbegin, _lend));
    DASH_LOG_TRACE("GlobHeapMem.shrink",
                   "total number of buckets:",  _buckets.size(),
                   "unattached buckets:",       std::distance(
                                                  _attach_buckets_first,
                                                  _buckets.end()));
    DASH_LOG_DEBUG("GlobHeapMem.shrink >");
  }

size()

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

constexpr size_type dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::size ( ) const

inlinenoexcept

Total number of elements in attached memory space, including size of local unattached memory segments.

Definition at line 422 of file GlobHeapMem.h.

References dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::local_size().

Referenced by dash::List< ElementType, LocalMemorySpace >::capacity(), dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::commit(), and dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::resize().

  {
    return _remote_size + local_size();
  }

team()

template<typename ElementType , class LocalMemorySpace = dash::HostSpace, global_allocation_policy AllocationPolicy = global_allocation_policy::epoch_synchronized, template< class > class LocalAlloc = allocator::DefaultAllocator>

constexpr dash::Team& dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::team ( ) const

inlinenoexcept

The team containing all units accessing the global memory space.

Returns

A reference to the Team containing the units associated with the global dynamic memory space.

Definition at line 465 of file GlobHeapMem.h.

References dash::Team::Null().

Referenced by dash::GlobHeapMem< ElementType, LocalMemorySpace, AllocationPolicy, LocalAlloc >::GlobHeapMem().

  {
    return (_team != nullptr) ? *_team : dash::Team::Null();
  }

---

The documentation for this class was generated from the following file:

• /tmp/tmporruphar/dash/include/dash/memory/GlobHeapMem.h