navigate Logo

Getting Started

  • Computer Considerations
  • Computer Power Configuration
  • Software Installation
  • Launching Navigate
  • Configuring Navigate
  • Acquiring Data
  • Smart Acquisition Routines

User Guide

  • Supported Hardware
  • Supported File Formats
  • User Interface Walkthrough
  • Setting Up a Microscope
  • Acquiring Data
  • Case Studies

Development

  • Contributing Guidelines
  • Developer Install
  • Software Architecture
  • Developer Architecture Concepts
  • Feature Container
  • REST API
  • Plugin Architecture
  • Adding New Hardware

Troubleshooting & Known Issues

  • Troubleshooting
  • Known Issues

Plugins

  • navigate Plugin Template
  • navigate Confocal Projection
  • navigate at Scale
  • navigate Constant Velocity Acquisition
  • navigate MMCore
  • navigate ilastik Server
  • navigate Photoactivation

Reference

  • Implemented Microscopes
  • Software API
    • navigate.config
    • navigate.controller
    • navigate.controller.configurator
    • navigate.log_files
    • navigate.model
      • navigate.model.analysis
      • navigate.model.concurrency
        • navigate.model.concurrency.concurrency_tools
      • navigate.model.data_sources
      • navigate.model.device_startup_functions
      • navigate.model.devices
      • navigate.model.features
      • navigate.model.metadata_sources
      • navigate.model.microscope
      • navigate.model.model
      • navigate.model.plugins_model
      • navigate.model.resolution_change
      • navigate.model.waveforms
    • navigate.tools
    • navigate.view
navigate
  • Software API
  • navigate.model
  • navigate.model.concurrency
  • navigate.model.concurrency.concurrency_tools
  • navigate.model.concurrency.concurrency_tools.SharedNDArray
  • View page source

navigate.model.concurrency.concurrency_tools.SharedNDArray

class navigate.model.concurrency.concurrency_tools.SharedNDArray(shape=None, dtype=<class 'float'>, shared_memory_name=None, offset=0, strides=None, order=None)

Bases: ndarray

A numpy array that lives in shared memory

Inputs and outputs to/from ObjectInSubprocess are ‘serialized’, which is pretty fast - except for large in-memory objects. The only large in-memory objects we regularly deal with are numpy arrays, so it makes sense to provide a way to pass large numpy arrays via shared memory (which avoids slow serialization).

Maybe you wanted to write code that looks like this:

data_buf = np.zeros((400, 2000, 2000), dtype=’uint16’) display_buf = np.zeros((2000, 2000), dtype=’uint8’)

camera = Camera() preprocessor = Preprocessor() display = Display()

camera.record(num_images=400, out=data_buf) preprocessor.process(in=data_buf, out=display_buf) display.show(display_buf)

…but instead you write code that looks like this:

data_buf = SharedNDArray(shape=(400, 2000, 2000), dtype=’uint16’) display_buf = SharedNDArray(shape=(2000, 2000), dtype=’uint8’)

camera = ObjectInSubprocess(Camera) preprocessor = ObjectInSubprocess(Preprocessor) display = ObjectInSubprocess(Display)

camera.record(num_images=400, out=data_buf) preprocessor.process(in=data_buf, out=display_buf) display.show(display_buf)

…and your payoff is, each object gets its own CPU core, AND passing large numpy arrays between the processes is still really fast!

To implement this we used memmap from numpy.core as a template.

Attributes

  • T

  • base

  • ctypes

  • data

  • device

  • dtype

  • flags

  • flat

  • imag

  • itemsize

  • mT

  • nbytes

  • ndim

  • real

  • shape

  • size

  • strides

T

View of the transposed array.

Same as self.transpose().

Examples

>>> import numpy as np
>>> a = np.array([[1, 2], [3, 4]])
>>> a
array([[1, 2],
       [3, 4]])
>>> a.T
array([[1, 3],
       [2, 4]])
>>> a = np.array([1, 2, 3, 4])
>>> a
array([1, 2, 3, 4])
>>> a.T
array([1, 2, 3, 4])

See also

transpose

base

Base object if memory is from some other object.

Examples

The base of an array that owns its memory is None:

>>> import numpy as np
>>> x = np.array([1,2,3,4])
>>> x.base is None
True

Slicing creates a view, whose memory is shared with x:

>>> y = x[2:]
>>> y.base is x
True
ctypes

An object to simplify the interaction of the array with the ctypes module.

This attribute creates an object that makes it easier to use arrays when calling shared libraries with the ctypes module. The returned object has, among others, data, shape, and strides attributes (see Notes below) which themselves return ctypes objects that can be used as arguments to a shared library.

Parameters:

None –

Returns:

c – Possessing attributes data, shape, strides, etc.

Return type:

Python object

See also

numpy.ctypeslib

Notes

Below are the public attributes of this object which were documented in “Guide to NumPy” (we have omitted undocumented public attributes, as well as documented private attributes):

_ctypes.data

A pointer to the memory area of the array as a Python integer. This memory area may contain data that is not aligned, or not in correct byte-order. The memory area may not even be writeable. The array flags and data-type of this array should be respected when passing this attribute to arbitrary C-code to avoid trouble that can include Python crashing. User Beware! The value of this attribute is exactly the same as: self._array_interface_['data'][0].

Note that unlike data_as, a reference won’t be kept to the array: code like ctypes.c_void_p((a + b).ctypes.data) will result in a pointer to a deallocated array, and should be spelt (a + b).ctypes.data_as(ctypes.c_void_p)

_ctypes.shape

A ctypes array of length self.ndim where the basetype is the C-integer corresponding to dtype('p') on this platform (see ~numpy.ctypeslib.c_intp). This base-type could be ctypes.c_int, ctypes.c_long, or ctypes.c_longlong depending on the platform. The ctypes array contains the shape of the underlying array.

Type:

(c_intp*self.ndim)

_ctypes.strides

A ctypes array of length self.ndim where the basetype is the same as for the shape attribute. This ctypes array contains the strides information from the underlying array. This strides information is important for showing how many bytes must be jumped to get to the next element in the array.

Type:

(c_intp*self.ndim)

_ctypes.data_as(obj)

Return the data pointer cast to a particular c-types object. For example, calling self._as_parameter_ is equivalent to self.data_as(ctypes.c_void_p). Perhaps you want to use the data as a pointer to a ctypes array of floating-point data: self.data_as(ctypes.POINTER(ctypes.c_double)).

The returned pointer will keep a reference to the array.

_ctypes.shape_as(obj)

Return the shape tuple as an array of some other c-types type. For example: self.shape_as(ctypes.c_short).

_ctypes.strides_as(obj)

Return the strides tuple as an array of some other c-types type. For example: self.strides_as(ctypes.c_longlong).

If the ctypes module is not available, then the ctypes attribute of array objects still returns something useful, but ctypes objects are not returned and errors may be raised instead. In particular, the object will still have the as_parameter attribute which will return an integer equal to the data attribute.

Examples

>>> import numpy as np
>>> import ctypes
>>> x = np.array([[0, 1], [2, 3]], dtype=np.int32)
>>> x
array([[0, 1],
       [2, 3]], dtype=int32)
>>> x.ctypes.data
31962608 # may vary
>>> x.ctypes.data_as(ctypes.POINTER(ctypes.c_uint32))
<__main__.LP_c_uint object at 0x7ff2fc1fc200> # may vary
>>> x.ctypes.data_as(ctypes.POINTER(ctypes.c_uint32)).contents
c_uint(0)
>>> x.ctypes.data_as(ctypes.POINTER(ctypes.c_uint64)).contents
c_ulong(4294967296)
>>> x.ctypes.shape
<numpy._core._internal.c_long_Array_2 object at 0x7ff2fc1fce60> # may vary
>>> x.ctypes.strides
<numpy._core._internal.c_long_Array_2 object at 0x7ff2fc1ff320> # may vary
data

Python buffer object pointing to the start of the array’s data.

device
dtype

Data-type of the array’s elements.

Warning

Setting arr.dtype is discouraged and may be deprecated in the future. Setting will replace the dtype without modifying the memory (see also ndarray.view and ndarray.astype).

Parameters:

None –

Returns:

d

Return type:

numpy dtype object

See also

ndarray.astype

Cast the values contained in the array to a new data-type.

ndarray.view

Create a view of the same data but a different data-type.

numpy.dtype

Examples

>>> import numpy as np
>>> x = np.arange(4).reshape((2, 2))
>>> x
array([[0, 1],
       [2, 3]])
>>> x.dtype
dtype('int64')   # may vary (OS, bitness)
>>> isinstance(x.dtype, np.dtype)
True
flags

Information about the memory layout of the array.

C_CONTIGUOUS(C)

The data is in a single, C-style contiguous segment.

F_CONTIGUOUS(F)

The data is in a single, Fortran-style contiguous segment.

OWNDATA(O)

The array owns the memory it uses or borrows it from another object.

WRITEABLE(W)

The data area can be written to. Setting this to False locks the data, making it read-only. A view (slice, etc.) inherits WRITEABLE from its base array at creation time, but a view of a writeable array may be subsequently locked while the base array remains writeable. (The opposite is not true, in that a view of a locked array may not be made writeable. However, currently, locking a base object does not lock any views that already reference it, so under that circumstance it is possible to alter the contents of a locked array via a previously created writeable view onto it.) Attempting to change a non-writeable array raises a RuntimeError exception.

ALIGNED(A)

The data and all elements are aligned appropriately for the hardware.

WRITEBACKIFCOPY(X)

This array is a copy of some other array. The C-API function PyArray_ResolveWritebackIfCopy must be called before deallocating to the base array will be updated with the contents of this array.

FNC

F_CONTIGUOUS and not C_CONTIGUOUS.

FORC

F_CONTIGUOUS or C_CONTIGUOUS (one-segment test).

BEHAVED(B)

ALIGNED and WRITEABLE.

CARRAY(CA)

BEHAVED and C_CONTIGUOUS.

FARRAY(FA)

BEHAVED and F_CONTIGUOUS and not C_CONTIGUOUS.

Notes

The flags object can be accessed dictionary-like (as in a.flags['WRITEABLE']), or by using lowercased attribute names (as in a.flags.writeable). Short flag names are only supported in dictionary access.

Only the WRITEBACKIFCOPY, WRITEABLE, and ALIGNED flags can be changed by the user, via direct assignment to the attribute or dictionary entry, or by calling ndarray.setflags.

The array flags cannot be set arbitrarily:

  • WRITEBACKIFCOPY can only be set False.

  • ALIGNED can only be set True if the data is truly aligned.

  • WRITEABLE can only be set True if the array owns its own memory or the ultimate owner of the memory exposes a writeable buffer interface or is a string.

Arrays can be both C-style and Fortran-style contiguous simultaneously. This is clear for 1-dimensional arrays, but can also be true for higher dimensional arrays.

Even for contiguous arrays a stride for a given dimension arr.strides[dim] may be arbitrary if arr.shape[dim] == 1 or the array has no elements. It does not generally hold that self.strides[-1] == self.itemsize for C-style contiguous arrays or self.strides[0] == self.itemsize for Fortran-style contiguous arrays is true.

flat

A 1-D iterator over the array.

This is a numpy.flatiter instance, which acts similarly to, but is not a subclass of, Python’s built-in iterator object.

See also

flatten

Return a copy of the array collapsed into one dimension.

flatiter

Examples

>>> import numpy as np
>>> x = np.arange(1, 7).reshape(2, 3)
>>> x
array([[1, 2, 3],
       [4, 5, 6]])
>>> x.flat[3]
4
>>> x.T
array([[1, 4],
       [2, 5],
       [3, 6]])
>>> x.T.flat[3]
5
>>> type(x.flat)
<class 'numpy.flatiter'>

An assignment example:

>>> x.flat = 3; x
array([[3, 3, 3],
       [3, 3, 3]])
>>> x.flat[[1,4]] = 1; x
array([[3, 1, 3],
       [3, 1, 3]])
imag

The imaginary part of the array.

Examples

>>> import numpy as np
>>> x = np.sqrt([1+0j, 0+1j])
>>> x.imag
array([ 0.        ,  0.70710678])
>>> x.imag.dtype
dtype('float64')
itemsize

Length of one array element in bytes.

Examples

>>> import numpy as np
>>> x = np.array([1,2,3], dtype=np.float64)
>>> x.itemsize
8
>>> x = np.array([1,2,3], dtype=np.complex128)
>>> x.itemsize
16
mT

View of the matrix transposed array.

The matrix transpose is the transpose of the last two dimensions, even if the array is of higher dimension.

New in version 2.0.

Raises:

ValueError – If the array is of dimension less than 2.

Examples

>>> import numpy as np
>>> a = np.array([[1, 2], [3, 4]])
>>> a
array([[1, 2],
       [3, 4]])
>>> a.mT
array([[1, 3],
       [2, 4]])
>>> a = np.arange(8).reshape((2, 2, 2))
>>> a
array([[[0, 1],
        [2, 3]],

       [[4, 5],
        [6, 7]]])
>>> a.mT
array([[[0, 2],
        [1, 3]],

       [[4, 6],
        [5, 7]]])
nbytes

Total bytes consumed by the elements of the array.

Notes

Does not include memory consumed by non-element attributes of the array object.

See also

sys.getsizeof

Memory consumed by the object itself without parents in case view. This does include memory consumed by non-element attributes.

Examples

>>> import numpy as np
>>> x = np.zeros((3,5,2), dtype=np.complex128)
>>> x.nbytes
480
>>> np.prod(x.shape) * x.itemsize
480
ndim

Number of array dimensions.

Examples

>>> import numpy as np
>>> x = np.array([1, 2, 3])
>>> x.ndim
1
>>> y = np.zeros((2, 3, 4))
>>> y.ndim
3
real

The real part of the array.

Examples

>>> import numpy as np
>>> x = np.sqrt([1+0j, 0+1j])
>>> x.real
array([ 1.        ,  0.70710678])
>>> x.real.dtype
dtype('float64')

See also

numpy.real

equivalent function

shape

Tuple of array dimensions.

The shape property is usually used to get the current shape of an array, but may also be used to reshape the array in-place by assigning a tuple of array dimensions to it. As with numpy.reshape, one of the new shape dimensions can be -1, in which case its value is inferred from the size of the array and the remaining dimensions. Reshaping an array in-place will fail if a copy is required.

Warning

Setting arr.shape is discouraged and may be deprecated in the future. Using ndarray.reshape is the preferred approach.

Examples

>>> import numpy as np
>>> x = np.array([1, 2, 3, 4])
>>> x.shape
(4,)
>>> y = np.zeros((2, 3, 4))
>>> y.shape
(2, 3, 4)
>>> y.shape = (3, 8)
>>> y
array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.]])
>>> y.shape = (3, 6)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: cannot reshape array of size 24 into shape (3,6)
>>> np.zeros((4,2))[::2].shape = (-1,)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: Incompatible shape for in-place modification. Use
`.reshape()` to make a copy with the desired shape.

See also

numpy.shape

Equivalent getter function.

numpy.reshape

Function similar to setting shape.

ndarray.reshape

Method similar to setting shape.

size

Number of elements in the array.

Equal to np.prod(a.shape), i.e., the product of the array’s dimensions.

Notes

a.size returns a standard arbitrary precision Python integer. This may not be the case with other methods of obtaining the same value (like the suggested np.prod(a.shape), which returns an instance of np.int_), and may be relevant if the value is used further in calculations that may overflow a fixed size integer type.

Examples

>>> import numpy as np
>>> x = np.zeros((3, 5, 2), dtype=np.complex128)
>>> x.size
30
>>> np.prod(x.shape)
30
strides

Tuple of bytes to step in each dimension when traversing an array.

The byte offset of element (i[0], i[1], ..., i[n]) in an array a is:

offset = sum(np.array(i) * a.strides)

A more detailed explanation of strides can be found in The N-dimensional array (ndarray).

Warning

Setting arr.strides is discouraged and may be deprecated in the future. numpy.lib.stride_tricks.as_strided should be preferred to create a new view of the same data in a safer way.

Notes

Imagine an array of 32-bit integers (each 4 bytes):

x = np.array([[0, 1, 2, 3, 4],
              [5, 6, 7, 8, 9]], dtype=np.int32)

This array is stored in memory as 40 bytes, one after the other (known as a contiguous block of memory). The strides of an array tell us how many bytes we have to skip in memory to move to the next position along a certain axis. For example, we have to skip 4 bytes (1 value) to move to the next column, but 20 bytes (5 values) to get to the same position in the next row. As such, the strides for the array x will be (20, 4).

See also

numpy.lib.stride_tricks.as_strided

Examples

>>> import numpy as np
>>> y = np.reshape(np.arange(2 * 3 * 4, dtype=np.int32), (2, 3, 4))
>>> y
array([[[ 0,  1,  2,  3],
        [ 4,  5,  6,  7],
        [ 8,  9, 10, 11]],
       [[12, 13, 14, 15],
        [16, 17, 18, 19],
        [20, 21, 22, 23]]], dtype=np.int32)
>>> y.strides
(48, 16, 4)
>>> y[1, 1, 1]
np.int32(17)
>>> offset = sum(y.strides * np.array((1, 1, 1)))
>>> offset // y.itemsize
np.int64(17)
>>> x = np.reshape(np.arange(5*6*7*8, dtype=np.int32), (5, 6, 7, 8))
>>> x = x.transpose(2, 3, 1, 0)
>>> x.strides
(32, 4, 224, 1344)
>>> i = np.array([3, 5, 2, 2], dtype=np.int32)
>>> offset = sum(i * x.strides)
>>> x[3, 5, 2, 2]
np.int32(813)
>>> offset // x.itemsize
np.int64(813)
Previous Next

© Copyright 2026, Dean Lab, UT Southwestern Medical Center.