Array#

A Caterva array is a multidimensional dataset (compressed or not) easy to handle using the Caterva functions. Furthermore, Caterva only stores item size instead of the data type and every item of a Caterva array has the same size. On the other hand, Caterva functions let users to perform different operations with these arrays like copying, slicing, setting them or converting them into buffers or files and vice versa.

struct caterva_array_t#

A multidimensional array of data that can be compressed.

Parameters#

General parameters#

struct caterva_params_t#

General parameters needed for the creation of a caterva array.

Public Members

uint8_t itemsize#

The size of each item of the array.

int64_t shape[CATERVA_MAX_DIM]#

The array shape.

int8_t ndim#

The array dimensions.

Storage parameters#

struct caterva_storage_t#

The storage properties for an array backed by a Blosc super-chunk.

Public Members

int32_t chunkshape[CATERVA_MAX_DIM]#

The shape of each chunk of Blosc.

int32_t blockshape[CATERVA_MAX_DIM]#

The shape of each block of Blosc.

bool contiguous#

Flag to indicate if the super-chunk is stored contiguously or sparsely.

char *urlpath#

The super-chunk name.

If urlpath is not NULL, the super-chunk will be stored on disk.

caterva_metalayer_t metalayers[CATERVA_MAX_METALAYERS]#

List with the metalayers desired.

int32_t nmetalayers#

The number of metalayers.

struct caterva_metalayer_t#

The metalayer data needed to store it on an array.

Public Members

char *name#

The name of the metalayer.

uint8_t *sdata#

The serialized data to store.

int32_t size#

The size of the serialized data.

Creation#

Fast constructors#

int caterva_empty(caterva_ctx_t *ctx, caterva_params_t *params, caterva_storage_t *storage, caterva_array_t **array)#

Create an empty array.

Parameters
  • ctx – The caterva context to be used.

  • params – The general params of the array desired.

  • storage – The storage params of the array desired.

  • array – The memory pointer where the array will be created.

Returns

An error code.

int caterva_zeros(caterva_ctx_t *ctx, caterva_params_t *params, caterva_storage_t *storage, caterva_array_t **array)#

Create an array, with zero being used as the default value for uninitialized portions of the array.

Parameters
  • ctx – The caterva context to be used.

  • params – The general params of the array.

  • storage – The storage params of the array.

  • array – The memory pointer where the array will be created.

Returns

An error code.

int caterva_full(caterva_ctx_t *ctx, caterva_params_t *params, caterva_storage_t *storage, void *fill_value, caterva_array_t **array)#

Create an array, with fill_value being used as the default value for uninitialized portions of the array.

Parameters
  • ctx – The caterva context to be used.

  • params – The general params of the array.

  • storage – The storage params of the array.

  • fill_value – Default value for uninitialized portions of the array.

  • array – The memory pointer where the array will be created.

Returns

An error code.

From/To buffer#

int caterva_from_buffer(caterva_ctx_t *ctx, void *buffer, int64_t buffersize, caterva_params_t *params, caterva_storage_t *storage, caterva_array_t **array)#

Create a caterva array from the data stored in a buffer.

Parameters
  • ctx – The caterva context to be used.

  • buffer – The buffer where source data is stored.

  • buffersize – The size (in bytes) of the buffer.

  • params – The general params of the array desired.

  • storage – The storage params of the array desired.

  • array – The memory pointer where the array will be created.

Returns

An error code.

int caterva_to_buffer(caterva_ctx_t *ctx, caterva_array_t *array, void *buffer, int64_t buffersize)#

Extract the data into a C buffer from a caterva array.

Parameters
  • ctx – The caterva context to be used.

  • array – The caterva array.

  • buffer – The buffer where the data will be stored.

  • buffersize – Size (in bytes) of the buffer.

Returns

An error code.

From/To file#

int caterva_open(caterva_ctx_t *ctx, const char *urlpath, caterva_array_t **array)#

Read a caterva array from disk.

Parameters
  • ctx – The caterva context to be used.

  • urlpath – The urlpath of the caterva array on disk.

  • array – The memory pointer where the array will be created.

Returns

An error code.

int caterva_save(caterva_ctx_t *ctx, caterva_array_t *array, char *urlpath)#

Save caterva array into a specific urlpath.

Parameters
  • ctx – The context to be used.

  • array – The array to be saved.

  • urlpath – The urlpath where the array will be stored.

Returns

An error code.

From Blosc object#

int caterva_from_schunk(caterva_ctx_t *ctx, blosc2_schunk *schunk, caterva_array_t **array)#

Create a caterva array from a super-chunk.

It can only be used if the array is backed by a blosc super-chunk.

Parameters
  • ctx – The caterva context to be used.

  • schunk – The blosc super-chunk where the caterva array is stored.

  • array – The memory pointer where the array will be created.

Returns

An error code.

From/To cframe buffer#

int caterva_from_cframe(caterva_ctx_t *ctx, uint8_t *cframe, int64_t cframe_len, bool copy, caterva_array_t **array)#

Create a caterva array from a serialized super-chunk.

Parameters
  • ctx – The caterva context to be used.

  • cframe – The buffer of the in-memory array.

  • cframe_len – The size (in bytes) of the in-memory array.

  • copy – Whether caterva should make a copy of the cframe data or not. The copy will be made to an internal sparse frame.

  • array – The memory pointer where the array will be created.

Returns

An error code.

int caterva_to_cframe(caterva_ctx_t *ctx, caterva_array_t *array, uint8_t **cframe, int64_t *cframe_len, bool *needs_free)#

Create a serialized super-chunk from a caterva array.

Parameters
  • ctx – The caterva context to be used.

  • array – The caterva array to be serialized.

  • cframe – The pointer of the buffer where the in-memory array will be copied.

  • cframe_len – The length of the in-memory array buffer.

  • needs_free – Whether the buffer should be freed or not.

Returns

An error code

Copying#

int caterva_copy(caterva_ctx_t *ctx, caterva_array_t *src, caterva_storage_t *storage, caterva_array_t **array)#

Make a copy of the array data.

The copy is done into a new caterva array.

Parameters
  • ctx – The caterva context to be used.

  • src – The array from which data is copied.

  • storage – The storage params of the array desired.

  • array – The memory pointer where the array will be created.

Returns

An error code

Slicing#

int caterva_get_slice_buffer(caterva_ctx_t *ctx, caterva_array_t *array, int64_t *start, int64_t *stop, void *buffer, int64_t *buffershape, int64_t buffersize)#

Get a slice from an array and store it into a C buffer.

Parameters
  • ctx – The caterva context to be used.

  • array – The array from which the slice will be extracted.

  • start – The coordinates where the slice will begin.

  • stop – The coordinates where the slice will end.

  • buffershape – The shape of the buffer.

  • buffer – The buffer where the data will be stored.

  • buffersize – The size (in bytes) of the buffer.

Returns

An error code.

int caterva_set_slice_buffer(caterva_ctx_t *ctx, void *buffer, int64_t *buffershape, int64_t buffersize, int64_t *start, int64_t *stop, caterva_array_t *array)#

Set a slice into a caterva array from a C buffer.

Parameters
  • ctx – The caterva context to be used.

  • buffer – The buffer where the slice data is.

  • buffersize – The size (in bytes) of the buffer.

  • start – The coordinates where the slice will begin.

  • stop – The coordinates where the slice will end.

  • buffershape – The shape of the buffer.

  • array – The caterva array where the slice will be set

Returns

An error code.

int caterva_get_slice(caterva_ctx_t *ctx, caterva_array_t *src, const int64_t *start, const int64_t *stop, caterva_storage_t *storage, caterva_array_t **array)#

Get a slice from an array and store it into a new array.

Parameters
  • ctx – The caterva context to be used.

  • src – The array from which the slice will be extracted

  • start – The coordinates where the slice will begin.

  • stop – The coordinates where the slice will end.

  • storage – The storage params of the array desired.

  • array – The memory pointer where the array will be created.

Returns

An error code.

int caterva_squeeze(caterva_ctx_t *ctx, caterva_array_t *array)#

Squeeze a caterva array.

This function remove single-dimensional entries from the shape of a caterva array.

Parameters
  • ctx – The caterva context to be used.

  • array – The caterva array.

Returns

An error code

int caterva_squeeze_index(caterva_ctx_t *ctx, caterva_array_t *array, const bool *index)#

Squeeze a caterva array.

This function remove selected single-dimensional entries from the shape of a caterva array.

Parameters
  • ctx – The caterva context to be used.

  • array – The caterva array.

  • index – Indexes of the single-dimensional entries to remove.

Returns

An error code

Destruction#

int caterva_free(caterva_ctx_t *ctx, caterva_array_t **array)#

Free an array.

Parameters
  • ctx – The caterva context to be used.

  • array – The memory pointer where the array is placed.

Returns

An error code.

int caterva_remove(caterva_ctx_t *ctx, char *urlpath)#

Remove a Caterva file from the file system.

Both backends are supported.

Parameters
  • ctx – The caterva context to be used.

  • urlpath – The urlpath of the array to be removed.

Returns

An error code