Array

A Caterva array is a multidimensional dataset (compressed or not) easy to handle using the Caterva functions. Furthermore, Caterva only stores item size instead of the data type and every item of a Caterva array has the same size. On the other hand, Caterva functions let users to perform different operations with these arrays like copying, slicing, setting them or converting them into buffers or files and vice versa.

struct caterva_array_t

A multidimensional array of data that can be compressed.

Parameters

General parameters

struct caterva_params_t

General parameters needed for the creation of a caterva array.

Public Members

uint8_t itemsize

The size of each item of the array.

int64_t shape[CATERVA_MAX_DIM]

The array shape.

uint8_t ndim

The array dimensions.

Storage parameters

struct caterva_storage_t

Storage parameters needed for the creation of a caterva array.

Public Members

caterva_storage_backend_t backend

The backend storage.

caterva_storage_properties_t properties

The specific properties for the selected backend.

enum caterva_storage_backend_t

The backends available to store the data of the caterva array.

Values:

enumerator CATERVA_STORAGE_BLOSC

Indicates that the data is stored using a Blosc super-chunk.

enumerator CATERVA_STORAGE_PLAINBUFFER

Indicates that the data is stored using a plain buffer.

union caterva_storage_properties_t
#include <caterva.h>

The storage properties for an array.

Public Members

caterva_storage_properties_blosc_t blosc

The storage properties when the array is backed by a Blosc super-chunk.

caterva_storage_properties_plainbuffer_t plainbuffer

The storage properties when the array is backed by a plain buffer.

struct caterva_storage_properties_blosc_t

The storage properties for an array backed by a Blosc super-chunk.

Public Members

int32_t chunkshape[CATERVA_MAX_DIM]

The shape of each chunk of Blosc.

int32_t blockshape[CATERVA_MAX_DIM]

The shape of each block of Blosc.

bool sequencial

Flag to indicate if the super-chunk is stored sequentially or sparsely.

char *urlpath

The super-chunk name.

If urlpath is not NULL, the super-chunk will be stored on disk.

caterva_metalayer_t metalayers[CATERVA_MAX_METALAYERS]

List with the metalayers desired.

int32_t nmetalayers

The number of metalayers.

struct caterva_metalayer_t

The metalayer data needed to store it on an array.

Public Members

char *name

The name of the metalayer.

uint8_t *sdata

The serialized data to store.

int32_t size

The size of the serialized data.

struct caterva_storage_properties_plainbuffer_t

The storage properties that have a caterva array backed by a plain buffer.

Public Members

char *urlpath

The plain buffer name.

If urlpath is not NULL, the plain buffer will be stored on disk. (Not implemented yet).

Creation

Fast constructors

int caterva_empty(caterva_ctx_t *ctx, caterva_params_t *params, caterva_storage_t *storage, caterva_array_t **array)

Create an empty array.

Parameters
  • ctx – The caterva context to be used.

  • params – The general params of the array desired.

  • storage – The storage params of the array desired.

  • array – The memory pointer where the array will be created.

Returns

An error code.

int caterva_zeros(caterva_ctx_t *ctx, caterva_params_t *params, caterva_storage_t *storage, caterva_array_t **array)

Create an array, with zero being used as the default value for uninitialized portions of the array.

Parameters
  • ctx – The caterva context to be used.

  • params – The general params of the array.

  • storage – The storage params of the array.

  • array – The memory pointer where the array will be created.

Returns

An error code.

int caterva_full(caterva_ctx_t *ctx, caterva_params_t *params, caterva_storage_t *storage, void *fill_value, caterva_array_t **array)

Create an array, with fill_value being used as the default value for uninitialized portions of the array.

Parameters
  • ctx – The caterva context to be used.

  • params – The general params of the array.

  • storage – The storage params of the array.

  • fill_value – Default value for uninitialized portions of the array.

  • array – The memory pointer where the array will be created.

Returns

An error code.

From/To buffer

int caterva_from_buffer(caterva_ctx_t *ctx, void *buffer, int64_t buffersize, caterva_params_t *params, caterva_storage_t *storage, caterva_array_t **array)

Create a caterva array from the data stored in a buffer.

Parameters
  • ctx – The caterva context to be used.

  • buffer – The buffer where source data is stored.

  • buffersize – The size (in bytes) of the buffer.

  • params – The general params of the array desired.

  • storage – The storage params of the array desired.

  • array – The memory pointer where the array will be created.

Returns

An error code.

int caterva_to_buffer(caterva_ctx_t *ctx, caterva_array_t *array, void *buffer, int64_t buffersize)

Extract the data into a C buffer from a caterva array.

Parameters
  • ctx – The caterva context to be used.

  • array – The caterva array.

  • buffer – The buffer where the data will be stored.

  • buffersize – Size (in bytes) of the buffer.

Returns

An error code.

From/To file

int caterva_open(caterva_ctx_t *ctx, const char *urlpath, caterva_array_t **array)

Read a caterva array from disk.

Parameters
  • ctx – The caterva context to be used.

  • urlpath – The urlpath of the caterva array on disk.

  • array – The memory pointer where the array will be created.

Returns

An error code.

int caterva_save(caterva_ctx_t *ctx, caterva_array_t *array, char *urlpath)

Save caterva array into a specific urlpath.

Parameters
  • ctx – The context to be used.

  • array – The array to be saved.

  • urlpath – The urlpath where the array will be stored.

Returns

An error code.

From Blosc object

int caterva_from_schunk(caterva_ctx_t *ctx, blosc2_schunk *schunk, caterva_array_t **array)

Create a caterva array from a super-chunk.

It can only be used if the array is backed by a blosc super-chunk.

Parameters
  • ctx – The caterva context to be used.

  • schunk – The blosc super-chunk where the caterva array is stored.

  • array – The memory pointer where the array will be created.

Returns

An error code.

int caterva_from_serial_schunk(caterva_ctx_t *ctx, uint8_t *serial_schunk, int64_t len, caterva_array_t **array)

Create a caterva array from a serialized super-chunk.

It can only be used if the array is backed by a blosc super-chunk.

Parameters
  • ctx – The caterva context to be used.

  • serial_schunk – The serialized super-chunk where the caterva array is stored.

  • len – The size (in bytes) of the serialized super-chunk.

  • array – The memory pointer where the array will be created.

Returns

An error code.

Copying

int caterva_copy(caterva_ctx_t *ctx, caterva_array_t *src, caterva_storage_t *storage, caterva_array_t **array)

Make a copy of the array data.

The copy is done into a new caterva array.

Parameters
  • ctx – The caterva context to be used.

  • src – The array from which data is copied.

  • storage – The storage params of the array desired.

  • array – The memory pointer where the array will be created.

Returns

An error code

Slicing

int caterva_get_slice_buffer(caterva_ctx_t *ctx, caterva_array_t *array, int64_t *start, int64_t *stop, void *buffer, int64_t *buffershape, int64_t buffersize)

Get a slice from an array and store it into a C buffer.

Parameters
  • ctx – The caterva context to be used.

  • array – The array from which the slice will be extracted.

  • start – The coordinates where the slice will begin.

  • stop – The coordinates where the slice will end.

  • buffershape – The shape of the buffer.

  • buffer – The buffer where the data will be stored.

  • buffersize – The size (in bytes) of the buffer.

Returns

An error code.

int caterva_set_slice_buffer(caterva_ctx_t *ctx, void *buffer, int64_t *buffershape, int64_t buffersize, int64_t *start, int64_t *stop, caterva_array_t *array)

Set a slice into a caterva array from a C buffer.

Parameters
  • ctx – The caterva context to be used.

  • buffer – The buffer where the slice data is.

  • buffersize – The size (in bytes) of the buffer.

  • start – The coordinates where the slice will begin.

  • stop – The coordinates where the slice will end.

  • buffershape – The shape of the buffer.

  • array – The caterva array where the slice will be set

Returns

An error code.

int caterva_get_slice(caterva_ctx_t *ctx, caterva_array_t *src, int64_t *start, int64_t *stop, caterva_storage_t *storage, caterva_array_t **array)

Get a slice from an array and store it into a new array.

Parameters
  • ctx – The caterva context to be used.

  • src – The array from which the slice will be extracted

  • start – The coordinates where the slice will begin.

  • stop – The coordinates where the slice will end.

  • storage – The storage params of the array desired.

  • array – The memory pointer where the array will be created.

Returns

An error code.

int caterva_squeeze(caterva_ctx_t *ctx, caterva_array_t *array)

Squeeze a caterva array.

This function remove single-dimensional entries from the shape of a caterva array.

Parameters
  • ctx – The caterva context to be used.

  • array – The caterva array.

Returns

An error code

Destruction

int caterva_free(caterva_ctx_t *ctx, caterva_array_t **array)

Free an array.

Parameters
  • ctx – The caterva context to be used.

  • array – The memory pointer where the array is placed.

Returns

An error code.

int caterva_remove(caterva_ctx_t *ctx, char *urlpath)

Remove a Caterva file from the file system.

Both backends are supported.

Parameters
  • ctx – The caterva context to be used.

  • urlpath – The urlpath of the array to be removed.

Returns

An error code