Expand description

Sealed traits and implementations to handle all physical types used in this crate.

Most physical types used in this crate are native Rust types, such as i32. The trait NativeType describes the interfaces required by this crate to be conformant with Arrow.

Every implementation of NativeType has an associated variant in PrimitiveType, available via NativeType::PRIMITIVE. Combined, these allow structs generic over NativeType to be trait objects downcastable to concrete implementations based on the matched NativeType::PRIMITIVE variant.

Another important trait in this module is Offset, the subset of NativeType that can be used in Arrow offsets (i32 and i64).

Another important trait in this module is BitChunk, describing types that can be used to represent chunks of bits (e.g. 8 bits via u8, 16 via u16), and BitChunkIter, that can be used to iterate over bitmaps in BitChunks according to Arrow’s definition of bitmaps.

Finally, this module contains traits used to compile code based on NativeType optimized for SIMD, at simd.

Modules

Contains traits and implementations of multi-data used in SIMD. The actual representation is driven by the feature flag "simd", which, if set, uses std::simd.

Structs

An [Iterator<Item=bool>] over a BitChunk. This iterator is often compiled to SIMD. The LSB corresponds to the first slot, as defined by the arrow specification.

An [Iterator<Item=usize>] over a BitChunk returning the index of each bit set in the chunk See https://lemire.me/blog/2018/03/08/iterating-over-set-bits-quickly-simd-edition/ for details

Range of Index, equivalent to (a..b). Step is unstable in Rust, which does not allow us to implement (a..b) for Index.

The in-memory representation of the DayMillisecond variant of arrow’s “Interval” logical type.

Type representation of the Float16 physical type

The in-memory representation of the MonthDayNano variant of the “Interval” logical type.

Enums

The set of all implementations of the sealed trait NativeType.

Traits

A chunk of bits. This is used to create masks of a given length whose width is 1 bit. In portable_simd notation, this corresponds to m1xY.

Sealed trait describing the subset of NativeType (i32, i64, u32 and u64) that can be used to index a slot of an array.

Sealed trait implemented by all physical types that can be allocated, serialized and deserialized by this crate. All O(N) allocations in this crate are done for this trait alone.

Sealed trait describing the subset (i32 and i64) of Index that can be used as offsets of variable-length Arrow arrays.