Enum arrow2::datatypes::DataType

pub enum DataType {
    Null,
    Boolean,
    Int8,
    Int16,
    Int32,
    Int64,
    UInt8,
    UInt16,
    UInt32,
    UInt64,
    Float16,
    Float32,
    Float64,
    Timestamp(TimeUnit, Option<String>),
    Date32,
    Date64,
    Time32(TimeUnit),
    Time64(TimeUnit),
    Duration(TimeUnit),
    Interval(IntervalUnit),
    Binary,
    FixedSizeBinary(usize),
    LargeBinary,
    Utf8,
    LargeUtf8,
    List(Box<Field>),
    FixedSizeList(Box<Field>, usize),
    LargeList(Box<Field>),
    Struct(Vec<Field>),
    Union(Vec<Field>, Option<Vec<i32>>, UnionMode),
    Map(Box<Field>, bool),
    Dictionary(IntegerType, Box<DataType>, bool),
    Decimal(usize, usize),
    Extension(String, Box<DataType>, Option<String>),
}

The set of supported logical types in this crate.

Each variant uniquely identifies a logical type, which define specific semantics to the data (e.g. how it should be represented). Each variant has a corresponding PhysicalType, obtained via DataType::to_physical_type, which declares the in-memory representation of data. The DataType::Extension is special in that it augments a DataType with metadata to support custom types. Use to_logical_type to desugar such type and return its correspoding logical type.

Variants

Null

Null type

Boolean

true and false.

Int8

An i8

Int16

An i16

Int32

An i32

Int64

An i64

UInt8

An u8

UInt16

An u16

UInt32

An u32

UInt64

An u64

Float16

An 16-bit float

Float32

A f32

Float64

A f64

Timestamp(TimeUnit, Option<String>)

A i64 representing a timestamp measured in TimeUnit with an optional timezone.

Time is measured as a Unix epoch, counting the seconds from 00:00:00.000 on 1 January 1970, excluding leap seconds, as a 64-bit signed integer.

The time zone is a string indicating the name of a time zone, one of:

• As used in the Olson time zone database (the “tz database” or “tzdata”), such as “America/New_York”
• An absolute time zone offset of the form +XX:XX or -XX:XX, such as +07:30 When the timezone is not specified, the timestamp is considered to have no timezone and is represented as is

Date32

An i32 representing the elapsed time since UNIX epoch (1970-01-01) in days.

Date64

An i64 representing the elapsed time since UNIX epoch (1970-01-01) in milliseconds. Values are evenly divisible by 86400000.

Time32(TimeUnit)

A 32-bit time representing the elapsed time since midnight in the unit of TimeUnit. Only TimeUnit::Second and TimeUnit::Millisecond are supported on this variant.

Time64(TimeUnit)

A 64-bit time representing the elapsed time since midnight in the unit of TimeUnit. Only TimeUnit::Microsecond and TimeUnit::Nanosecond are supported on this variant.

Duration(TimeUnit)

Measure of elapsed time. This elapsed time is a physical duration (i.e. 1s as defined in S.I.)

Interval(IntervalUnit)

A “calendar” interval modeling elapsed time that takes into account calendar shifts. For example an interval of 1 day may represent more than 24 hours.

Binary

Opaque binary data of variable length whose offsets are represented as i32.

FixedSizeBinary(usize)

Opaque binary data of fixed size. Enum parameter specifies the number of bytes per value.

LargeBinary

Opaque binary data of variable length whose offsets are represented as i64.

Utf8

A variable-length UTF-8 encoded string whose offsets are represented as i32.

LargeUtf8

A variable-length UTF-8 encoded string whose offsets are represented as i64.

List(Box<Field>)

A list of some logical data type whose offsets are represented as i32.

FixedSizeList(Box<Field>, usize)

A list of some logical data type with a fixed number of elements.

LargeList(Box<Field>)

A list of some logical data type whose offsets are represented as i64.

Struct(Vec<Field>)

A nested DataType with a given number of Fields.

Union(Vec<Field>, Option<Vec<i32>>, UnionMode)

A nested datatype that can represent slots of differing types. Third argument represents mode

Map(Box<Field>, bool)

A nested type that is represented as

List<entries: Struct<key: K, value: V>>

In this layout, the keys and values are each respectively contiguous. We do not constrain the key and value types, so the application is responsible for ensuring that the keys are hashable and unique. Whether the keys are sorted may be set in the metadata for this field.

In a field with Map type, the field has a child Struct field, which then has two children: key type and the second the value type. The names of the child fields may be respectively “entries”, “key”, and “value”, but this is not enforced.

Map

  - child[0] entries: Struct
    - child[0] key: K
    - child[1] value: V

Neither the “entries” field nor the “key” field may be nullable.

The metadata is structured so that Arrow systems without special handling for Map can make Map an alias for List. The “layout” attribute for the Map field must have the same contents as a List.

Dictionary(IntegerType, Box<DataType>, bool)

A dictionary encoded array (key_type, value_type), where each array element is an index of key_type into an associated dictionary of value_type.

Dictionary arrays are used to store columns of value_type that contain many repeated values using less memory, but with a higher CPU overhead for some operations.

This type mostly used to represent low cardinality string arrays or a limited set of primitive types as integers.

The bool value indicates the Dictionary is sorted if set to true.

Decimal(usize, usize)

Decimal value with precision and scale precision is the number of digits in the number and scale is the number of decimal places. The number 999.99 has a precision of 5 and scale of 2.

Extension(String, Box<DataType>, Option<String>)

Extension type.

Implementations

impl DataType

pub fn to_physical_type(&self) -> PhysicalType

the PhysicalType of this DataType.

pub fn to_logical_type(&self) -> &DataType

Returns &self for all but DataType::Extension. For DataType::Extension, (recursively) returns the inner DataType. Never returns the variant DataType::Extension.

Trait Implementations

impl Clone for DataType

fn clone(&self) -> DataType

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DataType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T: NativeType> From<DataType> for MutablePrimitiveArray<T>

fn from(data_type: DataType) -> Self

Converts to this type from the input type.

impl From<IntegerType> for DataType

fn from(item: IntegerType) -> Self

Converts to this type from the input type.

impl From<PrimitiveType> for DataType

fn from(item: PrimitiveType) -> Self

Converts to this type from the input type.

impl Hash for DataType

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H) where
    H: Hasher, 

Feeds a slice of this type into the given Hasher.

impl PartialEq<DataType> for DataType

fn eq(&self, other: &DataType) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for DataType

impl StructuralEq for DataType

impl StructuralPartialEq for DataType