Hadoop Data Types with Examples
Hadoop supports various data types for defining column or field types in Hive tables. In this blog, we will help you gain a strong knowledge of Hadoop Hive data types with detailed examples.
Majorly, Hadoop Data Types are categorized into five types as:
Enroll in our Hadoop Certification course today and develop a strong foundation in Big Data.
Hadoop Data Types
#1. Primitive Data Types
These data types and their sizes are similar to SQL/Java primitive data types and sizes.
Primary Data Types are further divided into four types as:
- Numeric Data Type
- Date/Time Data Type
- String Data Type
- Miscellaneous Data Type.
i) Numeric Data Type
It supports both integral and floating data types. These are equivalent to Java’s primitive types.
Below chart explains all the numeric types with their sizes and their ranges.
Integral Data Type:
| Type | Size | Range |
| TINYINT | 1 BYTE | -128 to 127 |
| -128 to 127 | 2 BYTE | -32, 768 to 32, 767 |
| INT | 4 BYTE | –2,147,483,648to 2,147,483,647 |
| BIGINT | 8 BYTE | –9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 |
Floating Data Type:
| Type | Size |
| FLOAT | 4 BYTE |
| DOUBLE | 8 BYTE |
| DECIMAL | 17 BYTE |
ii) Date/Time Data Type
It supports timestamp, date, and interval data types. Hive provides Timestamp and Date data types to UNIX timestamp format.
- TIMESTAMP- It uses nanosecond precision and is denoted by yyyy-mm-dd hh:mm: ss format.
- DATE- These are represented as YYYY-MM-DD
- INTERVAL
The below table shows how to cast string and timestamp values to Date format:
| Cast Type | Result |
| cast(date as date) | Same date value |
| cast(date as string) | Date formatted to ‘YYYY-MM-DD’. |
| cast(string as date) | Midnight of the year/month/day of the date value is returned as a timestamp. |
| cast(date as timestamp) | If the string is in the form of ‘YYYY-MM-DD’, then a date value related to that is returned. If the string value does not match this format, then NULL is returned. |
| cast(timestamp as date) | The year/month/day of the timestamp is returned as a date value. |
iii) String Data Type
It supports string, varchar, and char data types.
- STRING (Unbounded variable-length character string) - Either single or double quotes can be used to enclose characters.
- VARCHAR (Variable-length character string) - Maximum length is specified in braces and allowed up to 65355 bytes.
- CHAR (Fixed-length character string) - ‘m’ or “m”
iv) Miscellaneous Data Type
Miscellaneous data types support both boolean and binary data types.
- BOOLEAN - It stores True or false values
- BINARY - It is an array of bytes.
Frequently asked Hadoop Interview Questions
#2. Complex Data Types
In Hive, complex data types are further classified into Array, MAP, STRUCT, and UNION.
i) ARRAY:
an Ordered collection of similar types of fields that are indexable using zero-based integers. (These are just like arrays in java).
Syntax: ARRAY<data_type>
E.g. array (5, 6)
ii) MAP:
Unordered collection of key-value pairs. Keys may be values, primitives or any type.
For a specific map, the keys and values must be the same type.
Syntax: MAP<primitive_type, data_type>
E.g. MAP(‘m’, 8, ‘n’, 9).
iii) STRUCT:
Collection of named fields. This fields may be of various types. (It's similar to STRUCT in C language)
Syntax: STRUCT<col_name : data_type [COMMENT col_comment],…..>
E.g. struct(‘m', 1 1.0),[n] named_struct(‘col1', ‘m', ‘col2', 1, ‘col3', 1.0)
iv) UNION:
Union type can hold any data type that may be one of the specified data types. (It’s similar to Unions in C)
Syntax: UNIONTYPE<data_type, data_type, …>
E.g. create_union(5, ‘b', 60)
#3. Columns
In Hive, columns support integral type, string, timestamp, date, decimal, and union data types.
i) Integral type:
By default, the integral type is considered as int unless the range of the number exceeds.
- INT/INTEGER
- SMALLINT- Its data range is less than the range of INT.
- BIGINT- Its data range exceeds the range of INT.
- TINYINT- Its data range is less than SMALLINT
| Table | Postfix | Example |
| BIGINT | L | 100L |
| SMALLINT | S | 100S |
| TINYINT | Y | 100Y |
ii) Strings:
These are represented with either single(‘) or double(“) quotes and classified as:
- VARCHAR- These are built with a length specifier (1-65535). It represents the maximum number of characters passed in the character string.
- Char- These are same as VARCHAR but of fixed-length. Maximum fixed length is 255.
iii) Timestamp:
It maintains traditional UNIX timestamp with operational nanosecond precision.
The supported Timestamp format is ”YYYY-MM-DD HH:MM: SS.(.f...)” in text files.
Timestamps are further categorized as:
| Integer numeric type | UNIX timestamp in seconds |
| Floating-point numeric type | UNIX timestamp in seconds with decimal precision |
| String | Supports java.sql.Timestamp |
| format "YYYY-MM-DD HH:MM: SS.fffffffff" |
iv) Dates:
DATE value represents a particular year/month/day as YYYY-MM-DD format. For eg: DATE ‘2019-08-02’
It does not contain a time of day time component. It supports a range of 0000-01-01 to 9999-12-31.
v) Decimals:
The DECIMAL type is similar to Java’s Big Decimal format which denotes immutable arbitrary precision.
- Apache Hive 0.11 and 0.12 have the precision of the DECIMAL type fixed and defined to 38 digits.
- Apache Hive 0.13 users can specify scale and precision while designing tables with the DECIMAL datatype by a syntax DECIMAL(precision, scale).
- If precision is not specified its default value is 10, and if the scale is not specified its default value is 0.
not specified its default value is 0.
Eg:
CREATE TABLE foo (
a DECIMAL, -- Defaults to decimal(10,0)
b DECIMAL(10, 6)
)
vi) Union Types:
Union Hive data types are the set of independent data types. We can create an instance of this type by create_union UDF.
#4. Literals
Literals support both floating-point types and decimal types.
- Floating Point Types - These are assumed to be DOUBLE data types in the Hive.
- Decimal Types - These are assumed to be higher value than float point value with a range more than DOUBLE data type. It ranges between -10-308 to 10308.
#5. Null Values
In Hive, missing values are denoted by the specific value called NULL.
Conclusion:
Therefore, on this page, we have stated all the data types clearly. Hope this information will be helpful for you to gain a firm base in Hadoop Hive data types.
Explore Hadoop Sample Resumes! Download & Edit, Get Noticed by Top Employers!Download Now!