Avro

This article introduces you to usage examples, configuration options, and type mappings of the Avro format.

Background information

The Avro format allows reading and writing of Avro data based on Avro structures. Currently, the Avro structure is derived based on the table structure.

Example of use

    CREATE TABLE user_behavior (
        user_id BIGINT,
        item_id BIGINT,
        category_id BIGINT,
        behavior STRING,
        ts TIMESTAMP(3)
    ) WITH (
        'connector' = 'kafka',
        'topic' = 'user_behavior',
        'properties.bootstrap.servers' = 'localhost :9092',
        'properties.group.id' = 'testGroup',
        'format' = 'avro'
    )

Configuration options

Parameter	Description	Required	Default	Type
format	The format to use for the declaration. When the Avro format is used, the value of the parameter is avro.	yes	none	String
avro.codec	Specifies the codec for Avro compression, only applicable when the connector is Filesystem. The parameter values are as follows: snappy (default), null, deflate, bzip2, xz	yes	none	String

Type mapping

Flink SQL type	Avro type
CHAR/VARCHAR/STRING	string
BOOLEAN	boolean
BINARY / VARBINARY	bytes
DECIMAL	fixed (a decimal number with precision)
TINYINT	int
SMALLINT	int
INT	int
BIGINT	long
FLOAT	float
DOUBLE	double
DATE	int: date
time	int (the time in milliseconds)
TIMESTAMP	long (the time in milliseconds)
ARRAY	array
MAP (Element must be of type STRING, CHAR or VARCHAR.)	map
MULTISET (Element must be of type STRING, CHAR or VARCHAR.)	map
ROW	record

In addition to the above types, Flink supports reading and writing nullable types. Flink maps nullable types to Avrounion(something, null), where something is an Avro type converted from a Flink type.

note

For information about Avro types, see Avro specification.

note

This page is derived from the official Apache Flink® documentation.

Refer to the Credits page for more information.