Faker

Background Information

Simulation data generation Faker is a built-in Connector in the system, which generates test data according to the Java Faker expression provided by each field in the table. When you need to use some test data to verify business logic during development or testing, it is recommended that you use simulated data to generate Connectors.

The information supported by the simulated data generation connector is as follows:

Usage Restrictions

Only some data types are supported, including:

• CHAR(n)
• VARCHAR(n)
• STRING
• TINYINT
• SMALLINT
• INT
• BIGINT
• FLOAT
• DOUBLE
• DECIMAL
• BOOLEAN
• TIMESTAMP
• ARRAY
• MAP
• MULTISET
• ROW

Grammatical Structures

1    CREATE TABLE faker_source (
2    `name` STRING,
3    `age` INT
4    ) WITH (
5    'connector' = 'faker',
6    'fields.name.expression' = '#{superhero.name}',
7    'fields.age.expression' = ' #{number.numberBetween ''0'',''1000''}'
8    );

WITH Parameter

Universal Type

Source Exclusive Type

Field Expression

Operation method When using simulated data to generate a Connector, each field defined in the DDL needs to provide a specific expression in the WITH statement. The fixed format of the expression is fields..expression = #{className.methodName ‘‘parameter’’, …}. The relevant parameters are described in the table below.

Example:

This article uses the age field expression fields.age.expression = #{number.numberBetween ‘‘0’’,‘‘1000’’} in the Java Faker API documentation and grammar structure as an example to introduce how to Properly generate SQL expressions for fields in DDL.

In the Java Faker API documentation , find the Number class.

Find the numberBetween method in the Number class and look at its method description.

The numberBetween method means to return the value of the specified number range.

fields.age.expression = #{number.numberBetween ‘‘0’’,‘‘1000’’} for the age field according to the parameters 0 and 1000 passed in to the method by the class name Number and the method name numberBetween ’ . Indicates that the value of the generated age field is in the range of 0 to 1000.

Example of Use

Source example

1    CREATE TEMPORARY TABLE heros_source (
2      `name` STRING,
3      `power` STRING,
4      `age` INT
5    ) WITH (
6      'connector' = 'faker',
7      'fields.name.expression' = '#{superhero.name}',
8      'fields.power.expression' = '#{superhero.power}',
9      'fields.power.null-rate' = '0.05',
10      'fields.age.expression' = '#{number.numberBetween ''0'',''1000''}'
11    );
12
13    CREATE TEMPORARY table blackhole_sink(
14      `name` STRING,
15      `power` STRING,
16      `age` INT
17    ) WITH (
18      'connector' = 'blackhole'
19    );
20
21    INSERT INTO blackhole_sink SELECT * FROM heros_source;

Dimension example

1    CREATE TEMPORARY TABLE datagen_source (
2      `character_id` INT,
3      `location` STRING,
4      `proctime` AS PROCTIME()
5    ) WITH (
6      'connector' = 'datagen'
7    );
8
9    CREATE TEMPORARY TABLE faker_dim (
10      `character_id` INT,
11      `name` STRING
12    ) WITH (
13      'connector' = 'faker',
14      'fields.character_id.expression' = '#{number.numberBetween ''0'',''100''}',
15      'fields.name.expression' = '#{harry_potter.characters}'
16    );
17
18    SELECT
19      c.character_id,
20      l.location,
21      c.name
22    FROM datagen_source AS l
23    JOIN faker_dim FOR SYSTEM_TIME AS OF proctime AS c
24    ON l.character_id = c.character_id;
25
26    INSERT INTO blackhole_sink SELECT * FROM heros_source;