Redis

Summary

Redis Load Node supports writing data to redis.

Data Type

See detail: Redis Data Type

Plain

c1	c2	c3	c4	c5	c6	c7
rowKey

Redis strings commands are used for managing string values in Redis

The first element is Redis row key,must be string type, and the remaining fields(c2~c6) will be serialized into one value and put into Redis.

Hash

• A Redis hash is a data type that represents a mapping between a string field and a string value. There are two members in hash DataType:

• Redis Hash
• Redis Hash value

Set

Redis SET are simply lists of strings, sorted by insertion order. You can add elements in Redis SET in the head or the tail of the list.

BitMap

BitMaps are not an actual data type, but a set of bit-oriented operations defined on the String type.Since strings are binary safe blobs and their maximum length is 512 MB, they are suitable to set up to 2^32 different bits.

SchemaMappingMode

Dynamic

The DYNAMIC mode witch mapping a java.util.Map to RedisDataType. There are two members in DYNAMIC mode:

• Redis key
• java.util.Map object, witch will be iterated, the entry key is redis key, and the entry value is redis value.

Static Prefix Match

The are at least two fields, the first member is redis key, and each field from the second field represents a redis value.

key, field, value1, value2, value3, [value]...

Static KV Pair

There are two fields, the first field is key, and the other fields are pairs of field and value.

 key, field1, value1,field2,value2,field3,value3,[field,value]...

SQL Demo

Plain

Plain only support STATIC_PREFIX_MATCH schema mapping mode

CREATE TABLE sink (
    key STRING,
    aaa STRING,
    bbb DOUBLE,    
    ccc BIGINT,    
    PRIMARY KEY (`key`) NOT ENFORCED
) WITH (  
    'connector' = 'redis-inlong',  
    'sink.batch-size' = '1',  
    'format' = 'csv',  
    'data-type' = 'PLAIN',  
    'redis-mode' = 'standalone',  
    'host' = 'localhost',  
    'port' = '56615',  
    'maxIdle' = '8',  
    'minIdle' = '1',  
    'maxTotal' = '2',  
    'timeout' = '2000'
);

Hash with Prefix Match

c1	c2	c3	c4	c5	c6	c7
rowKey	field: String

The first element is Redis row key, must be string type. The second element is Redis field name in Hash DataType. The remaining fields(c2~c7) will be serialized into one value and put into redis

CREATE TABLE sink (
    key STRING, 
    field_name STRING, 
    value_1 DOUBLE,
    value_2 BIGINT, 
    PRIMARY KEY (`key`) NOT ENFORCED
) WITH (
   'connector' = 'redis-inlong',
   'sink.batch-size' = '1',
   'format' = 'csv',
   'data-type' = 'HASH',
   'redis-mode' = 'standalone',
   'host' = 'localhost',
   'port' = '56869',
   'maxIdle' = '8',
   'minIdle' = '1',
   'maxTotal' = '2',
   'timeout' = '2000'
);

Hash with Static KV Pair

c1	c2	c3	c4	c5	c6	c7
rowKey	field1: String	value 1:String	field 2: String	value 2:String	field 3: String	value 3:String

The first element is Redis row key, must be string type. The odd elements(c2 / c4 / c6) are Redis field names in Hash DataType, must be String type. The even elements(c3 / c5 / c7) are Redis field values in Hash DataType, must be String type.

CREATE TABLE sink (
    key STRING,
    field1 STRING,
    value1 STRING,
    field2 STRING,
    value2 STRING,
    PRIMARY KEY (`key`) NOT ENFORCED
) WITH (
  'connector' = 'redis-inlong',
  'sink.batch-size' = '1',
  'format' = 'csv',
  'data-type' = 'HASH',
  'schema-mapping-mode' = 'STATIC_KV_PAIR',
  'redis-mode' = 'standalone',
  'host' = 'localhost',
  'port' = '6379',
  'maxIdle' = '8',
  'minIdle' = '1',
  'maxTotal' = '2',
  'timeout' = '2000'
);

Hash with dynamic

c1	c2
rowKey	fieldValueMap

The first element is Redis row key, must be string type. The second element is must be Map<String,String> type: key is fieldName, value is fieldValue.

CREATE TABLE sink (
    key STRING,
    fieldValueMap MAP<STRING,STRING>,
    PRIMARY KEY (`key`) NOT ENFORCED
) WITH (
  'connector' = 'redis-inlong',
  'sink.batch-size' = '1',
  'format' = 'csv',
  'data-type' = 'HASH',
  'schema-mapping-mode' = 'DYNAMIC',
  'redis-mode' = 'standalone',
  'host' = 'localhost',
  'port' = '6379',
  'maxIdle' = '8',
  'minIdle' = '1',
  'maxTotal' = '2',
  'timeout' = '2000'
)"

BitMap with static KV pair

c1	c2	c3	c4	c5	c6	c7
rowKey	field1: Long	value 1:Boolean	field 2: Long	value 2:Boolean	field 3: Long	value 3:Boolean

The first element is Redis row key, must be string type. The odd elements(c2 /c4 /c6 ) are Redis offsets in Bitmap DataType, must be Long type. The even elements(c3 / c5 / c7) are Redis values in Bitmap DataType, must be Boolean type.

CREATE TABLE sink (
    key STRING,
    offset_1 BIGINT,
    value_1 BOOLEAN,
    offset_2 BIGINT,
    value_2 BOOLEAN,
    PRIMARY KEY (`key`) NOT ENFORCED
) WITH (
  'connector' = 'redis-inlong',
  'sink.batch-size' = '1',
  'format' = 'csv',
  'data-type' = 'BITMAP',
  'schema-mapping-mode' = 'STATIC_KV_PAIR',
  'redis-mode' = 'standalone',
  'host' = 'localhost',
  'port' = '6379',
  'maxIdle' = '8',
  'minIdle' = '1',
  'maxTotal' = '2',
  'timeout' = '2000'
)