TubeMQ

Overview

Apache InLong TubeMQ is a distributed, open source pub-sub messaging and steaming platform for real-time workloads, trillions of massive data precipitation.

Version

Load Node	Version
TubeMQ	TubeMQ: >=0.1.0

Dependencies

In order to set up the TubeMQ Load Node, the following provides dependency information for both projects using a build automation tool (such as Maven or SBT) and SQL Client with Sort Connectors JAR bundles.

Maven dependency

<dependency>
    <groupId>org.apache.inlong</groupId>
    <artifactId>sort-connector-tubemq</artifactId>
    <version>2.1.0-SNAPSHOT</version>
</dependency>

How to create a TubeMQ Load Node

Usage for SQL API

The example below shows how to create a TubeMQ Load Node with Flink SQL Cli :

-- Create a TubeMQ table 'tube_load_node' in Flink SQL Cli
Flink SQL>
CREATE TABLE tube_load_node
(
    id     INT,
    name   STRING,
    age    INT,
    salary FLOAT
) WITH (
      'connector' = 'tubemq',
      'topic' = 'topicName',
      'master.rpc' = 'rpcUrl', -- 127.0.0.1:8715
      'format' = 'json',
      'group.name' = 'groupName');

-- Read data from tube_load_node
Flink SQL>
SELECT *
FROM tube_load_node;

Usage for InLong Dashboard

TODO

Usage for InLong Manager Client

TODO

TubeMQ Load Node Options

Parameter	Required	Default value	Type	Description
connector	required	tubemq	String	Set the connector type. Available options are tubemq.
topic	required	(none)	String	Set the input or output topic
masterRpc	required	(none)	String	Set the TubeMQ master service address.
format	required	(none)	String	TubeMQ message value serialization format, support JSON, Avro, etc. For more information, see the Flink format.
groupId	required	(none)	String	Consumer group in TubeMQ