Skip to main content
Version: Next

Command-line Tools

Overview​

In addition to InLong Dashboard, you can view and create data resources through command-line tools.

inlongctl can be run from InLong's bin directory.

usage:

$ bin/inlongctl [options] [command] [command options]

command:

  • list
  • describe
  • create
  • update
  • delete
  • log

You can also use --help or -h to get help for the above commands, for example:

$ bin/inlongctl list -h

Configuration​

Go to the inlong-manager directory , modify the following configurations of the conf/application.properties file.

server.host=127.0.0.1
server.port=8080
default.admin.user=admin
default.admin.password=inlong

List​

list is used to display the core information of resources and display them in a table.

command:

  • group
  • stream
  • source
  • sink
  • cluster
  • cluster-tag
  • cluster-node
  • user

group​

$ bin/inlongctl list group

options:

parameterdescriptiondefault
-g, --groupinlong group id, support fuzzy query
-s, --statusinlong group status , Optional values: CREATE, REJECTED, INITIALIZING, OPERATING,
STARTED, FAILED, STOPPED, FINISHED, DELETED
-n, --nummaximum number of displays10
group status
group statusdescription
CREATEto be submit or to be approval
REJECTEDapproval rejected
INITIALIZINGconfiguring
OPERATINGdeleting, stopping or restarting
STARTEDsuccessful configuration and restart
FAILEDfailed to configure
STOPPEDsuspended
FINISHEDfinish
DELETEDdeleted

stream​

$ bin/inlongctl list stream

options:

parameterdescriptiondefault
-g, --group *inlong group id

* means required parameter.

source​

$ bin/inlongctl list source

options:

parameterdescriptiondefault
-g, --group *inlong group id
-s, --stream *inlong stream id
-t, --typestream source type, Optional values: AUTO_PUSH, TUBEMQ, PULSAR, KAFKA, FILE, MYSQL_SQL,
MYSQL_BINLOG, POSTGRESQL, ORACLE, SQLSERVER, MONGODB, REDIS
stream source type
stream source typedescription
AUTO_PUSHAuto Push
TUBEMQTubeMQ
PULSARPulsar
KAFKAKafka
FILEFile
MYSQL_SQLSQL
MYSQL_BINLOGBinlog
POSTGRESQLPostgreSQL
ORACLEOracle
SQLSERVERSQL server
MONGODBMongoDB
REDISRedis

sink​

$ bin/inlongctl list sink

options:

parameterdescriptiondefault
-g, --group *inlong group id
-s, --stream *inlong stream id

cluster-tag​

$ bin/inlongctl list cluster-tag

options:

parameterdescriptiondefault
--tagcluster tag, support fuzzy query

cluster​

$ bin/inlongctl list cluster

options:

parameterdescriptiondefault
--tagcluster tag
--typecluster type, Optional values: AGENT, TUBEMQ, PULSAR, DATAPROXY, KAFKA
cluster type
cluster typedescription
AGENTAgent
TUBEMQTubeMQ
PULSARPulsar
DATAPROXYDataProxy
KAFKAKafka

cluster-node​

$ bin/inlongctl list cluster-node

options:

parameterdescriptiondefault
--tag *cluster tag
--typecluster type, Optional values: AGENT, TUBEMQ, PULSAR, DATAPROXY, KAFKA

user​

$ bin/inlongctl list user

options:

parameterdescriptiondefault
-u, --usernameusername, support fuzzy query
--typeuser type, Optional values: ADMIN, OPERATOR
user type
user typedescription
ADMINadmin
OPERATORother user

Describe​

describe is used to display detailed information and output in json format.

command:

  • group
  • stream
  • source
  • sink
  • cluster
  • cluster-tag
  • cluster-node
  • user

group​

$ bin/inlongctl describe group

options:

parameterdescriptiondefault
-g, --groupinlong group id, support fuzzy query
-s, --statusinlong group status , Optional values: CREATE, REJECTED, INITIALIZING, OPERATING,
STARTED, FAILED, STOPPED, FINISHED, DELETED
-n, --nummaximum number of displays10
group status
group statusdescription
CREATEto be submit or to be approval
REJECTEDapproval rejected
INITIALIZINGconfiguring
OPERATINGdeleting, stopping or restarting
STARTEDsuccessful configuration and restart
FAILEDfailed to configure
STOPPEDsuspended
FINISHEDfinish
DELETEDdeleted

stream​

$ bin/inlongctl describe stream

options:

parameterdescriptiondefault
-g, --group *inlong group id

source​

$ bin/inlongctl describe source

options:

parameterdescriptiondefault
-g, --group *inlong group id
-s, --stream *inlong stream id
-t, --typestream source type, Optional values: AUTO_PUSH, TUBEMQ, PULSAR, KAFKA, FILE, MYSQL_SQL,
MYSQL_BINLOG, POSTGRESQL, ORACLE, SQLSERVER, MONGODB, REDIS
stream source type
stream source typedescription
AUTO_PUSHAuto Push
TUBEMQTubeMQ
PULSARPulsar
KAFKAKafka
FILEFile
MYSQL_SQLSQL
MYSQL_BINLOGBinlog
POSTGRESQLPostgreSQL
ORACLEOracle
SQLSERVERSQL server
MONGODBMongoDB
REDISRedis

sink​

$ bin/inlongctl describe sink

options:

parameterdescriptiondefault
-g, --group *inlong group id
-s, --stream *inlong stream id

cluster-tag​

$ bin/inlongctl describe cluster-tag

options:

parameterdescriptiondefault
-id, --id *cluster tag id

cluster​

$ bin/inlongctl describe cluster

options:

parameterdescriptiondefault
-id, --id *cluster id

cluster-node​

$ bin/inlongctl describe cluster-node

options:

parameterdescriptiondefault
-id, --id *cluster node id

user​

$ bin/inlongctl describe user

options:

parameterdescriptiondefault
-id, --id *user id

Create​

create is used to create resources, currently created by using a json file.

command:

  • group
  • cluster
  • cluster-tag
  • cluster-node
  • user

group​

$ bin/inlongctl create group

options:

parameterdescriptiondefault
-f, --filejson file name

json:

{
    "groupInfo": {
        "inlongGroupId": "test_group_ctl",
        "inlongClusterTag": "default_cluster",
        "mqType": "PULSAR"
    },
    "streamInfo": {
        "inlongStreamId": "test_stream_ctl",
        "fieldList": [
            {
                "fieldName": "name",
                "fieldType": "string"
            }
        ],
        "sourceList": [
            {
                "sourceType": "FILE",
                "sourceName": "test_source_ctl",
                "agentIp": "127.0.0.1",
                "pattern": "/data/test.txt"
            }
        ],
        "sinkList": [
            {
                "sinkType": "CLICKHOUSE",
                "sinkName": "test_sink_ctl",
                "dataNodeName": "test_clickhouse",
                "dbName": "db_test",
                "tableName": "table_test",
                "flushInterval": 1,
                "flushRecord": 1000,
                "retryTimes": 3,
                "engine": "Log",
                "isDistributed": 1,
                "sinkFieldList": [
                    {
                        "sourceFieldName": "name",
                        "sourceFieldType": "string",
                        "fieldName": "name",
                        "fieldType": "string"
                    }
                ]
            }
        ]
    }
}
  • This is an example of file → pulsar → clickhouse, if you want to use other data flow, just replace the corresponding part.

Source:

{ 
    "sourceType": "FILE",
    "sourceName": "test_source_ctl",
    "agentIp": "127.0.0.1",
    "pattern": "/data/test.txt"
}

Sink:

{
    "sinkType": "CLICKHOUSE",
    "sinkName": "test_sink_ctl",
    "dataNodeName": "test_clickhouse",
    "dbName": "db_test",
    "tableName": "table_test",
    "flushInterval": 1,
    "flushRecord": 1000,
    "retryTimes": 3,
    "engine": "Log",
    "isDistributed": 1,
    "enableCreateResource": 1,
    "sinkFieldList": []
}

cluster​

$ bin/inlongctl create cluster

options:

parameterdescriptiondefault
-f, --filejson file name

json:

{
  "name": "test_cluster",
  "url": "127.0.0.1:8080",
  "clusterTags": "test_cluster_tag",
  "extTag": null,
  "description": null,
  "inCharges": "admin",
  "type": "PULSAR",
  "adminUrl": "http://127.0.0.1:8080",
  "tenant": "public"
}

cluster-tag​

$ bin/inlongctl create cluster-tag

options:

parameterdescriptiondefault
-f, --filejson file name

json:

{
  "clusterTag": "test_cluster_tag",
  "inCharges": "ctl",
  "extParams": null,
  "description": null
}

cluster-node​

$ bin/inlongctl create cluster-node

options:

parameterdescriptiondefault
-f, --filejson file name

json:

{
  "parentId": 1,
  "type": "AGENT",
  "ip": "127.0.0.1",
  "port": 8008,
  "extParams": null,
  "description": "null"
}

parentId is the corresponding cluster id of this node, and the cluster id can be obtained through list cluster or describe cluster

user​

$ bin/inlongctl create user

options:

parameterdescriptiondefault
-u, --usernameusername
-p, --passwordpassword
-t, --typeaccount type
-d, --dayvalid days

Update​

update is used to update resources, currently update by using a json file.

command:

  • cluster
  • cluster-tag
  • cluster-node
  • user

The json file required by update can be modified on the json obtained by describe.

cluster​

$ bin/inlongctl update cluster

options:

parameterdescriptiondefault
-f, --filejson file name

cluster-tag​

$ bin/inlongctl update cluster-tag

options:

parameterdescriptiondefault
-f, --filejson file name

cluster-node​

$ bin/inlongctl update cluster-node

options:

parameterdescriptiondefault
-f, --filejson file name

user​

$ bin/inlongctl update user

options:

parameterdescriptiondefault
-u, --usernameusername
-p, --passwordnew password
-d, --daynew valid days

Suspend​

suspend is used to suspend inlong group task.

command:

  • group

group​

$ bin/inlongctl suspend group

options:

parameterdescriptiondefault
-g, --groupinlong group id

Restart​

restart is used to restart inlong group task.

command:

  • group

group​

$ bin/inlongctl restart group

options:

parameterdescriptiondefault
-g, --groupinlong group id

Delete​

delete is used to delete resources.

command:

  • group
  • stream
  • source
  • sink
  • cluster
  • cluster-tag
  • cluster-node
  • user

group​

$ bin/inlongctl delete group

options:

parameterdescriptiondefault
-g, --groupinlong group id, support fuzzy query
-s, --statusinlong group status , Optional values: CREATE, REJECTED, INITIALIZING, OPERATING,
STARTED, FAILED, STOPPED, FINISHED, DELETED
-n, --nummaximum number of displays10
group status
group statusdescription
CREATEto be submit or to be approval
REJECTEDapproval rejected
INITIALIZINGconfiguring
OPERATINGdeleting, stopping or restarting
STARTEDsuccessful configuration and restart
FAILEDfailed to configure
STOPPEDsuspended
FINISHEDfinish
DELETEDdeleted

stream​

$ bin/inlongctl delete stream

options:

parameterdescriptiondefault
-g, --group *inlong group id

source​

$ bin/inlongctl delete source

options:

parameterdescriptiondefault
-g, --group *inlong group id
-s, --stream *inlong stream id
-t, --typestream source type, Optional values: AUTO_PUSH, TUBEMQ, PULSAR, KAFKA, FILE, MYSQL_SQL,
MYSQL_BINLOG, POSTGRESQL, ORACLE, SQLSERVER, MONGODB, REDIS
stream source type
stream source typedescription
AUTO_PUSHAuto Push
TUBEMQTubeMQ
PULSARPulsar
KAFKAKafka
FILEFile
MYSQL_SQLSQL
MYSQL_BINLOGBinlog
POSTGRESQLPostgreSQL
ORACLEOracle
SQLSERVERSQL server
MONGODBMongoDB
REDISRedis

sink​

$ bin/inlongctl delete sink

options:

parameterdescriptiondefault
-g, --group *inlong group id
-s, --stream *inlong stream id

cluster-tag​

$ bin/inlongctl delete cluster-tag

options:

parameterdescriptiondefault
-id, --id *cluster tag id

cluster​

$ bin/inlongctl delete cluster

options:

parameterdescriptiondefault
-id, --id *cluster id

cluster-node​

$ bin/inlongctl delete cluster-node

options:

parameterdescriptiondefault
-id, --id *cluster node id

user​

$ bin/inlongctl delete user

options:

parameterdescriptiondefault
-id, --id *user id

Log​

After creating the task process, you can use the log command to view the execution log of each stage of the task.

command:

  • group

group​

$ bin/inlongctl log group

options:

parameterdescriptiondefault
-g, --groupinlong group id, not support fuzzy query