Command-Line Interface (CLI)

Pinot provides a rich CLI to perform almost every operation on the cluster. You can execute all the commands using the pinot-admin.sh . The script is located in the bin/ directory of the Pinot binary distribution or /opt/pinot/bin in docker container.

The following commands are supported by the admin script.

Add Schema

Upload the schema configuration to controller. If their is already a schema with same name, it will be updated.

pinot-admin.sh AddSchema -schemaFile /path/to/schema.json -controllerHost localhost -controllerPort 9000 -exec

Supported Options

All the options should be prefixed with - (hyphen)

Add Table

Upload the table configuration to controller.

pinot-admin.sh AddTable -tableConfigFile /path/to/table.json -schemaFile /path/to/schema.json -controllerHost localhost -controllerPort 9000 -exec

Supported Options

All the options should be prefixed with - (hyphen)

Add Tenant

Add a new tenant to the server

pinot-admin.sh AddTenant -name myTenant -role SERVER -instanceCount 10 -controllerHost localhost -controllerPort 9000 -exec

Supported Options

All the options should be prefixed with - (hyphen)

Check Offline Segment Intervals

Lists all the segments which have invalid time interval. Only OFFLINE segments are supported.

pinot-admin.sh CheckOfflineSegmentIntervals -zkAddress localhost:2181 -clusterName PinotCluster -tableName myTable

Supported Options

All the options should be prefixed with - (hyphen)

Change Num Replicas

This command changes the replicas of the table. The number of replicas are set from the latest available table config.

pinot-admin.sh ChangeNumReplicas -tableName myTable -clusterName PinotCluster -zkAddress localhost:2181 -exec

Supported Options

All the options should be prefixed with - (hyphen)

Change Table State

Enable, Disable or Drop the table available in database.

pinot-admin.sh ChangeTableState -tableName myTable -state disable -controllerHost localhost -controllerPort 9000 -exec

Supported Options

All the options should be prefixed with - (hyphen)

Create Segment

Create segment files from the input file in local filesystem.

pinot-admin.sh CreateSegment -dataDir /path/to/data/dir -format CSV -outDir /path/to/output/dir -overwrite -tableConfigFile /path/to/table.json -schemaConfigFile /path/to/schema.json

Supported Options

All the options should be prefixed with - (hyphen)

Convert Pinot Segment

Convert the segment file from Pinot specific format to other data formats. Currently only CSV, AVRO and JSON are supported.

pinot-admin.sh ConvertPinotSegment -dataDir /path/to/data/dir -outputDir /path/to/output/dir -outputFormat CSV -overwrite -csvDelimiter , 

Supported Options

All the options should be prefixed with - (hyphen)

Delete Cluster

Delete the cluster namespace from zookeeper.

pinot-admin.sh DeleteCluster -clusterName PinotCluster -zkAddress localhost:2181

All the options should be prefixed with - (hyphen)

Launch Data Ingestion Job

Run job to consume batch or streaming data and push it to Pinot.

pinot-admin.sh LaunchDataIngestionJob -jobSpecFile /path/to/job_spec.json -propertyFile /path/to/job.properties -values dt=2020-08-10 hour=15

Supported Options

All the options should be prefixed with - (hyphen)

Merge/Rollup Segments

Perform operations similar to the Minion Merge Rollup Task, where multiple segments can be merged based on the provided spec.

This command is mostly for debugging purpose. Use Minion Merge Rollup Task for production.

pinot-admin.sh SegmentProcessorFramework -segmentProcessorFrameworkSpec /path/to/framework_spec.json

Fields within the spec file

Move Replica Group

Command to migrate a subset of replica group from current servers to the provided destination servers. This command is intended to be run multiple times to migrate all the replicas of a table to the destination servers (if intended).

pinot-admin.sh MoveReplicaGroup -zkHost localhost:2181 -cluster PinotCluster -srcHosts host1,host2 -destHostsFile /path/to/destination/file -tableName myTable -exec

Supported Options

All the options should be prefixed with - (hyphen)

Operate Cluster Config

Modify cluster level configs for pinot. These are the configs which are applicable to all nodes in the cluster.

pinot-admin.sh OperateClusterConfig -controllerHost localhost -controllerPort 9000 -operation ADD -config pinot.broker.enable.query.limit.override=1

Supported Options

All the options should be prefixed with - (hyphen)

Post Query

Execute a SQL query on the cluster.

pinot-admin.sh PostQuery -brokerHost localhost -brokerPort 8000 -queryType sql -query "SELECT * FROM myTable"

Supported Options

All the options should be prefixed with - (hyphen)

Rebalance Table

Rebalance a table i.e. reassign instances and segments for a table in the cluster.

For segment reassignment, the following modes are offered:

• With-downtime rebalance: the IdealState is replaced with the target segment assignment in one go and there are no guarantees around replica availability. This mode returns immediately without waiting for ExternalView to reach the target segment assignment. Disabled tables will always be rebalanced with downtime.

• No-downtime rebalance: care is taken to ensure that the configured number of replicas of any segment are available (ONLINE or CONSUMING) at all times. This mode returns after ExternalView reaching the target segment assignment. In the edge case scenarios mentioned later, if best-efforts is disabled, rebalancer will fail the rebalance because the no-downtime contract cannot be achieved, and table might end up in a middle stage. User needs to check the rebalance result, solve the issue, and run the rebalance again if necessary. If best-efforts is enabled, rebalancer will log a warning and continue the rebalance, but the no-downtime contract will not be guaranteed. Downtime can occur in the following edge case scenarios -

  • Segment falls into ERROR state in ExternalView -> with best-efforts, count ERROR state as good state.

  • ExternalView has not converged within the maximum wait time -> with best-efforts, continue to the next stage

If the controller that handles the rebalance goes down/restarted, the rebalance isn't automatically resumed by other controllers

pinot-admin.sh RebalanceTable -zkAddress localhost:2181 -clusterName PinotCluster -tableName myTable -reassignInstances -includeConsuming -downtime  

Supported Options

All the options should be prefixed with - (hyphen)

Start Broker

Start a broker instance on host

pinot-admin.sh StartBroker -zkAddress localhost:2181 -clusterName PinotCluster -configFileName /path/to/broker.conf

Supported Options

All the options should be prefixed with - (hyphen)

Start Controller

Start a controller instance on host

pinot-admin.sh StartController -controllerMode helix_only -dataDir /path/to/data/dir -zkAddress localhost:2181 -clusterName PinotCluster -configFileName /path/to/controller.conf

Supported Options

All the options should be prefixed with - (hyphen)

Start Server

Start a server instance on host

pinot-admin.sh StartServer -dataDir /path/to/data/dir -zkAddress localhost:2181 -clusterName PinotCluster -configFileName /path/to/server.conf

Supported Options

All the options should be prefixed with - (hyphen)

Start Service Manager

Start multiple Pinot processes with all the default configurations using a single command.

pinot-admin.sh StartServiceManager -zkAddress localhost:2181 -clusterName PinotCluster -bootstrapServices CONTROLLER BROKER -bootstrapConfigPaths /path/to/controller.conf /path/to/broker.conf 

Supported Options

All the options should be prefixed with - (hyphen)

Show Cluster Info

Show all the available clusters namespaces along with metadata

pinot-admin.sh ShowClusterInfo -clusterName PinotCluster -zkAddress localhost:2181

Supported Options

All the options should be prefixed with - (hyphen)

Stop Process

Stop all the processes of the specified types running on the host.

pinot-admin.sh StopProcess -controller -broker -server

Supported Options

All the options should be prefixed with - (hyphen)

Upload Segments

Compress and upload segment files to server.

pinot-admin.sh UploadSegment -controllerHost localhost -controllerPort 9000 -segmentDir /path/to/local/dir -tableName myTable

Supported Options

All the options should be prefixed with - (hyphen)

Validate Config

Validate the table configs and schema present in Zookeeper.

pinot-admin.sh ValidateConfig -clusterName PinotCluster -zkAddress localhost:2181 -tableConfig -tableName mytable myTable2 -schema -schemaNames myschema myschema2

Supported Options

All the options should be prefixed with - (hyphen)

Validate Segment

Compares Helix Ideal state and External view for specified table prefixes.

pinot-admin.sh ValidateSegment -tablePrefix myTable -clusterName PinotCluster -zkAddress localhost:2181

Supported Options

All the options should be prefixed with - (hyphen)

Verify Cluster State

Verify if all the tables in the cluster have same Ideal State and External View.

pinot-admin.sh VerifyClusterState -zkAddress localhost:2181 -clusterName PinotCluster -tableName myTable -timeoutSec 10

Supported Options

All the options should be prefixed with - (hyphen)