Broker GRPC API

Apart from HTTP based query endpoint, Pinot also supports gRPC endpoint in broker.

Enable GRPC query entrypoint in broker

Add gRPC port config in pinot broker to enable the BrokerGrpcServer:

pinot.broker.grpc.port=8010

if you want to enable TLS, then use below configs:

pinot.broker.grpc.tls.enabled=true
pinot.broker.grpc.tls.port=8020

// Server side TLS is used. The client does not present a certificate.
// The server only verifies the client’s connection via its own certificate and doesn’t validate the client’s identity via TLS.
// Common in public APIs, where only the server needs to be trusted.
pinot.broker.grpctls.client.auth.enabled=false

// Config TLS keystore and truststore
pinot.broker.grpctls.keystore.path=/home/pinot/tls-store/keystore-internal.jks
pinot.broker.grpctls.keystore.password=changeit
pinot.broker.grpctls.keystore.type=JKS
pinot.broker.grpctls.truststore.path=/home/pinot/tls-store/truststore.jks
pinot.broker.grpctls.truststore.password=changeit
pinot.broker.grpctls.truststore.type=JKS

Broker GRPC Clients

Below are the examples of usage for pinot-java-client and pinot-jdbc-client .

Java Grpc Client

The main difference of the usage here is that ConnectionFactory will return a GrpcConnection instead of Connection.

If you want to use ARROW as the encoding type, you must start Java with

--add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED

(See https://arrow.apache.org/docs/java/install.html)

Below is a sample code to use the java client: GrpcConnection:

package org.apache.pinot.client.examples;

import java.io.IOException;
import java.util.Properties;
import org.apache.pinot.client.ConnectionFactory;
import org.apache.pinot.client.ResultSetGroup;
import org.apache.pinot.client.grpc.GrpcConnection;


public class PinotBrokerGrpcClientExample {

  private PinotBrokerGrpcClientExample() {
  }
  
  public static void main(String[] args)
      throws IOException {
    Properties properties = new Properties();
    properties.put("encoding", "JSON");
    properties.put("compression", "ZSTD");
    properties.put("blockRowSize", "10000");
    GrpcConnection grpcConnection = ConnectionFactory.fromControllerGrpc(properties, "localhost:9000");
    // GrpcConnection grpcConnection = ConnectionFactory.fromZookeeperGrpc(properties, "localhost:2123/QuickStartCluster");
    // GrpcConnection grpcConnection = ConnectionFactory.fromHostListGrpc(properties, List.of("localhost:8010"));
    ResultSetGroup resultSetGroup = grpcConnection.execute("SELECT * FROM airlineStats limit 1000");
    for (int i = 0; i < resultSetGroup.getResultSetCount(); i++) {
      org.apache.pinot.client.ResultSet resultSet = resultSetGroup.getResultSet(i);
      for (int rowId = 0; rowId < resultSet.getRowCount(); rowId++) {
        System.out.print("Row Id: " + rowId + "\t");
        for (int colId = 0; colId < resultSet.getColumnCount(); colId++) {
          System.out.print(resultSet.getString(rowId, colId) + "\t");
        }
        System.out.println();
      }
    }
    grpcConnection.close();
  }
}

JDBC Grpc Client

The main usage difference here is scheme is changed to pinotgrpc.

If you want to use ARROW as the encoding type, you must start Java with

--add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED

(See https://arrow.apache.org/docs/java/install.html)

Below is a sample code to use JDBC driver.

package org.apache.pinot.client.examples;

import java.io.IOException;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;
import java.sql.Statement;


public class PinotBrokerGrpcJdbcClientExample {

  private PinotBrokerGrpcJdbcClientExample() {
  }

  public static void main(String[] args)
      throws IOException {
    try (Connection connection = DriverManager.getConnection("jdbc:pinotgrpc://localhost:9000?blockRowSize=10000&encoding=JSON&compression=ZSTD");
        Statement statement = connection.createStatement();
        ResultSet resultSet = statement.executeQuery("SELECT * FROM airlineStats limit 1000")) {
      // Print results
      ResultSetMetaData metaData = resultSet.getMetaData();
      int columnCount = metaData.getColumnCount();
      while (resultSet.next()) {
        System.out.print("Row Id: " + resultSet.getRow() + "\t");
        for (int i = 1; i <= columnCount; i++) {
          System.out.print(metaData.getColumnName(i) + ": " + resultSet.getString(i) + "\t");
        }
        System.out.println();
      }
    } catch (SQLException e) {
      e.printStackTrace();
    }
  }
}

Client Configurations

Below are the parameters to set per connection basis:

1. blockRowSize: the number of rows per block that grpc response will return, default is 10000.

2. compression: the compression algorithm over the wire, default is ZSTD, other options:

   1. LZ4_FAST: fast than LZ4_HIGH but not as high compression rate

   2. LZ4_HIGH

   3. ZSTD (Default)

   4. DEFLATE

   5. GZIP

   6. SNAPPY

   7. PASS_THROUGH/NONE: no compression, fast but could be large data transfer over the wire.

3. encoding: how to do serialization and deserialization for ResultTable transport between BrokerGrpcServer and Grpc Client, default is JSON, other options:

   1. JSON(Default)

   2. ARROW

Benchmark

We did a simple benchmark of query:

SELECT * FROM airlineStats limit 1000

Will compress 97k (9.2MB) rows with block size 10K.

Compression

CompressionType	Compression Ratio	Response Latency(ms)	Latency Ratio
LZ4_FAST	42.19%	477.6	103.02%
LZ4_HIGH	30.04%	977.0	210.74%
ZSTD	26.10%	485.4	104.70%
DEFLATE	25.64%	810.0	174.72%
GZIP	25.64%	811.8	175.11%
SNAPPY	42.56%	461.2	99.48%
NONE	100%	463.6	100%

Block Size

We also tried for ZSTD the same data set with different block size:

BlockRowSize	Compression Ratio	Avg Latency(ms)
100	30.48%	539.6
1000	27.56%	478.3
10000	26.10%	485.4
100000	25.73%	561.2

Encoding

Tested with ZSTD compression for the same query but different encoding and block size:

BlockRowSize	Encoding	Compression	Avg Latency(ms)	Bytes	Compression Ratio
1000	JSON	ZSTD	473	2559246	27.56%
10000	JSON	ZSTD	473	2424082	26.10%
100000	JSON	ZSTD	594	2389717	25.73%
1000	ARROW	ZSTD	372	3054993	32.90%
10000	ARROW	ZSTD	367	2705245	29.13%
100000	ARROW	ZSTD	447	2809744	30.26%