# Broker GRPC API Apart from HTTP based query endpoint, Pinot also supports gRPC endpoint in broker. ## Enable GRPC query entrypoint in broker Add gRPC port config in pinot broker to enable the [`BrokerGrpcServer`](https://github.com/apache/pinot/blob/master/pinot-broker/src/main/java/org/apache/pinot/broker/grpc/BrokerGrpcServer.java): ```editorconfig pinot.broker.grpc.port=8010 ``` if you want to enable TLS, then use below configs: ```editorconfig pinot.broker.grpc.tls.enabled=true pinot.broker.grpc.tls.port=8020 // Server side TLS is used. The client does not present a certificate. // The server only verifies the client’s connection via its own certificate and doesn’t validate the client’s identity via TLS. // Common in public APIs, where only the server needs to be trusted. pinot.broker.grpctls.client.auth.enabled=false // Config TLS keystore and truststore pinot.broker.grpctls.keystore.path=/home/pinot/tls-store/keystore-internal.jks pinot.broker.grpctls.keystore.password=changeit pinot.broker.grpctls.keystore.type=JKS pinot.broker.grpctls.truststore.path=/home/pinot/tls-store/truststore.jks pinot.broker.grpctls.truststore.password=changeit pinot.broker.grpctls.truststore.type=JKS ``` ## Broker GRPC Clients Below are the examples of usage for `pinot-java-client` and `pinot-jdbc-client` . ### Java Grpc Client The main difference of the usage here is that `ConnectionFactory` will return a **`GrpcConnection`** instead of **`Connection`**. {% hint style="warning" %} If you want to use ARROW as the encoding type, you must start Java with `--add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED` (See ) {% endhint %} Below is a sample code to use the java client: **GrpcConnection**: ```java package org.apache.pinot.client.examples; import java.io.IOException; import java.util.Properties; import org.apache.pinot.client.ConnectionFactory; import org.apache.pinot.client.ResultSetGroup; import org.apache.pinot.client.grpc.GrpcConnection; public class PinotBrokerGrpcClientExample { private PinotBrokerGrpcClientExample() { } public static void main(String[] args) throws IOException { Properties properties = new Properties(); properties.put("encoding", "JSON"); properties.put("compression", "ZSTD"); properties.put("blockRowSize", "10000"); GrpcConnection grpcConnection = ConnectionFactory.fromControllerGrpc(properties, "localhost:9000"); // GrpcConnection grpcConnection = ConnectionFactory.fromZookeeperGrpc(properties, "localhost:2123/QuickStartCluster"); // GrpcConnection grpcConnection = ConnectionFactory.fromHostListGrpc(properties, List.of("localhost:8010")); ResultSetGroup resultSetGroup = grpcConnection.execute("SELECT * FROM airlineStats limit 1000"); for (int i = 0; i < resultSetGroup.getResultSetCount(); i++) { org.apache.pinot.client.ResultSet resultSet = resultSetGroup.getResultSet(i); for (int rowId = 0; rowId < resultSet.getRowCount(); rowId++) { System.out.print("Row Id: " + rowId + "\t"); for (int colId = 0; colId < resultSet.getColumnCount(); colId++) { System.out.print(resultSet.getString(rowId, colId) + "\t"); } System.out.println(); } } grpcConnection.close(); } } ``` ### JDBC Grpc Client The main usage difference here is scheme is changed to `pinotgrpc`. {% hint style="warning" %} If you want to use ARROW as the encoding type, you must start Java with `--add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED` (See ) {% endhint %} Below is a sample code to use JDBC driver. ```java package org.apache.pinot.client.examples; import java.io.IOException; import java.sql.Connection; import java.sql.DriverManager; import java.sql.ResultSet; import java.sql.ResultSetMetaData; import java.sql.SQLException; import java.sql.Statement; public class PinotBrokerGrpcJdbcClientExample { private PinotBrokerGrpcJdbcClientExample() { } public static void main(String[] args) throws IOException { try (Connection connection = DriverManager.getConnection("jdbc:pinotgrpc://localhost:9000?blockRowSize=10000&encoding=JSON&compression=ZSTD"); Statement statement = connection.createStatement(); ResultSet resultSet = statement.executeQuery("SELECT * FROM airlineStats limit 1000")) { // Print results ResultSetMetaData metaData = resultSet.getMetaData(); int columnCount = metaData.getColumnCount(); while (resultSet.next()) { System.out.print("Row Id: " + resultSet.getRow() + "\t"); for (int i = 1; i <= columnCount; i++) { System.out.print(metaData.getColumnName(i) + ": " + resultSet.getString(i) + "\t"); } System.out.println(); } } catch (SQLException e) { e.printStackTrace(); } } } ``` ## Client Configurations Below are the parameters to set per connection basis: 1. **blockRowSize**: the number of rows per block that grpc response will return, **default** is **10000**. 2. **compression**: the compression algorithm over the wire, **default** is **ZSTD**, other options: 1. LZ4\_FAST: fast than LZ4\_HIGH but not as high compression rate 2. LZ4\_HIGH 3. **ZSTD (Default)** 4. DEFLATE 5. GZIP 6. SNAPPY 7. PASS\_THROUGH/NONE: no compression, fast but could be large data transfer over the wire. 3. **encoding**: how to do serialization and deserialization for ResultTable transport between BrokerGrpcServer and Grpc Client, **default** is **JSON**, other options: 1. **JSON(Default)** 2. ARROW ## Benchmark We did a simple benchmark of query: ``` SELECT * FROM airlineStats limit 1000 ``` Will compress **97k (9.2MB)** rows with block size **10K**. ### Compression
CompressionTypeCompression RatioResponse Latency(ms)Latency Ratio
LZ4_FAST42.19%477.6103.02%
LZ4_HIGH30.04%977.0210.74%
ZSTD26.10%485.4104.70%
DEFLATE25.64%810.0174.72%
GZIP25.64%811.8175.11%
SNAPPY42.56%461.299.48%
NONE100%463.6100%
### Block Size We also tried for ZSTD the same data set with different block size: | BlockRowSize | Compression Ratio | Avg Latency(ms) | | ------------ | ----------------------------------------- | ---------------------------------------- | | 100 | 30.48% | 539.6 | | 1000 | 27.56% | 478.3 | | 10000 | 26.10% | 485.4 | | 100000 | 25.73% | 561.2 | ### Encoding Tested with ZSTD compression for the same query but different encoding and block size:
BlockRowSizeEncodingCompressionAvg Latency(ms)BytesCompression Ratio
1000JSONZSTD473255924627.56%
10000JSONZSTD473242408226.10%
100000JSONZSTD594238971725.73%
1000ARROWZSTD372305499332.90%
10000ARROWZSTD367270524529.13%
100000ARROWZSTD447280974430.26%
--- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.pinot.apache.org/release-1.4.0/for-users/api/broker-grpc-api.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.