release-1.2.0

Query Response Format

Find Pinot query response format examples for selection, aggregation, and group by queries formatted in a SQL-like structure. Also find details about each field included in the Pinot broker query response.

To learn more about how a Pinot broker routes and processes queries, computes the query explain plan, and ways to optimize queries, see the following topics:

Standard-SQL response

The query response is returned in a SQL-like tabular structure from the standard-SQL endpoint.

$ curl -H "Content-Type: application/json" -X POST \
   -d '{"sql":"SELECT moo, bar, foo FROM myTable ORDER BY foo DESC"}' \
   http://localhost:8099/query/sql
{
  "exceptions": [], 
  "minConsumingFreshnessTimeMs": 0, 
  "numConsumingSegmentsQueried": 0, 
  "numDocsScanned": 6, 
  "numEntriesScannedInFilter": 0, 
  "numEntriesScannedPostFilter": 18, 
  "numGroupsLimitReached": false, 
  "numSegmentsMatched": 2, 
  "numSegmentsProcessed": 2, 
  "numSegmentsQueried": 2, 
  "numServersQueried": 1, 
  "numServersResponded": 1, 
  "resultTable": {
    "dataSchema": {
      "columnDataTypes": [
        "LONG",
        "INT",
        "STRING"
      ], 
      "columnNames": [
        "moo", 
        "bar",
        "foo"
      ]
    }, 
    "rows": [
      [ 
        40015, 
        2019,
        "xyz"
      ], 
      [
        1002,
        2001,
        "pqr"
      ], 
      [
        20555,
        1988,
        "pqr"
      ],
      [ 
        203,
        2010,
        "pqr"
      ], 
      [
        500,
        2008,
        "abc"
      ], 
      [
        60, 
        2003,
        "abc"
      ]
    ]
  }, 
  "segmentStatistics": [], 
  "timeUsedMs": 4, 
  "totalDocs": 6, 
  "traceInfo": {}
}

Broker query response fields

Response FieldDescription

resultTable

Contains everything needed to process the response

resultTable.dataSchema

Describes the schema of the response, including columnNames and their dataTypes

resultTable.dataSchema.columnNames

columnNames in the response

resultTable.dataSchema.columnDataTypes

dataTypes for each column

resultTable.rows

Actual content with values. This is an array of arrays. The number of rows depends on the limit value in the query. The number of columns in each row is equal to the length of resultTable.dataSchema.columnNames

timeUsedms

Total time taken as seen by the broker before sending the response back to the client.

totalDocs

Number of documents/records in the table.

numServersQueried

Represents the number of servers queried by the broker (may be less than the total number of servers since the broker can apply some optimizations to minimize the number of servers).

numServersResponded

This should be equal to the numServersQueried. If this is not the same, then one of more servers might have timed out. If numServersQueried != numServersResponded, the results can be considered partial and clients can retry the query with exponential back off.

numSegmentsQueried

The total number of segmentsQueried for a query. May be less than the total number of segments if the broker applies optimizations.

The broker decides how many segments to query on each server, based on broker pruning logic. The server decides how many of these segments to actually look at, based on server pruning logic. After processing segments for a query, fewer may have the matching records. In general, numSegmentsQueried >= numSegmentsProcessed >= numSegmentsMatched.

numSegmentsMatched

The number of segments processed with at least one document matched in the query response.

numSegmentsProcessed

The number of segment operators used to process segments. Indicates the effectiveness of the pruning logic. For more information, see query plans for:

numDocScanned

The number of docs/records selected after the filter phase.

numEntriesScannedInFilter

The number of entries scanned in the filtering phase of query execution.

Can be larger than the total scanned doc count because of multiple filtering predicates or multi-value entries.

Can also be smaller than the total scanned doc count if indexing is used for filtering.

This along with numEntriesScannedInPostFilter indicates where most of the time is spent during query processing. If this value is high, enabling indexing for columns in tableConfig is a way to bring it down.

numEntriesScannedPostFilter

The number of entries scanned after the filtering phase of query execution, ie. aggregation and/or group-by phases. This is equivalent to numDocScanned * number of projected columns.

This along with numEntriesScannedInFilter indicates where most of the time is spent during query processing.

A high number for this means the selectivity is low (that is, Pinot needs to scan a lot of records to answer the query). If this is high, consider using star-tree index. (A regular inverted/bitmap index won't improve performance.)

numGroupsLimitReached

If the query has a group by clause and top K, Pinot drops new entries after the numGroupsLimit is reached. If this boolean is set to true, the query result may not be accurate. The default value for numGroupsLimit is 100k, and should be sufficient for most use cases.

exceptions

Will contain the stack trace if there is any exception processing the query.

segmentStatistics

N/A

stageStats

In multi-stage queries, this field contains the stats for each stage. See Understanding multi-stage stats to know more about how to interpret them.

PreviousBroker Query APINextController Admin API