LogoLogo
release-0.10.0
release-0.10.0
  • Introduction
  • Basics
    • Concepts
    • Architecture
    • Components
      • Cluster
      • Controller
      • Broker
      • Server
      • Minion
      • Tenant
      • Schema
      • Table
      • Segment
      • Deep Store
      • Pinot Data Explorer
    • Getting Started
      • Running Pinot locally
      • Running Pinot in Docker
      • Quick Start Examples
      • Running in Kubernetes
      • Running on public clouds
        • Running on Azure
        • Running on GCP
        • Running on AWS
      • Batch import example
      • Stream ingestion example
      • HDFS as Deep Storage
      • Troubleshooting Pinot
      • Frequently Asked Questions (FAQs)
        • General
        • Pinot On Kubernetes FAQ
        • Ingestion FAQ
        • Query FAQ
        • Operations FAQ
    • Import Data
      • Batch Ingestion
        • Spark
        • Hadoop
        • Backfill Data
        • Dimension Table
      • Stream ingestion
        • Apache Kafka
        • Amazon Kinesis
        • Apache Pulsar
      • Stream Ingestion with Upsert
      • File Systems
        • Amazon S3
        • Azure Data Lake Storage
        • HDFS
        • Google Cloud Storage
      • Input formats
      • Complex Type (Array, Map) Handling
    • Indexing
      • Forward Index
      • Inverted Index
      • Star-Tree Index
      • Bloom Filter
      • Range Index
      • Text search support
      • JSON Index
      • Geospatial
      • Timestamp Index
    • Releases
      • 0.10.0
      • 0.9.3
      • 0.9.2
      • 0.9.1
      • 0.9.0
      • 0.8.0
      • 0.7.1
      • 0.6.0
      • 0.5.0
      • 0.4.0
      • 0.3.0
      • 0.2.0
      • 0.1.0
    • Recipes
      • GitHub Events Stream
  • For Users
    • Query
      • Querying Pinot
      • Filtering with IdSet
      • Transformation Functions
      • Aggregation Functions
      • User-Defined Functions (UDFs)
      • Cardinality Estimation
      • Lookup UDF Join
      • Querying JSON data
      • Explain Plan
      • Grouping Algorithm
    • APIs
      • Broker Query API
        • Query Response Format
      • Controller Admin API
    • External Clients
      • JDBC
      • Java
      • Python
      • Golang
    • Tutorials
      • Use OSS as Deep Storage for Pinot
      • Ingest Parquet Files from S3 Using Spark
      • Creating Pinot Segments
      • Use S3 as Deep Storage for Pinot
      • Use S3 and Pinot in Docker
      • Batch Data Ingestion In Practice
      • Schema Evolution
  • For Developers
    • Basics
      • Extending Pinot
        • Writing Custom Aggregation Function
        • Segment Fetchers
      • Contribution Guidelines
      • Code Setup
      • Code Modules and Organization
      • Update Documentation
    • Advanced
      • Data Ingestion Overview
      • Ingestion Transformations
      • Null Value Support
      • Advanced Pinot Setup
    • Plugins
      • Write Custom Plugins
        • Input Format Plugin
        • Filesystem Plugin
        • Batch Segment Fetcher Plugin
        • Stream Ingestion Plugin
    • Design Documents
      • Segment Writer API
  • For Operators
    • Deployment and Monitoring
      • Setup cluster
      • Setup table
      • Setup ingestion
      • Decoupling Controller from the Data Path
      • Segment Assignment
      • Instance Assignment
      • Rebalance
        • Rebalance Servers
        • Rebalance Brokers
      • Tiered Storage
      • Pinot managed Offline flows
      • Minion merge rollup task
      • Access Control
      • Monitoring
      • Tuning
        • Realtime
        • Routing
      • Upgrading Pinot with confidence
    • Command-Line Interface (CLI)
    • Configuration Recommendation Engine
    • Tutorials
      • Authentication, Authorization, and ACLs
      • Configuring TLS/SSL
      • Build Docker Images
      • Running Pinot in Production
      • Kubernetes Deployment
      • Amazon EKS (Kafka)
      • Amazon MSK (Kafka)
      • Monitor Pinot using Prometheus and Grafana
  • Configuration Reference
    • Cluster
    • Controller
    • Broker
    • Server
    • Table
    • Schema
    • Ingestion Job Spec
    • Functions
      • ABS
      • ADD
      • arrayConcatInt
      • arrayConcatString
      • arrayContainsInt
      • arrayContainsString
      • arrayDistinctString
      • arrayDistinctInt
      • arrayIndexOfInt
      • arrayIndexOfString
      • ARRAYLENGTH
      • arrayRemoveInt
      • arrayRemoveString
      • arrayReverseInt
      • arrayReverseString
      • arraySliceInt
      • arraySliceString
      • arraySortInt
      • arraySortString
      • arrayUnionInt
      • arrayUnionString
      • AVGMV
      • ceil
      • CHR
      • codepoint
      • concat
      • count
      • COUNTMV
      • day
      • dayOfWeek
      • dayOfYear
      • DISTINCT
      • DISTINCTCOUNT
      • DISTINCTCOUNTBITMAP
      • DISTINCTCOUNTBITMAPMV
      • DISTINCTCOUNTHLL
      • DISTINCTCOUNTHLLMV
      • DISTINCTCOUNTMV
      • DISTINCTCOUNTRAWHLL
      • DISTINCTCOUNTRAWHLLMV
      • DISTINCTCOUNTRAWTHETASKETCH
      • DISTINCTCOUNTTHETASKETCH
      • DIV
      • DATETIMECONVERT
      • DATETRUNC
      • exp
      • FLOOR
      • FromDateTime
      • FromEpoch
      • FromEpochBucket
      • hour
      • JSONFORMAT
      • JSONPATH
      • JSONPATHARRAY
      • JSONPATHARRAYDEFAULTEMPTY
      • JSONPATHDOUBLE
      • JSONPATHLONG
      • JSONPATHSTRING
      • jsonextractkey
      • jsonextractscalar
      • length
      • ln
      • lower
      • lpad
      • ltrim
      • max
      • MAXMV
      • MD5
      • millisecond
      • min
      • minmaxrange
      • MINMAXRANGEMV
      • MINMV
      • minute
      • MOD
      • mode
      • month
      • mult
      • now
      • percentile
      • percentileest
      • percentileestmv
      • percentilemv
      • percentiletdigest
      • percentiletdigestmv
      • quarter
      • regexpExtract
      • remove
      • replace
      • reverse
      • round
      • rpad
      • rtrim
      • second
      • SEGMENTPARTITIONEDDISTINCTCOUNT
      • sha
      • sha256
      • sha512
      • sqrt
      • startswith
      • ST_AsBinary
      • ST_AsText
      • ST_Contains
      • ST_Distance
      • ST_GeogFromText
      • ST_GeogFromWKB
      • ST_GeometryType
      • ST_GeomFromText
      • ST_GeomFromWKB
      • STPOINT
      • ST_Polygon
      • strpos
      • ST_Union
      • SUB
      • substr
      • sum
      • summv
      • TIMECONVERT
      • timezoneHour
      • timezoneMinute
      • ToDateTime
      • ToEpoch
      • ToEpochBucket
      • ToEpochRounded
      • TOJSONMAPSTR
      • toGeometry
      • toSphericalGeography
      • trim
      • upper
      • VALUEIN
      • week
      • year
      • yearOfWeek
  • RESOURCES
    • Community
    • Team
    • Blogs
    • Presentations
    • Videos
  • Integrations
    • Tableau
    • Trino
    • ThirdEye
    • Superset
    • Presto
Powered by GitBook
On this page
  • 2022
  • 2021
  • 2020
  • 2019 and earlier

Was this helpful?

Export as PDF
  1. For Developers

Design Documents

Links to all the design docs

This page contains links to all the design documents

2022

Name
Authors
Date

Jackie Jiang

Apr 2022

Evan Noon

Mar 2022

Rong Rong

Feb 2022

Sajjad Moradi, Subbu Subramaniam

Feb 2022

Siddharth Teotia

Feb 2022

Yupeng Fu

Jan 2022

Richard Startin

Jan 2022

2021

Name
Authors
Date

Richard Startin

Nov 2021

Seunghyun Lee, Jialiang Li

Nov 2021

Atri Sharma

Oct 2021

Richard Startin

Sep 2021

Lakshmanan Velusamy, Weixiang Sun

Sep 2021

Atri Sharma

Sep 2021

Yupeng Fu

May 2021

Neha Pawar

Feb 2021

Qiaochu Liu

Feb 2021

2020

Name
Authors
Date

Neha Pawar

Dec 2020

Kishore Gopalakrishna

Oct 2020

Dharak Kharod

Oct 2020

Jackie Jiang

Oct 2020

Neha Pawar

Sep 2020

Yupeng Fu

Sep 2020

Jackie Jiang

Sep 2020

Neha

Aug 2020

Chinmay

Aug 2020

Jackie Jiang

July 2020

Seunghyun

June 2020

Neha

June 2020

Jialiang Li

May 2020

Neha

May 2020

Subbu

May 2020

Kishore

May 2020

Kishore

Apr 2020

Neha

Apr 2020

Yupeng Fu

Apr 2020

Neha

Mar 2020

Neha

Mar 2020

Alex Pucher

Mar 2020

2019 and earlier

Name
Authors
Date

Siddharth Teotia

Nov 2019

Xiang Fu

Nov 2019

Jackie Jiang

Nov 2019

Neha

Oct 2019

Jialiang Li

Sep 2019

Jialiang Li

Jun 2019

James

Jun 2019

Sunitha Beeram

May 2019

Ting, Chinmay

May 2019

Jia Guo

May 2019

Subbu

Feb 2019

Seunghyun Lee

2018

Jackie Jiang

2017

Subbu

2017

Subbu

2017

Subbu

2016

Subbu

2016

PreviousStream Ingestion PluginNextSegment Writer API

Last updated 3 years ago

Was this helpful?

Server Failure Detector
Generalized Pre-Aggregation
Multi-stage Query Engine
Pause/Resume Stream
Pinot Flink Connector
Pluggable Indexes
Improve raw variable length forward index format
Consistent Push and Rollback
Aggregation Filter
Range encoded bit-sliced indexes
Time Series Aggregate Functions
Native Text Indices
Geospatial Support
Segment Writer API
Partial Upsert
Kinesis Integration in Pinot
JSON Indexing
Lookup Join
Strict Replica-Group Routing
Cluster Manager UI
Pinot Upsert Revisited
Use IdSet for Id Filtering
Pinot managed offline flows
Server Health Checker
HAVING and Post-Aggregation Support
Segment Merge and Rollup (Updated)
Filtering during ingestion
Segment Preprocessing Hadoop Job
Flattening during ingestion
Compatibility Regression Testing
Refactor pinot-core and pinot-common
Range Indexing in Pinot
Deprecate TimeFieldSpec, make DateTimeFieldSpec mainstream
Geospatial support in Pinot
Column transformation during ingestion
Tiered Storage
Synthetic Data Generator for Pinot
Text Search
Pinot SQL migration Plan
Segment Admin Rest APIs
Order By in aggregations
Pinot Benchmark as a Service
Controller Separation between Helix and Pinot
Upsert
Pinot Freshness Metric
By passing deep-store requirement for Realtime segment completion
Project Tuna: Automatic Inverted Index Recommendation
Segment Completion Enhancement for deep storage support
Segment Merge and Rollup
Pinot-Minion Service
Consuming and indexing rows in realtime
Partition Aware Query Routing
Query Processing
Expressions and UDFs
Fault Domain Awareness