# FST Index

The FST (Finite State Transducer) index accelerates regex queries on STRING columns by building over dictionary values. It reduces the on-disk index size by 4-6x compared to scanning the full dictionary.

## When to use

Use an FST index when your queries use `LIKE` or `REGEXP_LIKE` predicates on string columns. It is especially effective for prefix-matching patterns. For full-text search with tokenization, use the [text index](/build-with-pinot/indexing/text-search-support.md) instead.

## Supported column types

* STRING columns only
* Must be single-valued
* Must have dictionary values available. This can come from a dictionary-encoded forward index, or Pinot can materialize a standalone dictionary while keeping the forward index RAW.

## Limitations

* Only supports regex queries (`LIKE` and `REGEXP_LIKE` predicates).
* Only supported on stored or completed segments (not consuming segments in real-time tables).
* Only supported on columns with dictionary values available.
* Works best for prefix queries. Suffix-only or infix-only patterns may not benefit as much.

{% hint style="info" %}
Lucene FST is **case-sensitive**. When using an FST index in queries, ensure your pattern matches the case stored in the data. For example, `WHERE colA LIKE '%Value%'` will match "Value" but not "value". For case-insensitive matching, use the [IFST index](#case-insensitive-fst-index-ifst) instead.
{% endhint %}

For more information on FST construction, see the [Lucene FST documentation](https://lucene.apache.org/core/9_10_0/core/org/apache/lucene/util/fst/FST.html).

## Configuration

To enable the FST index on a column:

{% code title="Recommended: fieldConfigList" %}

```json
{
  "fieldConfigList": [
    {
      "name": "text_col_1",
      "encodingType": "DICTIONARY",
      "indexType": "FST"
    }
  ]
}
```

{% endcode %}

The FST index generates one index file (`.lucene.fst`). If you keep the forward index RAW, Pinot materializes a standalone dictionary for the FST automatically. If an inverted index is also enabled on the column, FST can take advantage of it for faster lookups.

## Query examples

Prefix match:

```sql
SELECT productName
FROM products
WHERE productName LIKE 'iPhone%'
```

Regex match:

```sql
SELECT email
FROM users
WHERE REGEXP_LIKE(email, '^admin.*@example\\.com$')
```

Infix match:

```sql
SELECT description
FROM products
WHERE description LIKE '%wireless%'
```

## Case-insensitive FST index (IFST)

The case-insensitive FST index (IFST) provides the same functionality as the standard FST index but with case-insensitive matching. This eliminates the need to handle case sensitivity manually in queries.

* Supports case-insensitive regex queries.
* Only supported on stored or completed segments (not consuming segments).
* Only supported on STRING columns with dictionary values available.
* Works best for prefix queries with case-insensitive matching.

### Configuration

{% code title="IFST configuration" %}

```json
{
  "fieldConfigList": [
    {
      "name": "notes",
      "encodingType": "DICTIONARY",
      "indexes": {
        "ifst": {
          "enabled": true
        }
      }
    }
  ]
}
```

{% endcode %}

The case-insensitive FST index generates one index file (`.lucene.ifst`).

For more information about enabling indexes, see [enabling indexes](/build-with-pinot/indexing.md#how-pinot-applies-indexes).


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.pinot.apache.org/build-with-pinot/indexing/fst-index.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.