Edit

Examples and Scenarios

To help understand rebalance and its output better, here are some examples with different scenarios with table rebalance.

All examples below skip showing the instance assignment and segment assignment for brevity.

1. Increase replication factor

Changes:

  • Number of replicaGroups increased from 1 to 2 (replicaGroup based instance assignment)

  • New server tagged with correct DefaultTenant tag

{
  "jobId": "872d693f-07f2-48fd-9c11-98838ebaed6b",
  "status": "DONE",
  "description": "Dry-run summary mode",
  "rebalanceSummaryResult": {
    "serverInfo": {
      "numServersGettingNewSegments": 1,
      "numServers": {
        "valueBeforeRebalance": 1,
        "expectedValueAfterRebalance": 2
      },
      "serversAdded": [
        "Server_pinot-server-server-0-1_8098"
      ],
      "serversRemoved": [],
      "serversUnchanged": [
        "Server_pinot-server-server-0-0_8098"
      ],
      "serversGettingNewSegments": [
        "Server_pinot-server-server-0-1_8098"
      ],
      "serverSegmentChangeInfo": {
        "Server_pinot-server-server-0-1_8098": {
          "serverStatus": "ADDED",
          "totalSegmentsAfterRebalance": 15,
          "totalSegmentsBeforeRebalance": 0,
          "segmentsAdded": 15,
          "segmentsDeleted": 0,
          "segmentsUnchanged": 0,
          "tagList": [
            "DefaultTenant_OFFLINE",
            "DefaultTenant_REALTIME"
          ]
        },
        "Server_pinot-server-server-0-0_8098": {
          "serverStatus": "UNCHANGED",
          "totalSegmentsAfterRebalance": 15,
          "totalSegmentsBeforeRebalance": 15,
          "segmentsAdded": 0,
          "segmentsDeleted": 0,
          "segmentsUnchanged": 15,
          "tagList": [
            "DefaultTenant_OFFLINE",
            "DefaultTenant_REALTIME"
          ]
        }
      }
    },
    "segmentInfo": {
      "totalSegmentsToBeMoved": 15,
      "maxSegmentsAddedToASingleServer": 15,
      "estimatedAverageSegmentSizeInBytes": 478983831,
      "totalEstimatedDataToBeMovedInBytes": 7184757465,
      "replicationFactor": {
        "valueBeforeRebalance": 1,
        "expectedValueAfterRebalance": 2
      },
      "numSegmentsInSingleReplica": {
        "valueBeforeRebalance": 15,
        "expectedValueAfterRebalance": 15
      },
      "numSegmentsAcrossAllReplicas": {
        "valueBeforeRebalance": 15,
        "expectedValueAfterRebalance": 30
      }
    },
    "tagsInfo": [
      {
        "tagName": "DefaultTenant_OFFLINE",
        "numSegmentsToDownload": 15,
        "numSegmentsUnchanged": 15,
        "numServerParticipants": 2
      }
    ]
  },
  "instanceAssignment": {
    ...
  },
  "segmentAssignment": {
    ...
  }
}

2. Change instance assignment from balanced to replicaGroup based

Changes:

  • Change TableConfig from balanced to replicaGroup based assignment by adding the instanceAssignmentConfigMap

    • Replication factor remains the same. Instances per replica group chosen as 1

  • No change in tagged servers

3. Increase instances per replicaGroup

Changes:

  • Increase the number of instances per replica group from 1 to 2

4. Move table to a different Tenant

Changes:

  • Change the table's tenant tag to point to the new tenant tag

  • Tag servers on new tenant with new tenant tag

5. Scale Down table with balanced assignment

Changes:

  • Untag servers that should no longer host the given table

6. minimizeDataMovement flag comparison for increasing replication factor of replicaGroup based assignment

Changes for both scenarios:

  • Increase number of replicaGroups from 2 to 3, keep instances per replicaGroup the same

  • Ensure enough servers are tagged with the tenant tag

For each scenario, note the server stats in terms of how the server topology is changing. This can have a large effect on how much data is moved as part of the rebalance, and checking the summary along with the pre-checks can help identify if the changes are as expected.

Scenario 1: minimizeDataMovement=false

  • 2 servers added, 1 removed

Scenario 2: minimizeDataMovement=true

  • 1 server added

  1. Consuming segments rebalance in a REALTIME table

Under a table with a default assignment strategy, untag one of the two servers, then rebalance.

PreviousRebalance ServersNextRebalance Brokers

Last updated

Was this helpful?