Dependency Management
Executive Summary
This document outlines guidelines for managing external dependencies in the Apache Pinot Project. By enforcing these guidelines, we can ensure that contributors continually apply dependency management best practices in a consistent and centralized manner across the code base. These practices lead to a more predictable dependency graph and allows third-party builds dependent on OSS to repeatably override dependencies as needed for compliance or other reasons. See OSS issue for more details on the motivation.
Dependency Management Guidelines
When adding new dependencies or updating the version of an existing dependency, follow these guidelines.
For standard (non plugin) Pinot subprojects:
Define all versions inside of the top-level Pinot POM’s dependencyManagement section.
Use properties to define dependency versions and refer to the property when declaring your dependency in the Pinot POM. This allows the version to be easily overridden using Maven’s Versions plugin.
Include the dependency inside the dependencies section of the subproject but do not define a version.
If a BOM version of a dependency exists, favor the BOM so that any transitive dependencies in the dependency group are also pinned at the same version. For example, prefer
com.fasterxml.jackson:jackson-bom
overcom.fasterxml.jackson.core:jackson-annotations
to enforce that all transitive dependencies for the Jackson libraries are at the same version.
For Pinot plugin subprojects:
If the required dependency (and version) is present in the Pinot POM, do not shade the dependency and reference it without a version in the subproject’s dependencies section.
If the dependency is not present in the Pinot POM, or is present at a different version, use the maven-shade-plugin to relocate the library, so you can avoid class loading conflicts due to duplicate classes from Pinot POM or other plugins. In such cases, you can also define the dependency inside the subproject’s POM to pin its version. It is the plugin’s responsibility to ensure that no Pinot methods are called that reference conflicting libraries, and thus avoid failing with a
MethodNotFoundException
at runtime. For example, a plugin may be able to callJsonUtils.objectToString
but notJsonUtils.objectToJsonNode
when shading Jackson libraries.