Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TDP 1.1 BOM #90

Closed
wants to merge 12 commits into from
Closed

TDP 1.1 BOM #90

Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
d163d81
feat: update README with TDP 1.1 BOM
PACordonnier Oct 5, 2023
78f9a9a
feat: update upstream releases
PACordonnier Oct 23, 2023
6111cc6
fix: knox upstream link
PACordonnier Oct 23, 2023
7564f7e
feat(BOM): update hive with 3 digits branch
PACordonnier Nov 9, 2023
582b63d
fix(BOM): bad tdp version number on hadoop
PACordonnier Nov 9, 2023
cbf290a
fix(bom): hive 2 has been rebased
PACordonnier Nov 9, 2023
80a1d1a
feat(bom): rever to tez 0.9.1
PACordonnier Nov 9, 2023
14513a5
feat(bom): change branch for spark2
PACordonnier Nov 9, 2023
4f08d73
feat(bom): use new branch for spark 2 and 3
PACordonnier Nov 10, 2023
dfe4cca
feat(bom): updated bom for TDP stack 1.1
Mar 15, 2024
ecacf63
feat(bom): corrected Spark3 version
Mar 15, 2024
8acca71
feat(bom): updated tdp core, still work to do on tdp extra
Mar 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 26 additions & 25 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,26 +25,26 @@ Each component of TDP also has its own repository.

### TDP Core

The following table shows the core components of TDP as well as the Apache branch they were based on and the TDP branch which serves as base for our releases.

| Component | Version | Apache Git branch | TDP Git Branch | TDP commits |
| --------------------------- | ---------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
| Apache ZooKeeper | 3.4.6 | release-3.4.6 | XXX | X.X.X |
| Apache Hadoop | 3.1.1-0.0 | [rel/release-3.1.1](https://github.com/apache/hadoop/commits/branch-3.1.1) | [branch-3.1.1-TDP](https://github.com/TOSIT-IO/hadoop/commits/branch-3.1.1-TDP) | [compare](https://github.com/TOSIT-IO/hadoop/compare/branch-3.1.1...branch-3.1.1-TDP) |
| Apache Hive | 3.1.3-1.0 | [branch-3.1](https://github.com/apache/hive/commits/branch-3.1) | [branch-3.1-TDP](https://github.com/TOSIT-IO/hive/commits/branch-3.1-TDP) | [compare](https://github.com/TOSIT-IO/hive/compare/branch-3.1...branch-3.1-TDP) |
| Apache Hive 2 (for Spark 3) | 2.3.9-1.0 | [branch-2.3](https://github.com/apache/hive/commits/branch-2.3) | [branch-2.3-TDP](https://github.com/TOSIT-IO/hive/commits/branch-2.3-TDP) | [compare](https://github.com/TOSIT-IO/hive/compare/branch-2.3...branch-2.3-TDP) |
| Apache Hive 1 (for Spark 2) | 1.2.3-1.0 | [branch-1.2](https://github.com/apache/hive/commits/branch-1.2) | [branch-1.2-TDP](https://github.com/TOSIT-IO/hive/commits/branch-1.2-TDP) | [compare](https://github.com/TOSIT-IO/hive/compare/branch-1.2...branch-1.2-TDP) |
| Apache Tez | 0.9.1-1.0 | [branch-0.9.1](https://github.com/apache/tez/commits/branch-0.9.1) | [branch-0.9.1-TDP](https://github.com/TOSIT-IO/tez/commits/branch-0.9.1-TDP) | [compare](https://github.com/TOSIT-IO/tez/compare/branch-0.9.1...branch-0.9.1-TDP) |
| Apache Spark | 2.3.4-1.0 | [branch-2.3](https://github.com/apache/spark/commits/branch-2.3) | [branch-2.3-TDP](https://github.com/TOSIT-IO/spark/commits/branch-2.3-TDP) | [compare](https://github.com/TOSIT-IO/spark/compare/branch-2.3...branch-2.3-TDP) |
| Apache Spark 3 | 3.2.2-0.0 | [branch-3.2](https://github.com/apache/spark/commits/branch-3.2) | [branch-3.2-TDP](https://github.com/TOSIT-IO/spark/commits/branch-3.2-TDP) | [compare](https://github.com/TOSIT-IO/spark/compare/branch-3.2...branch-3.2-TDP) |
| Apache Ranger | 2.0.0-1.0 | [ranger-2.0](https://github.com/apache/ranger/commits/ranger-2.0) | [ranger-2.0-TDP](https://github.com/TOSIT-IO/ranger/commits/ranger-2.0-TDP) | [compare](https://github.com/TOSIT-IO/ranger/compare/ranger-2.0...ranger-2.0-TDP) |
| Apache Solr (for Ranger) | 7.7.3 | releases/lucene-solr/7.7.3 | XXX | X.X.X |
| Apache HBase | 2.1.10-1.0 | [branch-2.1](https://github.com/apache/hbase/commits/branch-2.1) | [branch-2.1-TDP](https://github.com/TOSIT-IO/hbase/commits/branch-2.1-TDP) | [compare](https://github.com/TOSIT-IO/hbase/compare/branch-2.1...branch-2.1-TDP) |
| Apache Phoenix | 5.1.3-1.0 | [5.1](https://github.com/apache/phoenix/commits/5.1) | [5.1.3-TDP](https://github.com/TOSIT-IO/phoenix/commits/5.1.3-TDP) | [compare](https://github.com/TOSIT-IO/phoenix/compare/5.1...5.1.3-TDP) |
| Apache Phoenix Query Server | 6.0.0-0.0 | [6.0.0](https://github.com/apache/phoenix-queryserver/commits/6.0.0) | [6.0.0-TDP](https://github.com/TOSIT-IO/phoenix-queryserver/commits/6.0.0-TDP) | [compare](https://github.com/TOSIT-IO/phoenix-queryserver/compare/6.0.0...6.0.0-TDP) |
| Apache Knox | 1.6.1-0.0 | [v1.6.1](https://github.com/apache/knox/commits/v1.6.1) | [v1.6.1-TDP](https://github.com/TOSIT-IO/knox/commits/v1.6.1-TDP) | [compare](https://github.com/TOSIT-IO/knox/compare/v1.6.1...v1.6.1-TDP) |
| Apache HBase Connectors | 1.0.0-0.0 | [rel/1.0.0](https://github.com/apache/hbase-connectors/commits/rel/1.0.0) | [branch-2.3.4-1.0.0-TDP](https://github.com/TOSIT-IO/hbase-connectors/commits/branch-2.3.4-1.0.0-TDP) | [compare](https://github.com/TOSIT-IO/hbase-connectors/compare/1.0.0...branch-2.3.4-1.0.0-TDP) |
| Apache HBase Operator tools | 1.1.0-0.0 | [rel/1.1.0](https://github.com/apache/hbase-operator-tools/commits/rel/1.1.0) | [branch-1.1.0-TDP](https://github.com/TOSIT-IO/hbase-operator-tools/commits/branch-1.1.0-TDP) | [compare](https://github.com/TOSIT-IO/hbase-operator-tools/compare/branch-1.1.0...branch-1.1.0-TDP) |
The following table shows the core components of TDP as well as the Apache branch they were based on and the `-basic` branch which serves as base for our releases and the `-fix` branch which includes patches for the correct functionning.

| Component | Version | Apache Git release | Basic Branch | Basic commits | Fix branch | Fix commits |
| --------------------------- | ---------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |---------------------------------|-----------------------------------|
| Apache ZooKeeper | 3.4.14 | [rel/release-3.4.14](https://github.com/apache/zookeeper/releases/tag/release-3.4.14)| XXX | X.X.X | XXX | X.X.X
| Apache Hadoop | 3.1.1-0.2 | [rel/release-3.1.1](https://github.com/apache/hadoop/commits/rel/release-3.1.1) | [branch-3.1.1-basic](https://github.com/TOSIT-IO/hadoop/commits/branch-3.1.1-basic) | [compare](https://github.com/apache/hadoop/compare/branch-3.1.1...TOSIT-IO:hadoop:branch-3.1.1-basic)| [branch-3.1.1-fix](https://github.com/TOSIT-IO/hadoop/commits/branch-3.1.1-fix) | [compare](https://github.com/TOSIT-IO/hadoop/compare/branch-3.1.1-basic...branch-3.1.1-fix)
| Apache Hive | 3.1.3-2.1 | [rel/release-3.1.3](https://github.com/apache/hive/commits/rel/release-3.1.3) | [branch-3.1.3-basic](https://github.com/TOSIT-IO/hive/commits/branch-3.1.3-basic) | [compare](https://github.com/TOSIT-IO/hive/compare/branch-3.1...branch-3.1.3-basic) | [branch-3.1.3-fix](https://github.com/TOSIT-IO/hive/commits/branch-3.1.3-fix) | [compare](https://github.com/TOSIT-IO/hive/compare/branch-3.1.3-basic...branch-3.1.3-fix)
| Apache Hive 2 (for Spark 3) | 2.3.9-1.0 | [rel/release-2.3.9](https://github.com/apache/hive/commits/rel/release-2.3.9) | [branch-2.3.9-basic](https://github.com/TOSIT-IO/hive/commits/branch-2.3.9-basic) | [compare](https://github.com/TOSIT-IO/hive/compare/branch-2.3...branch-2.3.9-basic) | [branch-2.3.9-fix](https://github.com/TOSIT-IO/hive/commits/branch-2.3.9-fix)| [compare](https://github.com/TOSIT-IO/hive/compare/branch-2.3.9-basic...branch-2.3.9-fix)
| Apache Hive 1 (for Spark 2) | 1.2.1.spark2-1.0 | [release-1.2.1-spark2](https://github.com/TOSIT-IO/spark-hive/commits/release-1.2.1-spark2) | [release-1.2.1-spark2-basic](https://github.com/TOSIT-IO/spark-hive/commits/release-1.2.1-spark2-basic) | [compare](https://github.com/TOSIT-IO/spark-hive/compare/release-1.2.1-spark2...release-1.2.1-spark2-basic) | [release-1.2.1-spark2-fix](https://github.com/TOSIT-IO/spark-hive/commits/release-1.2.1-spark2-fix) | [compare](https://github.com/TOSIT-IO/spark-hive/compare/release-1.2.1-spark2-basic...release-1.2.1-spark2-fix)
| Apache Tez | 0.9.2-1.0 | [branch-0.9.2](https://github.com/apache/tez/commits/rel/release-0.9.2) | [branch-0.9.2-basic](https://github.com/TOSIT-IO/tez/commits/branch-0.9.2-basic) | [compare](https://github.com/TOSIT-IO/tez/compare/branch-0.9.2...branch-0.9.2-basic) | [branch-0.9.2-fix](https://github.com/TOSIT-IO/tez/commits/branch-0.9.2-fix) | [compare](https://github.com/TOSIT-IO/tez/compare/branch-0.9.2-build...branch-0.9.2-fix)
| Apache Spark | 2.3.4-1.0 | [Last commit of branch-2.3](https://github.com/apache/spark/commits/75cc3b2da9ee0b51ecf0f13169f2b634e36a60c4) | [branch-2.3.4-basic](https://github.com/TOSIT-IO/spark/commits/branch-2.3.4-basic) | [compare](https://github.com/TOSIT-IO/spark/compare/branch-2.3...branch-2.3.4-basic) | [branch-2.3.4-fix](https://github.com/TOSIT-IO/spark/commits/branch-2.3.4-fix) | [compare](https://github.com/TOSIT-IO/spark/compare/branch-2.3.4-basic...branch-2.3.4-fix)
| Apache Spark 3 | 3.2.4-1.0 | [v3.2.4](https://github.com/apache/spark/commits/v3.2.4) | [branch-3.2-4-basic](https://github.com/TOSIT-IO/spark/commits/branch-3.2.4-basic) | [compare](https://github.com/TOSIT-IO/spark/compare/v3.2.4...TOSIT-IO:spark:branch-3.2.4-basic) | [branch-3.2-4-fix](https://github.com/TOSIT-IO/spark/commits/branch-3.2.4-fix) | [compare](https://github.com/TOSIT-IO/spark/compare/branch-3.2.4-basic...branch-3.2.4-fix)
| Apache Ranger | 2.0.0-2.0 | [Last commit of branch ranger-2.0](https://github.com/apache/ranger/commits/8d617c626b949cdadf8d914259f78d050556cc5d) | XXX | X.X.X | [ranger-2.0-0-fix](https://github.com/TOSIT-IO/ranger/commits/ranger-2.0.0-fix) | [compare](https://github.com/TOSIT-IO/ranger/compare/ranger-2.0...ranger-2.0.0-fix)
| Apache Solr (for Ranger) | 7.7.3 | 7 | XXX | X.X.X | XXX | X.X.X
| Apache HBase | 2.1.10-1.1 | [Last commit of branch-2.1](https://github.com/apache/hbase/commits/9f7e856a5f84a6f52d0197a03b7a56e1a3f0446c) | [branch-2.1.10-basic](https://github.com/TOSIT-IO/hbase/commits/branch-2.1.10-basic) | [compare](https://github.com/TOSIT-IO/hbase/compare/branch-2.1...branch-2.1.10-basic) | [branch-2.1.10-fix](https://github.com/TOSIT-IO/hbase/commits/branch-2.1.10-fix) | [compare](https://github.com/TOSIT-IO/hbase/compare/branch-2.1.10-basic...branch-2.1.10-fix)
| Apache Phoenix | 5.1.3-1.0 | [5.1.3](https://github.com/apache/phoenix/commits/5.1.3) | [branch-5.1.3-basic](https://github.com/TOSIT-IO/phoenix/commits/branch-5.1.3-basic) | [compare](https://github.com/TOSIT-IO/phoenix/compare/5.1...branch-5.1.3-basic) | [branch-5.1.3-fix](https://github.com/TOSIT-IO/phoenix/commits/branch-5.1.3-fix) | [compare](https://github.com/TOSIT-IO/phoenix/compare/branch-5.1.3-basic...branch-5.1.3-fix)
| Apache Phoenix Query Server | 6.0.0-0.1-SNAPSHOT | [6.0.0](https://github.com/apache/phoenix-queryserver/commits/6.0.0) | [branch-6.0.0-basic](https://github.com/TOSIT-IO/phoenix-queryserver/commits/branch-6.0.0-basic) | [compare](https://github.com/TOSIT-IO/phoenix-queryserver/compare/6.0.0...branch-6.0.0-basic) | [branch-6.0.0-fix](https://github.com/TOSIT-IO/phoenix-queryserver/commits/branch-6.0.0-fix) | [compare](https://github.com/TOSIT-IO/phoenix-queryserver/compare/branch-6.0.0-basic...branch-6.0.0-fix)
| Apache Knox | 1.6.1-0.1-SNAPSHOT | [v1.6.1](https://github.com/apache/knox/commits/v1.6.1-release) | [branch-1.6.1-basic](https://github.com/TOSIT-IO/knox/commits/branch-1.6.1-basic) | [compare](https://github.com/TOSIT-IO/knox/compare/v1.6.1...branch-1.6.1-basic) | [branch-1.6.1-fix](https://github.com/TOSIT-IO/knox/commits/branch-1.6.1-fix) | [compare](https://github.com/TOSIT-IO/knox/compare/branch-1.6.1-basic...branch-1.6.1-fix)
| Apache HBase Connectors | 1.0.0-0.0 | [rel/1.0.0](https://github.com/apache/hbase-connectors/commits/rel/1.0.0) | [branch-2.3.4-1.0.0-basic](https://github.com/TOSIT-IO/hbase-connectors/commits/branch-2.3.4-1.0.0-basic) | [compare](https://github.com/TOSIT-IO/hbase-connectors/compare/1.0.0...branch-2.3.4-1.0.0-basic) | [branch-2.3.4-1.0.0-fix](https://github.com/TOSIT-IO/hbase-connectors/commits/branch-2.3.4-1.0.0-fix) | [compare](https://github.com/TOSIT-IO/hbase-connectors/compare/branch-2.3.4-1.0.0-basic...branch-2.3.4-1.0.0-fix)
| Apache HBase Operator tools | 1.1.0-0.0 | [rel/1.1.0](https://github.com/apache/hbase-operator-tools/commits/rel/1.1.0) | [branch-1.1.0-basic](https://github.com/TOSIT-IO/hbase-operator-tools/commits/branch-1.1.0-basic) | [compare](https://github.com/TOSIT-IO/hbase-operator-tools/compare/branch-1.1.0...branch-1.1.0-basic) | [branch-1.1.0-fix](https://github.com/TOSIT-IO/hbase-operator-tools/commits/branch-1.1.0-fix) | [compare](https://github.com/TOSIT-IO/hbase-operator-tools/compare/branch-1.1.0-basic...branch-1.1.0-fix)

Versions are approximately based on the [HDP 3.1.5 release](https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/release-notes/content/hdp_relnotes.html).

Expand All @@ -58,12 +58,13 @@ Versions are approximately based on the [HDP 3.1.5 release](https://docs.clouder
- The project is not active enough. This is the case of Livy that has not been updated in 2 years.
- The project has some incompatibilities with other "TDP Core" projects' releases. This is the case of Kafka 2.X that relies on ZooKeeper 3.5.X (and cannot use the ZooKeeper 3.4.6 of "TDP Core").

| Component | Version | Apache Git branch | TDP Git Branch | TDP commits |
| Component | Version | Apache Git branch | Fix Branch | Fix commits |
| ---------------------------------- | ------- | ---------------------------------------------------------------- | ------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- |
| Apache ZooKeeper 3.5.9 (for Kafka) | 3.5.9 | release-3.5.9 | XXX | X.X.X |
| Apache Kafka | 2.8.2 | [2.8](https://github.com/TOSIT-IO/kafka/tree/2.8) | [2.8-TDP](https://github.com/TOSIT-IO/kafka/tree/2.8-TDP) | [compare](https://github.com/TOSIT-IO/kafka/compare/2.8...2.8-TDP) |
| Apache Livy | 0.8.0 | [master](https://github.com/TOSIT-IO/incubator-livy/tree/master) | [branch-0.8.0-TDP](https://github.com/TOSIT-IO/incubator-livy/tree/branch-0.8.0-TDP) | [compare](https://github.com/TOSIT-IO/incubator-livy/compare/master...branch-0.8.0-TDP) |
| Apache Airflow | 2.2.2 | 2.2.2 | XXX | X.X.X |
| Apache ZooKeeper 3.5.9 (for Kafka) | 3.5.9 | [release-3.5.9](https://github.com/apache/zookeeper/releases/tag/release-3.5.9) | XXX | X.X.X |
| Apache Kafka | 2.8.2-TDP-0.1.0-SNAPSHOT | [2.8](https://github.com/TOSIT-IO/kafka/tree/2.8) | [2.8.2-fix](https://github.com/TOSIT-IO/kafka/tree/2.8.2-fix) | [compare](https://github.com/TOSIT-IO/kafka/compare/2.8...2.8.2-fix) |
| Apache Livy | 0.8.0-incubating-TDP-0.1.0-SNAPSHOT | [master](https://github.com/TOSIT-IO/incubator-livy/tree/master) | [branch-0.8.0-fix](https://github.com/TOSIT-IO/incubator-livy/tree/branch-0.8.0-fix) | [compare](https://github.com/TOSIT-IO/incubator-livy/compare/master...branch-0.8.0-fix) |
| Apache Hue | 4.11.0-rocky8-py38 | [master](https://github.com/TOSIT-IO/hue/tree/master) | [release-4.11.0-fix-rocky8-py38](https://github.com/TOSIT-IO/hue/tree/release-4.11.0-fix-rocky8-py38) | [compare](https://github.com/TOSIT-IO/hue/compare/release-4.11.0-fix-rocky8-py38...cloudera%3Ahue%3Amaster) |
| Jupyterhub | 2.3.1-TDP-0.1.0 | [master](https://github.com/TOSIT-IO/jupyterhub-pkg) | XXX | X.X.X |

**Note:** A project can graduate from "TDP Extras" to "TDP Core" if enough people are supporting it and/or if it is made compatible with all the other projects of the stack.

Expand Down
Loading