Vendors librdkafka v2.12.1 as a submodule (default) and teaches the
notification teuthology task to set up Kafka 4.x KRaft brokers. The
distro-shipped librdkafka 1.6.1 cannot SCRAM-auth against Kafka 4.x
brokers (#75900); the bundled version fixes that. Build with
WITH_SYSTEM_RDKAFKA=ON to opt back into the distro library (floor 2.6.1).
follwinf changes needed with teh submodule:
* build kafka_stub only in case of system librdkafka
* use the same header location in submodule and system
* supress compilation warning to avoid -WError issues
2.12.1 matches Fedora(44)'s current librdkafka-devel, so the vendored copy
tracks a version that distro packagers have already vetted rather than
whichever release happens to be latest upstream.
The teuthology version matrix picks one of {3.9.2, 4.1, 4.2, 4.3} per run.
3.9.2 covers the last Zookeeper-based release for the migration cohort.
4.1/4.2/4.3 track Apache's current three supported minor lines and are
resolved to the latest patch at test time via dlcdn.apache.org/kafka/, so
new patches within a supported minor need no PR update. A new minor line
(4.4) requires renaming one YAML.
* new java version (17) is needed for kafka 4.2 and up (we wil use it on
all version)
* kerberos task need to run before the kafka install task
After pulling this branch, fetch the new submodule: