Confluent Platform Component Changelogs¶

This topic provides changelogs for the individual Confluent Platform components.

Version 6.1.0¶

Community Features¶

Common¶

PR-309 - SEC-1789 Bump bouncycastle
PR-306 - Bump Junit version to 4.13.1
PR-305 - SEC-1474—Upgraded bcpkix-jdk15on to 1.66
PR-301 - Upgrade scala to 2.13.3 to align with Kafka

Kafka¶

PR-9678 - KAFKA-10798; Ensure response is delayed for failed SASL authentication with connection close delay
PR-9702 - CVE-2020-25649: bumping jackson to patched version 2.10.5.1
PR-9661 - KAFKA-10776: update the doc to add version attribute in RequstsPerSec metrics
PR-9729 - KAFKA-10839: improve consumer group coordinator unavailable message
PR-9747 - KAFKA-10852: AlterIsr should not be throttled
PR-9723 - Cherry-pick stream producer fix 10813
PR-9698 - KAFKA-10811: Correct the MirrorConnectorsIntegrationTest to correctly mask the exit procedures
PR-8965 - KAFKA-8147: Add changelog topic configuration to KTable suppress
PR-9669 - KAFKA-10792: Prevent source task shutdown from blocking herder thread
PR-9674 - KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState
PR-9677 - KAFKA-10799 AlterIsr utilizes ReplicaManager ISR metrics
PR-9648 - KAFKA-10758: ProcessorTopology should only consider its own nodes when updating regex source topics
PR-9634 - KAFKA-10755: Should consider commit latency when computing next commit timestamp
PR-9633 - KAFKA-10706; Ensure leader epoch cache is cleaned after truncation to end offset
PR-9613 - Cherry-pick KAFKA-10687 to 2.7
PR-9515 - KAFKA-10651: read offsets directly from checkpoint for uninitialized tasks
PR-9534 - KAFKA-10664: Delete existing checkpoint when writing empty offsets
PR-9529 - Revert initial principal from 2.7
PR-9521 - KAFKA-10638: Fix QueryableStateIntegrationTest
PR-9506 - KAFKA-10647; Only serialize owned partitions when consumer protocol version >= 1
PR-9486 - KAFKA-9381: Fix publishing valid scaladoc for streams-scala
PR-9464 - KAFKA-10616: Always call prepare-commit before suspending for active tasks
PR-9479 - KAFKA-10631: Handle ProducerFencedException on offset commit
PR-9270 - KAFKA-10284: Group membership update due to static member rejoin should be persisted
PR-9406 - KAFKA-10520; Ensure transactional producers poll if leastLoadedNode not available with max.in.flight=1
PR-9237 - KAFKA-10454 / Update copartitionSourceGroups when optimization algorithm is triggered
PR-9338 - KAFKA-10515: Properly initialize nullable Serdes with default values
PR-9431 - KAFKA-10426: Deadlock on session key update.
PR-9455 - KAFKA-10599: Implement basic CLI tool for feature versioning system
PR-9429 - KAFKA-10572 mirror-maker config changes for KIP-629
PR-9448 - KAFKA-10605: KIP-478: Deprecate old PAPI registration methods
PR-9383 - KAFKA-10455: Ensure that probing rebalances always occur
PR-9343 - KAFKA-10332: Update MM2 refreshTopicPartitions() logic
PR-9425 - KAFKA-10600: Connect should not add error to connector validation values for properties not in connector’s ConfigDef
PR-9432 - KAFKA-10559: Not letting TimeoutException shutdown the app during internal topic validation
PR-9422 - KAFKA-10602: Make RetryWithToleranceOperator thread safe
PR-9407 - KAFKA-10611: Merge log error to avoid double error
PR-9438 - KAFKA-10613: Only set leader epoch when list-offset version >= 4
PR-8150 - KAFKA-9587: Producer configs are omitted in the documentation
PR-9353 - KAFKA-10521; Skip partition watch registration when AlterIsr is expected
PR-9403 - KAFKA-10573 Update connect transforms configs for KIP-629
PR-9415 - KAFKA-10494: eager handling of sending old values
PR-9396 - KAFKA-10437: Implement new PAPI support for test-utils
PR-8891 - KAFKA-10143: Improve test coverage for throttle changes during reassignment
PR-9375 - KAFKA-10574: Fix infinite loop in Values::parseString
PR-9408 - KAFKA-10598: Improve IQ name and type checks
PR-9020 - KAFKA-10271 Performance regression while fetching a key from a single partition
PR-9380 - KAFKA-7334: Suggest changing config for state.dir in case of FileNotF…
PR-9393 - KAFKA-10028: Minor fixes to describeFeatures and updateFeatures apis
PR-9139 - KAFKA-9929: Support backward iterator on SessionStore
PR-9388 - KAFKA-10562: Properly invoke new StateStoreContext init
PR-9373 - KAFKA-10564: only process non-empty task directories when internally cleaning obsolete state stores
PR-9385 - KAFKA-9274: fix incorrect default value for task.timeout.ms config
PR-9247 - KAFKA-10362: When resuming Streams active task with EOS, the checkpoint file is deleted
PR-9001 - KAFKA-10028: Implement write path for feature versioning system (KIP-584)
PR-9196 - KAFKA-10402: Upgrade system tests to python3
PR-9280 - KAFKA-10186: Abort transaction with pending data with TransactionAbortedException
PR-9321 - KAFKA-9929: fix: add missing default implementations
PR-8910 - KAFKA-10188: Prevent SinkTask::preCommit from being called after SinkTask::stop
PR-9345 - KAFKA-10338; Support PEM format for SSL key and trust stores (KIP-651)
PR-9348 - KAFKA-10527; Voters should not reinitialize as leader in same epoch
PR-9099 - KAFKA-6733: Printing additional ConsumerRecord fields in DefaultMessageFormatter
PR-9320 - KAFKA-10439: Connect’s Values to parse BigInteger as Decimal with zero scale.
PR-9347 - KAFKA-10531: Check for negative values to Thread.sleep call
PR-9361 - KAFKA-10535: Split ProcessorContext into Processor/StateStore/Record Contexts
PR-8730 - KAFKA-10048: Possible data gap for a consumer after a failover when u…
PR-9306 - KAFKA-10477: Enabling the same behavior of NULL JsonNodeType to MISSI…
PR-9281 - KAFKA-10478: Allow duplicated ports in advertised.listeners
PR-9354 - KAFKA-10134 Follow-up: Set the re-join flag in heartbeat failure
PR-9356 - KAFKA-10556: NPE if sasl.mechanism is unrecognized
PR-9309 - KAFKA-10503: MockProducer doesn’t throw ClassCastException when no
PR-9102 - KAFKA-10326 Both serializer and deserializer should be able to see th…
PR-9333 - KAFKA-9274: Revert deprecation of retries for producer and admin clients
PR-9064 - KAFKA-10205: Documentation and handling of non deterministic Topologies
PR-9186 - KAFKA-10277: Allow null keys with non-null mappedKey in KStreamKGlobalTable join
PR-9121 - KAFKA-10351: add tests for IOExceptions for GlobalStateManagerImpl/OffsetCheckpoint
PR-9255 - KAFKA-6585: Consolidate duplicated logic on reset tools
PR-9266 - KAFKA-10469: Resolve logger levels hierarchically
PR-9284 - KAFKA-10479 Throw exception if users try to update configs of existen…
PR-9332 - KAFKA-10511; Ensure monotonic start epoch/offset updates in MockLog
PR-8973 - KAFKA-10218: Stop reading config topic in every subsequent tick if catchup fails once
PR-8181 - KAFKA-9584 Headers ConcurrentModificationException
PR-9317 - KAFKA-10509: Added throttle connection accept rate metric (KIP-612)
PR-9304 - KAFKA-10502:TimestampRouter may occur threadlocal leak
PR-9156 - KAFKA-10077: Filter downstream of state-store results in spurious tombstones
PR-9337 - KAFKA-10519; Add missing unit test for VotedState
PR-9334 - KAFKA-10516; Disable automatic retry of THROTTLING_QUOTA_EXCEEDED errors in the kafka-topics command (KIP-599)
PR-9100 - Add AlterISR RPC and use it for ISR modifications
PR-8295 - KAFKA-9627: Replace ListOffset request/response with automated protocol
PR-9083 - KAFKA-9450: Follow-up; Forbid process after closed
PR-9312 - KAFKA-10505: Fix parsing of generation log string.
PR-9323 - KAFKA-10514: Fix unit test for state directory cleanup
PR-9130 - KAFKA-10492; Core Kafka Raft Implementation (KIP-595)
PR-8892 - KAFKA-10068: verify assignment performance with large cluster
PR-9251 - KAFKA-10459: Document IQ APIs where order does not hold between stores
PR-9202 - KAFKA-10401: Fix the currentStateTimeStamp doesn’t get set correctly
PR-9223 - KAFKA-10438 Lazy initialization of record header to reduce memory usa…
PR-9290 - KAFKA-10487; Fetch response should return diverging epoch and end offset
PR-9275 - KAFKA-10435; Fetch protocol changes for KIP-595
PR-9286 - KAFKA-10292: Set min.insync.replicas to 1 of __consumer_offsets
PR-9274 - KAFKA-10131: Remove use_zk_connection flag
PR-9257 - KAFKA-10463 the necessary utilities in Dockerfile should include git
PR-9272 - KAFKA-10458; Updating controller quota does not work since Token Bucket
PR-9178 - KAFKA-8362: fix the old checkpoint won’t be removed after alter log dir
PR-9239 - Adding reverse iterator usage for sliding windows processing (extending KIP-450)
PR-9264 - KAFKA-5636: Add Sliding Windows documentation
PR-9231 - KAFKA-10447: Migrate tools module to JUnit 5 and mockito
PR-8834 - KAFKA-10134: Enable heartbeat during PrepareRebalance and Depend On State For Poll Timeout
PR-8657 - KAFKA-8334 Make sure the thread which tries to complete delayed reque…
PR-9157 - KAFKA-5636: Update for KIP-450 to handle early records
PR-9221 - KAFKA-10436: Implement KIP-478 Topology changes
PR-9182 - KAFKA-10403 Replace scala collection by java collection in generating…
PR-9219 - KAFKA-10432: LeaderEpochCache is incorrectly recovered for leader epoch 0
PR-9032 - KAFKA-10259: KIP-554 Broker-side SCRAM Config API
PR-9215 - KAFKA-10133: MM2 readme update on config
PR-9122 - KAFKA-10314: KafkaStorageException on reassignment when offline log d…
PR-9191 - KAFKA-10355: Throw error when source topic was deleted
PR-9232 - KAFKA-9924: Add remaining property-based RocksDB metrics as described in KIP-607
PR-9138 - KAFKA-9929: Support backward iterator on WindowStore
PR-9023 - KAFKA-10272: Add IBM i support to “stop” scripts
PR-8798 - KAFKA-10098: Remove unnecessary escaping in regular expression.
PR-9240 - KAFKA-10456: Wrong description in kafka-console-producer.sh help
PR-8955 - KAFKA-10020: Create a new version of a scala Serdes without name clash (KIP-616)
PR-9039 - KAFKA-5636: SlidingWindows (KIP-450)
PR-9177 - KAFKA-9924: Add RocksDB metric num-entries-active-mem-table
PR-9194 - KAFKA-10384: Separate converters from generated messages
PR-9094 - KAFKA-10054: KIP-613, add TRACE-level e2e latency metrics
PR-9112 - KAFKA-10312 Fix error code returned by getPartitionMetadata
PR-9131 - KAFKA-10367: Allow running the Streams demo app with a config file
PR-9136 - KAFKA-10211: Add DirectoryConfigProvider
PR-9137 - KAFKA-9929: Support reverse iterator on KeyValueStore
PR-9146 - KAFKA-10316 Updated Kafka Streams upgrade-guide.html
PR-9028 - KAFKA-10035: Safer conversion of consumer timeout parameters
PR-7222 - KAFKA-8806 Reduce calls to validateOffsetsIfNeeded
PR-9197 - Revert KAFKA-9309: Add the ability to translate Message to JSON
PR-9148 - KAFKA-10379: Implement the KIP-478 StreamBuilder#addGlobalStore()
PR-9189 - KAFKA-10407: Have KafkaLog4jAppender support linger.ms and batch.size
PR-9144 - KAFKA-9705: (part-1) add redirection fields in the request header
PR-9174 - KAFKA-10395: relax output topic check in TTD to work with dynamic routing
PR-9172 - KAFKA-10387: Fix inclusion of transformation configs when topic creation is enabled in Connect
PR-8768 - KAFKA-10023: Enforce broker-wide and per-listener connection creation…
PR-9183 - KAFKA-10404; Use higher poll timeout to avoid rebalance in testCoordinatorFailover
PR-9184 - KAFKA-8033; Wait for NoOffsetForPartitionException in testFetchInvalidOffset
PR-9181 - KAFKA-9516; Increase timeout in testNonBlockingProducer to make it more reliable
PR-9108 - KAFKA-9273: Extract testShouldAutoShutdownOnIncompleteMetadata from S…
PR-9098 - KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording
PR-9163 - KAFKA-10386; Fix flexible version support for records type
PR-9170 - KAFKA-10391: Overwrite checkpoint in task corruption to remove corrupted partitions
PR-8549 - KAFKA-9911: Add new PRODUCER_FENCED error code
PR-9166 - KAFKA-10388; Fix struct conversion logic for tagged structures
PR-8964 - KAFKA-9450: Decouple flushing state from commiting
PR-9004 - KAFKA-10261: Introduce the KIP-478 apis with adapters
PR-9140 - KAFKA-10371; Partition reassignments can result in crashed ReplicaFetcherThreads
PR-9060 - KAFKA-9274: Remove retries from InternalTopicManager
PR-9120 - KAFKA-10316: Consider renaming getter method for Interactive Queries
PR-9114 - KAFKA-10162; Use Token Bucket algorithm for controller mutation quota (KIP-599, Part III)
PR-9047 - KAFKA-9274: Remove retries for global task
PR-9116 - KAFKA-10341: Add 2.6.0 to system tests and streams upgrade tests
PR-9092 - KAFKA-10163; Define controller_mutation_rate as a Double instead of a Long
PR-9008 - KAFKA-9629 Use generated protocol for Fetch API
PR-9097 - KAFKA-10319: Skip unknown offsets when computing sum of changelog offsets
PR-9007 - KAFKA-10120: Deprecate DescribeLogDirsResult.all() and .values()
PR-9095 - KAFKA-10321: fix infinite blocking for global stream thread startup
PR-9081 - KAFKA-10309: KafkaProducer’s sendOffsetsToTransaction should not block infinitively
PR-9012 - KAFKA-10270: A broker to controller channel manager
PR-9089 - KAFKA-10224: Update jersey license from CDDL to EPLv2
PR-9066 - KAFKA-10287: Skip unknown offsets when computing sum of changelog offsets
PR-9086 - FIX: Remove staticmethod tag to be able to use logger of instance
PR-9075 - KAFKA-10306: GlobalThread should fail on InvalidOffsetException
PR-9034 - KAFKA-10246 : AbstractProcessorContext topic() throws NPE
PR-9071 - KAFKA-10305: Print usage when parsing fails for ConsumerPerformance
PR-9061 - MINOR: removed incorrect deprecation annotations
PR-9065 - KAFKA-10301: Do not clear Partition#remoteReplicasMap during partition assignment updates
PR-9051 - KAFKA-10268: dynamic config like “–delete-config log.retention.ms” doesn’t work
PR-8200 - KAFKA-5876: IQ should throw different exceptions for different errors(part 1)
PR-9026 - KAFKA-10274; Consistent timeouts in transactions_test
PR-8968 - KAFKA-10164; Throttle Create Topic, Create Partition and Delete Topic Operations (KIP-599, Part II, Admin Changes)
PR-8933 - KAFKA-10163; Throttle Create Topic, Create Partition and Delete Topic Operations (KIP-599, Part I, Broker Changes)
PR-8864 - KAFKA-9274: Mark retries config as deprecated and add new task.timeout.ms config
PR-9044 - KAFKA-10279; Allow dynamic update of certificates with additional SubjectAltNames
PR-8935 - KAFKA-10189: reset event queue time histogram when queue is empty
PR-9043 - KAFKA-10295: Wait for connector recovery in test_bounce
PR-9040 - KAFKA-10286: Connect system tests should wait for workers to join group
PR-8979 - KAFKA-10223; Use NOT_LEADER_OR_FOLLOWER instead of non-retriable REPLICA_NOT_AVAILABLE for consumers
PR-8948 - KAFKA-10174: Prefer –bootstrap-server for configs command in ducker tests
PR-8239 - KAFKA-9666: Don’t increase transactional epoch when trying to fence if the log append fails
PR-9021 - KAFKA-10257 system test kafkatest.tests.core.security_rolling_upgrade…
PR-8928 - KAFKA-10192: Wait for REST API to become available before testing blocked connectors in integration tests
PR-9003 - KAFKA-10240: Stop throwing WakeupExceptions during sink task shutdown
PR-8672 - KAFKA-10002; Improve performances of StopReplicaRequest with large number of partitions to be deleted
PR-9013 - KAFKA-10044 Deprecate ConsumerConfig#addDeserializerToConfig and Prod…
PR-9009 - KAFKA-6453: Document how timestamps are computed for aggregations and joins
PR-8994 - KAFKA-10247: Correctly reset state when task is corrupted
PR-9010 - KAFKA-10262: Ensure that creating task directory is thread safe
PR-9005 - KAFKA-10263: Do not assign standby for revoking stateless tasks
PR-8996 - KAFKA-10249: don’t try to read un-checkpointed offsets of in-memory stores
PR-8995 - Restore stream-table duality description
PR-8902 - KAFKA-10179: Pass correct changelog topic to state serdes
PR-8974 - KAFKA-10225 Increase default zk session timeout for system tests
PR-8934 - KAFKA-10134: Use long poll if we do not have fetchable partitions
PR-8990 - KAFKA-10243; ConcurrentModificationException while processing connection setup timeouts
PR-8989 - KAFKA-10239: Make GroupInstanceId ignorable in DescribeGroups
PR-8579 - KAFKA-9930: Prevent ReplicaFetcherThread from throwing UnknownTopicOrPartitionException upon topic creation and deletion.
PR-8962 - KAFKA-10166: checkpoint recycled standbys and ignore empty rocksdb base directory
PR-8983 - KAFKA-8398: Prevent NPE in forceUnmap
PR-8479 - KAFKA-9769: Finish operations for leaderEpoch-updated partitions up to point ZK Exception
PR-8944 - KAFKA-10209: Fix connect_rest_test.py after the introduction of new connector configs
PR-8604 - KIP-597: MirrorMaker2 internal topics Formatters
PR-8938 - KAFKA-10173: Use SmokeTest for upgrade system tests
PR-8712 - KAFKA-10006: Don’t create internal topics when LeaderNotAvailableException
PR-7496 - KAFKA-9018: Throw clearer exceptions on serialisation errors
PR-8858 - KAFKA-10153: Error Reporting in Connect Documentation
PR-8929 - KAFKA-4996: Fix findbugs multithreaded correctness warnings for streams
PR-8927 - KAFKA-10200: Fix testability of PAPI with windowed stores
PR-8683 - KAFKA-9893: Configurable TCP connection timeout and improve the initial metadata fetch
PR-8925 - KAFKA-9974: Integration test shouldApplyUpdatesToStandbyStore; Make produce-sync flush
PR-8917 - KAFKA-10180: Fix security_config caching.
PR-8905 - KAFKA-10173: Fix suppress changelog binary schema compatibility
PR-8926 - KAFKA-10166: always write checkpoint before closing an (initialized) task
PR-7577 - KAFKA-9076: support consumer offset sync across clusters in MM 2.0
PR-8924 - KAFKA-10198: guard against recycling dirty state
PR-8900 - KAFKA-10169: swallow non-fatal KafkaException and don’t abort transaction during clean close
PR-8887 - KAFKA-10135: Extract Task#executeAndMaybeSwallow to be a general utility function into TaskManager…
PR-8845 - KAFKA-10126:Add a warning message for ConsumerPerformance
PR-8174 - KAFKA-9439: add KafkaProducer API unit tests
PR-8865 - KAFKA-10168: fix StreamsConfig parameter name variable
PR-8896 - KAFKA-10185: Restoration info logging
PR-8890 - KAFKA-9891: add integration tests for EOS and StandbyTask
PR-8898 - KAFKA-10138: Prefer –bootstrap-server for reassign_partitions command in ducktape tests
PR-8850 - KAFKA-10141: Add more detail to log segment delete messages
PR-8822 - KAFKA-10113; Specify fetch offsets correctly in LogTruncationException
PR-8876 - KAFKA-10167: use the admin client to read end-offset
PR-8841 - KAFKA-10123 Fix incorrect value for AWAIT_RESET#hasPosition
PR-8853 - KAFKA-10147 MockAdminClient#describeConfigs(Collection<ConfigResource…
PR-8882 - KAFKA-10165: Remove Percentiles from e2e metrics
PR-8856 - KAFKA-10150: task state transitions/management and committing cleanup
PR-8880 - KAFKA-10169: Error message when transit to Aborting / AbortableError / FatalError
PR-8872 - Fix log message for transition from standby to active
PR-8873 - Avoid WARN log message when re-init from checkpoint skipped
PR-8849 - KAFKA-10144: clean up corrupted standby tasks before attempting a commit
PR-8857 - KAFKA-10157: Fix broken tests due to InterruptedException from FinalizedFeatureChangeListener
PR-8312 - KAFKA-9432 automated protocol for DescribeConfigs
PR-8764 - KAFKA-10049: Fixed FKJ bug where wrapped serdes are set incorrectly when using default StreamsConfig serdes
PR-8680 - KAFKA-10027: Implement read path for feature versioning system (KIP-584)
PR-8787 - KAFKA-10085: correctly compute lag for optimized source changelogs
PR-8818 - KAFKA-10086: Integration test for ensuring warmups are effective
PR-8069 - KAFKA-9374: Make connector interactions asynchronous
PR-8829 - KAFKA-10115: Incorporate errors.tolerance with the Errant Record Reporter
PR-8828 - KAFKA-9216: Enforce that Connect’s internal topics use compact cleanup policy
PR-8455 - KAFKA-9845: Warn users about using config providers with plugin.path property
PR-8502 - KAFKA-9066: Retain metrics for failed tasks
PR-8847 - KAFKA-7833: Add missing test
PR-8453 - KAFKA-9841: Revoke duplicate connectors and tasks when zombie workers return with an outdated assignment
PR-8663 - KAFKA-9985: Sink connector may exhaust broker when writing in DLQ
PR-8833 - KAFKA-9441: remove prepareClose() to simplify task management
PR-8825 - KAFKA-7833: Add StateStore name conflict check in InternalTopologyBuilder
PR-8630 - KAFKA-9969: Exclude ConnectorClientConfigRequest from class loading isolation
PR-8685 - KAFKA-10014 Always try to close all channels in Selector#close
PR-8775 - KAFKA-10079: improve thread-level stickiness
PR-7384 - KAFKA-8938: Improve allocations during Struct validation in ConnectSchema
PR-8827 - KAFKA-9849: Fix issue with worker.unsync.backoff.ms creating zombie workers when incremental cooperative rebalancing is used
PR-8664 - KAFKA-9716: Clarify meaning of compression rate metrics
PR-8805 - KAFKA-9848: Avoid triggering scheduled rebalance delay when task assignment fails but Connect workers remain in the group
PR-8783 - KAFKA-10063 UnsupportedOperation when querying cleaner metrics after …
PR-8803 - KAFKA-10102: update ProcessorTopology instead of rebuilding it
PR-8676 - KAFKA-10005: Decouple RestoreListener from RestoreCallback
PR-8807 - KAFKA-10106: log time taken to handle LeaderAndIsr request
PR-8684 - KAFKA-10012 Reducing memory overhead associated with strings in Metri…
PR-8270 - KAFKA-9216: Enforce internal config topic settings for Connect workers during startup
PR-8820 - KAFKA-10097: Internalize checkpoint data
PR-8804 - KAFKA-9851: Revoking Connect tasks due to connectivity issues should also clear the running assignment
PR-8486 - KAFKA-9840: Skip End Offset validation when the leader epoch is not reliable
PR-8797 - Fixing KAFKA-10094
PR-8135 - KAFKA-9570: Define SSL configs in all worker config classes, not just distributed
PR-8814 - KAFKA-10111: Make SinkTaskContext.errantRecordReporter() a default method
PR-8811 - KAFKA-10110: Corrected potential NPE when null label value added to KafkaMetricsContext
PR-8776 - KAFKA-9441: Improve Kafka Streams task management
PR-8724 - KAFKA-10040; Make computing the PreferredReplicaImbalanceCount metric more efficient
PR-8759 - KAFKA-10066: TestOutputTopic should pass record headers into deserializers
PR-8755 - KAFKA-10069: Correctly remove user-defined “predicate” and “negate” configs from transformation properties
PR-8311 - KAFKA-9434: automated protocol for alterReplicaLogDirs
PR-8644 - KAFKA-9313: Set use_all_dns_ips as the new default for client.dns.lookup (KIP-602)
PR-8784 - KAFKA-9788: Use distinct names for transaction and group load time se…
PR-8793 - KAFKA-9514; The protocol generator generated useless condition when a field is made nullable and flexible version is used
PR-8792 - KAFKA-10089 The stale ssl engine factory is not closed after reconfigure
PR-8782 - KAFKA-10080; Fix race condition on txn completion which can cause duplicate appends
PR-8785 - KAFKA-10084: Fix EosTestDriver end offset
PR-8786 - KAFKA-10083: fix failed testReassignmentWithRandomSubscriptionsAndChanges tests
PR-8737 - KAFKA-9945: TopicCommand should support –if-exists and –if-not-exists when –bootstrap-server is used
PR-8695 - KAFKA-9320: Enable TLSv1.3 by default (KIP-573)
PR-8777 - KAFKA-10082: Fix the failed testMultiConsumerStickyAssignment
PR-8779 - [MINOR] Fixing spotbug fail - removing unused variable.
PR-8774 - KAFKA-10081: Remove an unused local variable to pass spotbugsMain check
PR-8706 - KAFKA-10030 allow fetching a key from a single partition
PR-8760 - Kafka-10064 Add documentation for KIP-571
PR-8668 - KAFKA-9987: optimize sticky assignment algorithm for same-subscription case
PR-7956 - KAFKA-9392; Clarify deleteAcls javadoc and add test for create/delete timing
PR-8769 - KAFKA-10074: Improve performance of matchingAcls
PR-8767 - KAFKA-10060 GroupMetadataManager should not log if there are no offse…
PR-8761 - Remove redundant code of KafkaProducer
PR-8723 - KIP-569-KAFKA-9494: DescribeConfigsResponse - include additional metadata information
PR-8739 - KAFKA-10056; Ensure consumer metadata contains new topics on subscription change
PR-8705 - KAFKA-10029; Don’t update completedReceives when channels are closed to avoid ConcurrentModificationException
PR-8238 - KAFKA-9130: KIP-518 Allow listing consumer groups per state
PR-8248 - KAFKA-9501: convert between active and standby without closing stores

ksqlDB¶

PR-6800 - fix: LDAP Authentication
PR-6732 - fix: (minor) don’t use deprecated jersey calls
PR-6702 - fix: use Java’s Base64 instead of jersey’s
PR-6668 - fix: fix error categorization on NPE from streams (#6655)
PR-6608 - fix: Check for index before removing value in undo of COLLECT_LIST (#…
PR-6558 - fix: add back configs for setting TLS protocols and cipher suites
PR-6551 - fix: avoid RUN SCRIPT to override CLI session variables/properties
PR-6550 - fix: support joins on key formats with different default serde features
PR-6541 - feat: CASE expressions can now handle 12+ conditions in docker + cloud cherry picking
PR-6537 - fix: ‘-e/-f’ CLI parameters are not handling session variables correctly
PR-6482 - fix: Internal Server Error for /healthcheck endpoint in RBAC-enabled
PR-6467 - fix: check for nested UnspportedVersionException during auth op check
PR-6400 - feat: Add support for ALTER STREAM|TABLE
PR-6437 - fix: Properly clean up state when executing a command fails
PR-6451 - fix: fixes performance regression in pull queries introduced by sessionConfig
PR-6448 - feat: support non-primitive types as keys
PR-6354 - feat: add aggregated rocksdb metrics
PR-6446 - fix: support unwrapped struct value inference
PR-6410 - fix: #6319 default port for CLI: set default port if not mentionned
PR-6411 - feat: enable support for JSON key format
PR-6393 - Join format mismatch
PR-6353 - feat: Add an endpoint for returning the query limit configuration
PR-6405 - feat: avoid spurious tombstones in table output
PR-6397 - fix: recovery hangs when using TERMINATE ALL
PR-6351 - refactor: split repartition node classes
PR-6366 - Topic info cache
PR-6370 - refactor: change QTT to store serialized data in mock Kafka
PR-6365 - Query schemas
PR-6346 - feat: add commandRunnerCheck to healthcheck detail
PR-6348 - fix: backup files are re-created on every restart
PR-6349 - feat: NONE format for key-less streams
PR-6267 - feat: new CLI parameter to execute a command and quit (without CLI interaction)
PR-6344 - feat: support for DELIMITED key format
PR-6292 - Json keys
PR-6149 - feat: Support Comparisons on complex types
PR-6148 - feat: Add metrics for pull query request/response size in bytes
PR-6295 - fix: JSON format to set correct scale of decimals
PR-6278 - fix: don’t use queryId of last terminate command after restore
PR-6253 - feat: add KSQL processing log message on uncaught streams exceptions
PR-6240 - feat: surface error to user when command topic deleted while server running
PR-6166 - bug: Improved error message for reserved keywords
PR-6238 - fix: delimited format should write decimals in a format it can read
PR-6164 - feat: CommandRunner enters degraded state when corruption detected in metastore
PR-6194 - feat: add KEY_FORMAT and FORMAT WITH clause properties
PR-6251 - revert: “fix: improve join predictability by increasing max task idle …
PR-6236 - fix: scale of ROUND() return value
PR-6245 - fix: format cast arguments with passed context
PR-6239 - fix: improve error handling of invalid Avro identifier
PR-6201 - feat: substantially improve avro deserialization performance
PR-6188 - fix: register correct unwrapped schema
PR-6173 - feat: Support IF NOT EXISTS on CREATE TYPE
PR-6182 - fix: fail on non-string MAP keys
PR-6036 - feat: Support for IF EXISTS on CREATE CONNECTOR
PR-6160 - fix: delete zombie consumer groups 🧟
PR-6172 - fix: missing topic classifier now uses MissingSourceTopicException
PR-6163 - fix: allow expressions in flat map
PR-6109 - feat: clarify key or value in (de)serialization processing log messages
PR-6118 - feat: Adds support for 0x, X’…’, x’…’ type hex strings in udf:encode
PR-6135 - fix: wrong value schema being registered for unwrapped primitives
PR-6096 - feat: support PARTITION BY NULL for creating keyless stream
PR-6039 - feat: add Ksql warning to KsqlResource response when CommandRunner degraded
PR-6066 - fix: Use a SandboxedPersistentQueryMetadata to not interact with KafkStreams
PR-6084 - feat: add serialization exceptions to processing logger
PR-6085 - fix: properly handle restore on CREATE OR REPLACE
PR-6067 - feat: Support IF EXISTS keyword on DROP CONNECTOR
PR-6040 - feat: add a serverStatus to ServerInfo and display the status in the CLI
PR-5983 - fix: Uses pull query metrics for all paths, not just /query
PR-6068 - fix: dont cleanup external resources on CREATE OR REPLACE
PR-6054 - Single value wrapping
PR-5998 - feat: Support subscript and nested functions in grouping queries
PR-6032 - feat: CommandRunner enters degraded states when it processes command with higher version than it supports
PR-6061 - feat: hard delete schemas for push queries
PR-6031 - feat: DistributingExecutor fails DDL statement if CommandRunner DEGRADED
PR-6012 - feat: move command topic deserialization to CommandRunner and introduce DEGRADED CommandRunnerStatus
PR-6048 - fix: remove unnecessary parser token (MINOR)
PR-6005 - fix: allow implicit cast of numbers literals to decimals on insert/select
PR-6033 - fix: protobuf format does not support unwrapping
PR-5944 - feat(client): support describe source in Java client
PR-6018 - fix: set restarted query as healthy during a time threshold
PR-5984 - refactor: Call the correct pull/push query validator
PR-5982 - feat: support UDAFs with and without init Args with same param type
PR-5979 - fix: replace kafka-streams deprecated code
PR-5960 - fix: NPE when udf metrics enabled
PR-5969 - feat: add config for suppress buffer
PR-5962 - feat: Support [IF EXISTS] on DROP TYPE command
PR-5476 - feat: Add consumer offsets to DESCRIBE EXTENDED
PR-5877 - feat: New ksql.properties.overrides.denylist to deny clients configs overrides
PR-5884 - feat: add suppress functionality
PR-5807 - feat: add service to restart failed persistent queries
PR-5594 - fix: improve join predictability by increasing max task idle
PR-5883 - fix: close query on invalid use of HTTP/2 with /query endpoint
PR-5874 - fix: change default grace period to zero
PR-5879 - fix: create the metastore backups directory if it does not exist
PR-5849 - feat: Enable datagen to set the message timestamp
PR-5779 - feat: add suppress node
PR-5843 - feat: new array_remove udf
PR-5831 - feat: Replay command topic to local file to backup KSQL Metastore
PR-5835 - fix: configure topic retention based on retention clause for windowed tables
PR-5823 - fix: always use the changelog subject in table state stores
PR-5816 - feat(client): return query ID when persistent query started via Java client
PR-5814 - feat: expose query ID in CommandStatusEntity (MINOR)
PR-5775 - feat(client): support DDL/DML statements in Java client
PR-5813 - feat: organize UDFs by category
PR-5818 - fix: Allow backported ALPN to be used and warn if no OpenSSL
PR-5769 - fix: ensure null values cast to varchar/string remain null
PR-5743 - fix: windowed tables now have cleanup policy compact+delete
PR-5781 - fix: circumvent KAFKA-10179 by forcing changelog topics for tables
PR-5744 - fix: Make sure UDTF describe shows actual function description
PR-5739 - feat: add syntax for suppression
PR-5766 - feat: support CREATE OR REPLACE w/ config guard but w/o restrictions
PR-5742 - fix: Reuse KsqlClient instance for inter node requests
PR-5763 - fix: ksqlDB should not truncate decimals
PR-5750 - fix: show overridden props in CLI
PR-5679 - refactor: Convert remaining Kudfs
PR-5730 - fix: make sure old query stream doesn’t block on close
PR-5641 - feat(client): support streaming inserts in Java client
PR-5682 - feat(client): support list queries in Java client
PR-5680 - feat: support WINDOWEND in WHERE of pull queries
PR-5671 - feat(client): support admin operations in Java client
PR-5672 - fix: simplify pull query error message
PR-5644 - fix: support GROUP BY with no source columns used
PR-5645 - fix: ensure only deserializable cmds are written to command topic
PR-5482 - feat: Adds SSL mutual auth support to intra-cluster requests
PR-5474 - feat: implements ARRAY_JOIN as requested in (#5028)
PR-5563 - feat: new split_to_map udf
PR-5559 - feat: add CHR UDF
PR-5597 - fix: fail AVRO/Protobuf/JSON Schema statements if SR is missing
PR-5573 - Stream with no key should not fail if key has data

REST Proxy¶

PR-773 - KREST-287: Syncronize resource pools/managers lazy instantiation.
PR-756 - KREST-26: Add a blocklist config to allow disabling individual endpoints.
PR-761 - Add toBuilder method to Cluster
PR-755 - Update JUnit dependency to JUnit 5.
PR-216 - MMA-7520: add CsrfTokenProtectionFilter
PR-209 - (backport) SEC-592: Customize Jetty request queue and threadpool size (#172)
PR-215 - ff-3928: bump version of Guava in parent of schema-registry.
PR-210 - ST-4542: Upgrade jetty to address CVE-2020-27216
PR-205 - CIAM-261: Add Jetty ThreadPool Metrics
PR-194 - Fix build failure

Schema Registry¶

PR-1747 - DG-1189 Add latest.compatibility.strict flag
PR-1746 - DG-931: throw ConfigException in Avro/Json/Protobuf Converter
PR-1724 - DG-1124 NPE in ProtobufMessageFormatter
PR-1722 - DG-1122 Treat reordering Protobuf msgs as incompatible
PR-1711 - add testUsingSubjectNameStrategy()
PR-1709 - QEC-2994: fix bug in getting subject name for value
PR-1700 - DG-293: Improve exception messages in JsonSchemaConverter and ProtobufConverter
PR-1699 - DG-1040 Register ids for soft deletes
PR-1688 - DG-1007 Preserve refs when converting schema to envelope
PR-1687 - DG-924 Disable default validation by default
PR-1686 - MMA-8608 Handle duplicate configs after stripping prefix
PR-1684 - MMA-8608: parse configs before calling restService.configure()
PR-1678 - DG-990 Allow SpecificAvroSerde to work with array/map
PR-1676 - DG-972 Fix name collisions with field and message
PR-1675 - DG-962 Handle const schemas in JSON Schema converter
PR-1668 - Removed property ${confluent.maven.repo}
PR-1658 - DG-951 Handle duplicate message names during deserialization
PR-1654 - Adding an NPE check for permanent delete
PR-1648 - DG-935 Treat removing Protobuf message types as incompatible
PR-1642 - Avro Console producer should support subject name strategy option
PR-1647 - DG-933 Resend old value when handling invalid key-value
PR-1638 - DG-924 Enable Avro default validation for new schemas
PR-1634 - DG-921 Fix Avro specific reader for unions
PR-1625 - DG-410 Fix doc/defaults in Avro converter
PR-1614 - Add failing compatibility reason in rest response
PR-1621 - DG-848 Enable mode mutability by default, fixes #1610
PR-1591 - Add offset checkpoints for persistent stores
PR-1571 - Add setTenant
PR-1559 - Add API to get subjects by predicate
PR-1544 - DG-616 Permanent Delete support in SR client
PR-1533 - DG-534 Enhance compat endpoint to validate schema
PR-1532 - DG-540 Add batch get schema API
PR-1527 - NetworkClient fix for upstream Kafka changes
PR-1519 - #1515 - Cache Avro DatumReader and DatumWriter instances to increase performance
PR-1498 - update README to include protobuf, json schema
PR-1491 - Revert “Add node information to SchemaRegistryProtocol (#1485)”

Commercial Features¶

Confluent Server¶

KSTORAGE-1348 Flush epoch cache before reinitialization in Log.renameDir
SEC-1805: Reduce failed MDS request send log threshold from ERROR to Debug
Fix test failure in test_kibosh_service (CPKAFKA-6180)
SEC-1802 Check for topic creation
SEC-1802 Don’t block TopicManager creation on describeTopics
SEC-1770 NonBlockingKafkaExporter should notice that topics exist
KSTORAGE-1178: Moved missing tier state file recovery code under assignTopicId (#2702)
KSQL-5843: prevent log spam from MDS RestClient on client close
Add back specific log message for unclean recovery test (#2938)
SEC-1789 Bump Bouncy Castle dep to resolve CVE-2020-28052
AUDIT-249– Added mime type “application/protobuf” supporting to BinaryDeserializer
CPKAFKA-6413: test_offset_migration_single_group_offset_not_exists_on_dest
CPKAFKA-6375: Decode python3 bytes object in tier fence restore test
CPKAFKA-6374: Handle None in tier roundtrip system test
CPKAFKA-6370: Use python3 range (versus xrange) to have BrokerStorageFailureTest work with python3
CPKAFKA-6371: Use python3 sys.maxsize in tiered storage system tests
KC-1275: Update KafkaReader to handle entries from multiple partitions
CPKAFKA-6373: Fix deprecated has_key python2 method
CPKAFKA-6376: Use range over xrange to fix TierMetadataCorruptionTest to work with python3
KSTORAGE-1290: Bump google storage client dependency to 1.113.4
CPKAFKA-6372: Fix bad assertion in cluster link test
CPKAFKA-3011: fix timing issue with onConfigChange and ConfluentLeaderAndIsrRequest
Backport of CPKAFKA-6268
CNK-1431: Add confluent.ansible.managed property
KC-1228 cherry pick for 6.1.x
CPKAFKA-6018: Fix cluster link test timing issue with metrics and replica status
KSTORAGE-1260: tiered segments do not correctly block retention for local segments (#2831)
CNK-1325: Add a metric for initiated executions.
CNK-819: Don’t self-heal if a rebalance doesn’t improve detection goals.
Port fix for KSTORAGE-1230 to 6.1.x
CPKAFKA-6126: Update commons codec (#2785)
CNK-1210: Introduce kafka-add-brokers CLI
CPKAFKA-6233: Update lastUpdateTimeMs before updating the current broker addition state
CNK-1396: Register BrokerAddCount metric in CDBE
CNK-617: Temporary pull out CloudNetworkFault + deps
CNK-1349: Switch broker removal state from PLAN_EXECUTION_INITIATED to PLAN_COMPUTATION_INITIATED on failover
KC-1226: Enable confluent.cluster.link.io.max.bytes.per.second to be configured at cluster-level
KC-1225: Fix NPE on broker restart with paused links, fix blocked state
CNK-1228: Update SBK add_broker system tests to utilize kafka-add-brokers for checking operation completion
KC-1027 backport into 6.1.x
Backport Update system test to use –comparator.json (#2743)
CNK-1332: Handle TopicExistsException when creating SBK topics
KC-1203: Stop mirroring if destination partition is under-min-isr, metrics changes
KC-1195: Ensure connections are closed if principal cannot be built
backport METRICS-2459: update default include list for hosted monitoring
backport METRICS-2396: Add extra checks to ensure that we do not leak EventExporter objects
backport METRICS-2110: document Telemetry Reporter packaging (#2710)
backport METRICS-1805: remove duplicate resource type label (#2744, #2755)
Add unit tests to disable tier storage at broker level
KSTORAGE-1180 Update tier enable flag at FileTierPartitionState unconditionally
KC-1183: Fix exception in writer metric for non-MDS brokers
KC-1025 automatic observer promotion/demotion in MRC (#2649) cherry-pick
CNK-1304: Update remove broker validation to use strict collections
CNK-1331: Fix typo in SBK Persistence Store initialization
CNK-1209: Describe AddBroker functionality and integration tests
KC-1132: Add cluster link request metrics for source cluster
CPKAFKA-6030: Fix broker removal system test
CNK-1290: Update the sub task status field in BrokerAdditionDescription and BrokerRemovalDescription and CNK-1288 - add description() method to DescribeBrokerRemovalsResult for feature parity
CNK-1233: Adds support for logical cluster type link-healthcheck.
CNK-1208: Return a high level balancer operation status in describe broker removals
KSTORAGE-1075: Add debug logging when PendingFetch completes
[METRICS-2356] patch version for thread leak fix
Increase test tolerance in non deterministic test
CC-12141: Confluent’s WorkerConfigDecorator in Connect should handle connector validate responses with extra properties
KSTORAGE-1004: Restore tool should be able to work standalone without comparator tool
KC-1149: Ensure stopped mirrors are not treated as read-only partitions
CNK-1012: Add broker state machine
CNKAF-1236: Add startTimeMs and lastUpdateTimeMs to the describe broker removal RPC
KC-1148: Fixing metric conflict issues where link can’t be recreated due to metric conflict issues
Add config option to override schema validator interceptor class
SEC-1477 Add trace logging on LDAP search success
CCS -> CE sync upstream 28 sept 2020
CONFLUENT: Add config for the tests that rely on MockAuditLogProvider.
KSTORAGE-1176: chmod -R ag+w in ccloud startup script increases startup time
CPKAFKA-4479: Make operationally important Kafka configs dynamic (fetch.max.bytes)
KC-907: Update to cc-base 7.4.0 and remove prometheus and async profiler downloads
CNK-1126: Request quota auto-tuning sorts by an unstable value
CNK-374: Return task history for only configured retention period
ce-metrics: add space to reconcileConfigs() log message
KC-375: Include cluster-link-id in requests from destination brokers to source
METRICS-2358 : only initialize event logger when enabled, properly close it, and add better support for reconfiguration
CNKAF-1134: Close AdminClient, Producers and Consumers immediately
KC-1120: Ensure that RBAC metadata write failures are reset after writer elections
CNKAF-856: DescribeBrokerAdditions API - RPC and Admin API skeleton
KC-938: Add config for dedicated-cluster topics for cluster-linking soak setup.
KC-955: Adds describeMirrors API.
CNKAF-1068: Add validation for broker removal for rf equal to alive brokers
CNKAF-1235: Revise state machine for broker removal
CIAM-349 CloudClusterAdmin can see Schema Registry
KSTORAGE-1100: Offset cache can’t be disabled although we have the facility to do so
KSTORAGE-986: Check for invalid topic partitions within the recovery tool
CNK-1206: Update dependencies for SBK serializers
CNK-42: Only enable Databalancer Count metric if balancer is enabled
CNKAF-1109: Use AdminClient for Leader Elections, handle exceptions
KSTORAGE-1160: avoid unnecessary maxCompactionDelay calculations when now < max.compaction.lag.ms
KC-899: Monitor auto-topic creation and compaction
Propagate CC release branch to ce-kafka-images
CNKAF-1001: Retry plan computation in Broker Removal
KSTORAGE-1150: Allow customers to change log.cleaner.max.compaction.lag.ms cluster default
CIAM-371 Add Delete operation to Cluster for CloudClusterAdmin
CNKAF-1174: improve fetch session cache performance
KC-1017: Extends replica status to include mirror information.
KC-44: Maintain topics per tenant in metadata cache
KC-1021: Add RequestConvertToJson tests
KSTORAGE-797: allow recovery path materializer to use secured listeners
CC-11832: add connector name to metrics context when calling Kafka Cl…
CNKAF-1062: Add validation for broker removal leading to under min isr
CNKAF-1224: Log all uncaught SBK exception and errors
CNKAF-920: Raise invalid broker removal exception with specific error message
KC-1019: ScramUserCredentials matches missing in RequestConvertToJson
[CNKAF-1195] Avoid stale cluster metadata for AnomalyDetection plans.
CIAM-321 Add Audit Resource Type for OrgAdmin role
Configure ListenerConnectionQuota on broker startup
Suspending rbac system tests till we fix hang issue
local start offset check is done outside committed offset check block.
KSTORAGE-785: allow CCloud users to set max.compaction.lag.ms
CNKAF-915 Improve log level in executor
Durability Audit fixes
KSTORAGE-1013: add broker internal log dir storage probe
CNKAF-1075: Improve Confluent SASL secrets caching
KC-190: add metric to count responding to disconnected clients
cherry-pick from AK - Fix that support, resolve of “kafka.apache.org ” to 3 IP addresses (#9294)
Reverted changes in TestUtils which added additional parameter for enabling durability audit.
Fixed unit test in AuditManager
KSTORAGE-1023: Durability audit follow up
CNKAF-765; Update the TenantQuotaCallback to support Controller Mutation Quota
CPKAFKA-6013: Skip tier state presence validation for deleted partitions
KSTORAGE-972: Modify TopicIdPartition.toString() implementation to use ‘:’ and ‘-‘ as delimiters
KSTORAGE-885: add lagging and realtime consumer to retention ms/bytes system test
KC-883: Update soak-cluster configuration for source and destination.
Migrated AuditJob away from KafkaScheduler
KSTORAGE-973: Check for invalid topic partitions within the fencing tool
KSTORAGE-724 Have TierTasks writing to online tier partitions make progress when few tier partitions are offline
KSTORAGE-725: Exclude segments where the high offset is lower than the lowerBoundOffset for tiering.
KSTORAGE-1074: retry should log at INFO level
KSTORAGE-1082: tier replication causes incorrect HWM after restore (#2507)
KSTORAGE-1075: Add more debug info for consumption tracking
feat: Add load metrics for logical cluster metadata
KSTORAGE-1086: Better validations to check if tiering is disabled
CNKAF-1138: Emit JSONs with auto-generated schema
CNKAF-546: Reinitialize expired partition detail sensors
KSTORAGE-909: Follow-up PR to durability audit PR
KSTORAGE-1054: add another Azure auth method via config file
KC–603: Fix NPE in KafkaBasedLog during broker shutdown if license topic creation fails
KC-902: Fix cluster link authorization, add end-to-end authorization tests
KSTORAGE-970: Fix help screen in TierTopic tooling
KSTORAGE-1084: use file name convention for tier_unclean_leader_election_test.py
CPKAFKA-5114: Add group size metrics
Sync upstream august 18 2020
KSTORAGE-1082: tier replication causes incorrect HWM after restore
CNKAF-1169: Correctly handle single racks and no racks when creating topics in the MT interceptor
CPKAFKA-5114: Add group size metrics
Update mk-include
KSTORAGE-887: Integrate TierFetcherBenchmark into compatibility checker
KSTORAGE-887: Make TierFetcherBenchmark configurable for Pure Storage testing
CNKAF-1087: Fix rack aware topic placement for cloud use case
KC-854: Allow socket buffer size configurable for high latency cluster links on Cloud
KC-801: Reassignment succeeds with empty constraints
[CNKAF-1144] Confluent.operator.managed property (#2484)
[Backport]METRICS-2089 Adding the correct NOTICE / COPYRIGHT for the confluent-telemetry package
KSTORAGE-1042: Harden compatibility framework #1
METRICS-2089 Adding the correct NOTICE / COPYRIGHT for the confluent-telemetry package
KSTORAGE-1067: Dump fetch context when tiered storage runs into an unexpected error
[cnkaf-1144] Confluent.operator.managed property
Durability Audit - Audit Manager, Materialization and Reporting and Event generation
CNKAF-902: Add metrics to track SSL handshake failures and avg/max time
switch to using DefaultAzureCredentials for Azure authentication
CNKAF-108: Improve log message for dynamic certificate update
KC-166: Add metric for controller loading time
KSTORAGE-974: Print tiertopic partition and timestamp for fencing tool
KSTORAGE-859: MergedLog.createUploadableSegment deleted segment check
CNKAF-758:Add cpu load metric
KSTORAGE-1040: Better checks to prevent tiering from being disabled dynamically
METRICS-2159: use a loading cache to avoid expensive pattern matching calls
METRICS-2160: optimize local exporter regex
METRICS-2159: use a loading cache to avoid expensive pattern matching calls
KAFKA-10439: Connect’s Values to parse BigInteger as Decimal with zero scale.
avoid calls to gettimeofday on every metric key in telemetry
KC-102 Add cc-zookeeper docker build
CPKAFKA-5882. Add copyDependantLibs dependency to ce-sbk and ce-broker-plugins module
[CNKAF-1112 WORKAROUND] Do not allow enabling of strict quotas.
feat: Cache auth failures
SEC-1016: Changes to NonBlockingKafkaExporter
CONFLUENT: Upgrade netty-tcnative to 2.0.34.Final
CONFLUENT: Change visibility of ConfluentProvider.createRbacAuthorizer() to public
KC-845: Ensure LogConfig includes broker defaults to pass schema config validation
CPKAFKA-5873 Don’t load Netty classes when using default SSL engine
CONFLUENT: Remove shading from ce-broker-plugins and ce-sbk
[KSTORAGE-829] Storage Fault Injection: Integrate Kibosh with Ducktape
CPKAFKA-5851: Align joda-time version with common
KSTORAGE-1000: TierDeletedPartitionsCoordinator Handling of Emigration During Read
CNKAF-863: Only showed broker removal status for given broker.
KC-613: Use kafka-rest-lib instead of kafka-rest on classpath.
KC-613: Use kafka-rest-lib instead of kafka-rest on classpath.
Update compatibility checker README.md
SEC-1373 EmbeddedKafkaCluster sets Audit Log replication factor to 1
CPKAFKA-5849: Don’t log password configs in request log filter
CONFLUENT: Remove unused jacksonDatabind version definition
CPKAFKA-5795: Change guava dependency to 28.1-jre
CPKAFKA-5796: Remove ZK dependency from ce-licensing
CPKAFKA-5835: Fix regression in cluster link compatibility test
Add secrets to default RBAC policies
KC-810: Add cluster link destination tenant metrics suitable for billing
METRICS-2108 Telemetry-client dependency version update from 1.174.0 to 1.192.0 to update default base URL for TelemetryHttpClient
KC-770: Support backpressure for cluster link when disk usage is high
METRICS-2108 Telemetry-client dependency version update from 1.174.0 to 1.192.0 to update default base URL for TelemetryHttpClient
KSTORAGE-996 Fix logic to check bytes fetched from local log
[CNKAF-1089][RM-32] Remove SBK dependencies on grpc-stub. (#2408)
KSTORAGE-991 Implementing Additional Config Validations
KSTORAGE-334, KSTORAGE-893, KSTORAGE-894: DeleteRecords, retention time and bytes system tests
backport METRICS-2098 exclude zstd/lz4 from telemetry-clients to avoid conflicting versions
KSTORAGE-994 Improved Configuration Documentation
[CNKAF-1089][RM-32] Remove SBK dependencies on grpc-stub.
Redo SEC-866
CNKAF-655: SBK Soak cluster workload
METRICS-2090: Remove duplicate labels on metrics
METRICS-2097: Don’t capture ‘total_size` and ‘tier_size` metrics
METRICS-2098 exclude zstd / lz4 from telemetry-clients to avoid conflicting versions
KC-637 cherry pick to 6.0.x
KC-637 Offset migration system tests.
KSTORAGE-957: Only set SSEAwsKeyManagementParams if a customer encryption key is supplied and sseAlgorithm is aws:kms. (#2353)
KSTORAGE-886 [MINOR] Move scripts from bin to compatibility checker directory
SEC-866-6.0.x again
KSTORAGE-886: Implementing basic tiering load test utils
CONFLUENT: Remove the soak clients status reporter cronjob
KSTORAGE-957: Only set SSEAwsKeyManagementParams if a customer encryption key is supplied and sseAlgorithm is aws:kms.
KSTORAGE-506: Prevent accidental change to critical configurations for tier topic
CIAM-227: Add requestor principal to AuthWriter interface methods
METRICS-2031 Prune the hardcoded Kafka broker default include list to avoid collecting high cardinality metrics.
METRICS-2048 Sanitize ksql service ids from their metric name to make them more practical to use. Also ignore legacy metrics.
[CNKAF-1038] SBK Rebalance Cluster Test: add rolling bounces (#2326)
METRICS-2031 Prune the hardcoded Kafka broker default include list to avoid collecting high cardinality metrics.
CNKAF-1058; Ensure that Control Request Batching does not send LeaderAndIsrRequest with yet unknown broker(s)
CNKAF-896: Avoid expensive calls to MetadataCache#getNonExistingTopics
KSTORAGE-947: batch tier segment deletion delays to improve deletion SLA (#2309)
METRICS-2048 Sanitize ksql service ids from their metric name to make them more practical to use. Also ignore legacy metrics.
KC-238: add max rebalance time gauge
CNKAF-1028:Passed SNI host name to OAuthBearerValidatorCallbackHandler
CNKAF-1039: Update error for insufficient metrics to compute plan
backport METRICS-2010: create a new ‘releaseTarGz’ task for ce-metrics (#2282)
METRICS-2010: create a new ‘releaseTarGz’ task for ce-metrics
CPKAFKA-2864: Update base docker image for new tooling
CONFLUENT: Allow both AK and Confluent class for system test process
CPKAFKA-5762: Fix topic deletion in system tests
CNKAF-1039: Update error for insufficient metrics to compute plan
KC-671: SBK system test with CL
METRICS-1988: add confluent-telemetry directory to kafka classpath
CONFLUENT: Bump cc-trogdor chart docker image to cc-trogdor:v0.207.0-5.3.0-ce-SNAPSHOT
KAFKA-10371; Partition reassignments can result in crashed ReplicaFetcherThreads
[Cherry 2367] [SEC-1345] Read HTTP Reason phrase when connection closed
CC-4539: Use plugin.path mechanism to load rest servlet initializers in Connect
Revert “SEC-866 - Migrate audit log “other” category into “management” category. (#2330)”
[SEC-1345] Read HTTP Reason phrase when connection closed
backport METRICS-2037 remove non-relocated dependencies from confluent-metrics fat jar (#2332)
[CLIENTS-1633] Schedule MDS url refresher asynchronously
Increase root FS to 20GB
METRICS-1763, METRICS-1767 Configure telemetry metrics whitelist for Connect and MQTT Proxy Proactive Support (#2241)
SEC-866 - Migrate audit log “other” category into “management” category.
METRICS-1763,METRICS-1767 Configure telemetry metrics include list for Connect and MQTT Proxy Proactive Support
SEC-866 - Migrate audit log “other” category into “management” category.
KC-391: Source topic deletion system test for CL
METRICS-2037 remove non-relocated dependencies from confluent-metrics fat jar
KSTORAGE-947: batch tier segment deletion delays to improve deletion SLA
METRICS-2056 Update api.secret datatype from String to password in event’s HttpExporterConfig.
KC-727: Prevent source topic config mirroring from violating alter config policies
[CLIENTS-1633] Authorizer RestClient should fail updating active node…
KAFKA-10282; Remove Log metrics immediately when deleting log (#2342)
[CNKAF-1038] SBK Rebalance Cluster Test: add rolling bounces
CNKAF-1036: Fix issue where dynamic throttle applies for the current rebalance only - cherry pick to 6.0
Rebalance Cluster System Test
KAFKA-10282: Remove Log metrics immediately when deleting log
CNKAF-1008: Add Broker/Remove Broker proposal logging
CNKAF-1036: Fix issue where dynamic throttle applies for the current rebalance only
KC-633: Auth failures system test for Cluster Linking
Add CloudNetworkFault to Trogdor
[CHERRY-2308] Move kafka-http-server project into clients project
METRICS-2007 Updating Telemetry Reporter to exclude exporter configs or disabled exporters with no type set.
Move kafka-http-server project into clients project
KC-638: System tests with tier storage enabled on destination
KC-754: Cluster link compatibility system tests
KC-631 Good path cluster linking test using SASL_SSL
backport METRICS-1992: Fix schema registry cluster id resource label (#2285)
SBK: System test for cluster rebalancing
METRICS-1760 Configuring additional Kafka Broker include list metrics for Proactive Support.
KC-749: initial single-topic setup for cluster-linking soak clusters
KC-755: Fix NoSuchElementException warnings for consumer groups with no new offsets
CONFLUENT: Fix broken cluster link system tests in 6.0.x
CNKAF-481: Oauth validation of SNI HostName
METRICS-1950 Adding integration test to verify proxy configurations are propagated correctly.
CNKAF-481: Passed down SNI hostname and enable parsing in transport layer.
CNKAF-1008: Log full proposal dump for add/remove
METRICS-1992: Follow up comment based on review
CNKAF-1029: Ensure Executor state is reset after unexpected exceptions
KC-735: Improve cluster link addition.
KSTORAGE 806: Tiering Functionality Tests
KC-726: Cluster link config support for CCloud.
CNKAF-1035: Roll cluster while a new broker is getting added
clean up auto aborting input stream
KSTORAGE-888: Add tier fetches metrics for tiered storage
METRICS-1998: add rbac metrics to default metrics list (#2269)
METRICS-1992: Fix schema registry cluster id resource label
backport METRICS-1993: add connect.cluster.id label (#2264)
Updated default topic replicas factor to 3 for audit log.
KC-741: Don’t include passwords in cluster link request logs
KC-736: Allow deletion of cluster links if mirrored topics are marked for deletion
KSTORAGE-896: fix tier deletion manager system test (#2256)
CNKAF-822: Cherry pick add broker system test to 6.0.x
KC-725: Backport of KC-725 to 6.0.x
METRICS-1993: add connect.cluster.id to telemetry resource labels
CNKAF-822: Adds an SBK AddBroker system test
METRICS-1998: add rbac metrics to default metrics list
KC-725: Fixed issue where metrics were not cleaned up when cluster links were deleted
KSTORAGE-823: Log.maybeIncrementLogStartOffset should not flush leader epoch cache
backport METRICS-1986 Fix to give higher precedence to “confluent.telemetry.metrics.collector.whitelist” over default provider whitelist. (#2266)
backport METRICS-1985 Add missing metrics to the _local exporter for SBK feature to work as expected.
CONFLUENT: Make RbacClusters.java to work with Writer/AuthWriter interfaces
METRICS-1985 Add missing metrics to the _local exporter for SBK feature to work as expected.
KC-728: Return INVALID_CONFIG error for invalid link configs, log unexpected errors
METRICS-1986 Fix to give higher precedence to “confluent.telemetry.metrics.collector.whitelist” over default provider whitelist.
KSTORAGE-920: TierTopicConsumer primary consumer long poll throttles catchup (#2253)
KSTORAGE-864 Fetch requests to return NotLeaderForPartition during ULE recovery
CNKAF-819: Test SBK broker removal with roll
METRICS-1762 Configure telemetry metrics whitelist for KSQL Proactive Support
KSTORAGE-896: fix tier deletion manager system test
KSTORAGE-920: TierTopicConsumer primary consumer long poll throttles catchup
CIAM-12: update mk-include
METRICS-1762 Configure telemetry metrics whitelist for KSQL Proactive Support
Backporting fix for METRICS-1764 and METRICS-1766
update kafkatest template for balancer replication factor (for < 3-node clusters)
CNKAF-1015: Fix issue in SBK startup by moving CruiseControl instantiation after the startup components
KSTORAGE-876: Adding Tiered Storage TotalSizeAggregate Metric
KC-638: Cluster linking system test with tiered storage enabled in source cluster
KSTORAGE-890: Initial commit with the AuditJob infrastructure
CONFLUENT: Fix reassign partitions system tests
METRICS-1764 Configure telemetry metrics whitelist for Schema Registry Proactive Support
METRICS-1766 Configure telemetry metrics whitelist for Rest proxy Proactive Support
CNKAF-825: Introduce a system test for replica placement constraints
CNKAF-1014: Rename usages of SBK in logs to DataBalancer
CNKAF-613: Introduce a broker removal system test
METRICS-1894:Telemetry Reporter is missing labels configured in prope…
METRICS-1817: add include regex default value to Provider interface
METRICS-1938: upgrade jose4j version (#2219)–backport
METRICS-1929: use advertised listeners/host-port when deriving local producer config
METRICS-1938: upgrade jose4j version
[CNKAF-963] Switch AddBroker plan computation to wait METADATA_TTL.
CNKAF-954: Removes GoalOptimizer proposal caching
KSTORAGE-892 Re add couple of lines of ULE code that got deleted duri…
Backporting config/events Telemetry Reporter changes along with all shading issues.
Added one more toJson method in protoserde for webhook alert
(backport) MMA-8432: fix wrong package name
MMA-8432: fix wrong package name
KC-428: Always log authorize results to file based on log4j settings
KAFKA-10279; Allow dynamic update of certificates with additional SubjectAltNames
CNKAF-1004: Don’t describe removed broker configs
METRICS-1894:Telemetry Reporter is missing labels configured in properties file
KC-714: SslTransportLayer write should invoke SSLEngine.wrap once with all the buffers
CNKAF-1004: Don’t describe removed broker configs
CNKAF-762: Abstract CruiseControl startup and config translation logic into own package
METRICS-1895: Update telemetry client version
KC-705: Retry metadata topic consumer.position() until authorizer startup times out
METRICS-1918: bump gson from 2.8.5 -> 2.8.6
METRICS-1895: upgrade telemetry client version
(backport) prevent ‘io.confluent.controlcenter’ from being renamed by shadow plugin
KSTORAGE-526 Recover from corrupt or missing TierState file during bootup
KSTORAGE-822: Expose Tiered Storage configs for GA in 6.0
KSTORAGE-822: Expose Tiered Storage configs for GA in 6.0
(Backport) include UnderMinIsrPartitionCount in the default broker metrics list
Add UnderMinIsrPartitionCount to default broker metrics list
CNKAF-481: Parse SNI in sslTransportLayer
CPKAFKA-2852: CCloud metadata listener thread should not die on unexpected exception
METRICS-1875 fix shading issues with cloudevent proto generated classes
SEC-1276 Allow empty resourceNames in CRNs
MINOR: Improve AuthenticationException logging
SEC-384: Use the correct log level for debug output
KC-634: Cluster link system tests with broker failures
CIAM-110 Associate RBAC Roles with organizations
METRICS-1875: Shade generated ce-events protobuf
prevent ‘io.confluent.controlcenter’ from being renamed by shadow plugin
CNKAF-985: Bump leader distribution goal priority
KSTORAGE-500 Remove TierTopicAppender.becomeArchiver from the interface and inline the implementation
KC-614: Check that topic name is not empty before tenant prefixing
CNKAF-964: Shade opencensus protobufs into SBK jar (#2155)
CONFLUENT: Update log statements in alterBrokerConfigs/alterTopicConfigs methods
SBK: cherry pick javadoc updates for removeBroker
METRICS-1878: Shade telemetry client classes
Add KafkaRest Metrics Provider
KSTORAGE-845 Check if tier topic exists already before calling AdminZkClient#createTopic
CNKAF-952: Update javadoc for removeBroker ConfluentAdmin API with InvalidBrokerRemovalException
CNKAF-851: log goals skipped during detection in one line
Add KafkaRest Metrics Provider
CNKAF-964: Shade opencensus protobufs into SBK jar
KSTORAGE-806 Modifying Correctness Framework to accept .properties
KC-630: Cluster link system test support with good path mirror test
CONFLUENT: Prevent log start offset overrun during retention
METRICS-1862 Remove config events proto defition from ce-kafka and use the one checked in into Schroedinger instead.
KSTORAGE-861: fatal NPE in TierTopicConsumer thread caused by initialization order (#2133)
Back port [METRICS-1815] Connect client metrics missing some resource level tag…
KSTORAGE-806 Patch to add Confluent License Information
KAFKA-10239: Make GroupInstanceId ignorable in DescribeGroups (#8989)
CPKAFKA-5456: Shade telemetry-api classes
KSTORAGE-806: Implementing a CLI for compatibility correctness tests & building object store correctness tests
CNKAF-936: Remove dot-handling logic in metrics processor (#2121)
KSTORAGE-861: fatal NPE in TierTopicConsumer thread caused by initialization order
METRICS-1833 Special handling of KSQL metrics in telemetry reporter that are getting mangled.
[METRICS-1815] Connect client metrics missing some resource level tags
METRICS-1833 Special handling of KSQL metrics in telemetry reporter that are getting mangled.
CNKAF-936: Remove dot-handling logic in metrics processor
Azure block blob implementation for tiered storage
Deprecate KafkaHttpServlet in favour of rest-utils Application. (#2135)
CLIENTS-1599: Deprecate KafkaHttpServlet in favour of rest-utils Application.
KC-532: Support observers in replica assignment tool
CONFLUENT: Remove unused val and stale documentation in ReplicaManager
[METRICS-1844, METRICS-1839, METRICS-1756,METRICS-1853] - Reconcile configs in Telemetry Reporter before creating a ConfluentTelemetryConfig object. Use password object for api.secret in ConfluentTelemetryConfig.java.
[CNKAF-885, CNKAF-946, CNKAF-947] Set SBK Default Throttle
[METRICS-1844, METRICS-1839, METRICS-1756, METRICS-1853] - Reconcile configs in Telemetry Reporter before creating a ConfluentTelemetryConfig object. Use password object for api.secret in ConfluentTelemetryConfig.java.
fix: Make EmbeddedKafka threadsafe
METRICS-1635: Generate and send config events
METRICS-1801: align telemetry domain metric names
METRICS-1835 Shade confluent-serializers dependencies in confluent-metrics (#2111)
[CNKAF-885, CNKAF-946, CNKAF-947] Set SBK Default Throttle, handle absent brokers when setting throttle.
METRICS-1835 Shade ce-serializer dependencies in ce-metrics
CNKAF-481: Verify Cluster Id in FileBasedPlainSaslAuthenticator
KSTORAGE-708: Durability Topic Partition management
METRICS-1733 replace ‘whitelist’ with ‘include’ for telemetry and metrics reporter (#2096)
METRICS-1810 use SBK topic replicas config if present (#2109)
METRICS-1733 replace ‘whitelist’ with ‘include’ for telemetry and metrics reporter
CNKAF-481: Allow embedding of logical cluster name in advertised listener config
METRICS-1810 use SBK topic replicas config if present
fix: fixes exception in server shutdown due to metrics registration
METRICS-1809: add telemetry settings to default server.properties file
KC-681: Make num.partitions a dynamic config
CNKAF-849: Sanitize log messages when checking for startup conditions
METRICS-1819: support using port 0 for telemetry local exporter
[CNKAF-927] Fix typo in server.properties. (#2090)
[CNKAF-924, CNKAF-926] Fix Databalancer Topic RF config, remove Teleme…
[CNKAF-927] Fix typo in server.properties.
CONFLUENT: Added check for CL metrics
CNKAF-918: Relax GoalUtils.eligibleBrokers criteria to allow leadership movements off observers.
CONFLUENT: Added check for CL metrics for 6.0.x
CIAM-109 Add option to fetch role bindings for multiple scopes
[CNKAF924, CNKAF-926] Fix Databalancer Topic RF config, remove Telemetry RF dependency
6.0.x
CPKAFKA-4955: Fix failing testTopicDeletion
CIAM-109 Add ALL DescribeAccess and AlterAccess for resource specific grants
cherry-pick PR 2047, 2095 from 6.0.x
KC-655 Offset migration configs with racially neutral terms.
Removed two testing files accidentally added.
SEC-1130-Inject ConfluentAdmin from ce-kafka
backport METRICS-1820: use password object for telemetry api secret
CNKAF-893: Update SBK to work with secure Zookeeper
CIAM-96 Added cloud-cluster scope and describe access for higher levels.
[METRICS-1820] use password object for telemetry api secret
SEC-1034: use confluent-log4j instead of log4j
METRICS-1816, METRICS-1705 local telemetry exporter metrics whitelist configuration for SBK (#2031)
CONFLUENT: deleting extra topic config change in cluster linking test
KC-505: Integration tests for CL metrics
CNKAF-909: Add direct dependency on opencensus for SBK (#2057)
KAFKA-10149: Allow auto preferred leader election when partitions are…
CNKAF-918: Relax GoalUtils.eligibleBrokers criteria to allow leadership movements off observers.
CNKAF-909: Add direct dependency on opencensus for SBK
Add RackAwareGoal to goal list
Add RackAwareGoal to goal list
CNKAF-884: Remove unnecessary metrics (#2059)
Migrate MDS to run on KafkaHttpServer implementation
KAFKA-10149: Allow auto preferred leader election when partitions are…
Fix ce-http-server import control violation for SslContextFactory
KC-505: integration tests for CL metrics for 6.0.x
[METRICS-1816] [METRICS-1705] Fixing some metric names in default whitelist and configured local telemetry exporter metrics whitelist configuration for SBK
[CNKAF-867] Handle empty proposals for addBroker
METRICS-1717 use shared domain for jvm level metrics (#1949)
Migrate MDS to KafkaHttpServlet implementation
CNKAF-854: Disabling SBK should persist the broker removal cancellation status
KC-138: Implements cluster link pausing.
[CNKAF-873] Don’t require network capacity configs. (#1996)
CNKAF-884: Remove unnecessary metrics
SEC-1183 - Treat empty authority in CRN pattern as a wildcard
CNKAF-882: Add proposal logging for add/remove
CNKAF-837: Add broker should be able to add removed brokers.
METRICS-1717 use shared domain for system level metrics
KC-393: Integration tests for mirroring compacted topics
Add inter broker listener endpoint information to HTTP server binder.
Add inter broker listener endpoint information to HTTP server binder.
CNKAF-773: Do not throw an exception when the broker is not present in the cluster model
CNKAF-847: Complete removal callback when there are no proposals to execute
KSTORAGE-802: Allow specifying AWS credentials through configuration file
[METRICS-1552][METRICS-1719] remove telemetry broker assumptions and use metrics context for broker check
[CNKAF-848] Set preferred user SBK config in server.properties. (#1999)
[CNKAF-873] Don’t require network capacity configs.
[CNKAF-719] Add option to disable dynamic throttle setting when stati…
KSTORAGE-819: Force roll segments only for partitions eligible for tiering
KSTORAGE-799: reduce Log.lock contention in ArchiveTask segment selection logic
KSTORAGE-774: Fix active segment roll logic when roll.min.bytes setting is enabled
KSTORAGE-766: Ensure delete segment throttler does not block active segment rolls
KSTORAGE-769: Fix to remove the costly toSet conversions in disk throttler
KSTORAGE-758: Include ERROR status in TierPartitionStatus.isOpen()
KSTORAGE-595 NPE in TierDeletedPartitionsCoordinator after tier topic is reassigned / deleted from broker
[CNKAF-848] Set preferred user SBK config in server.properties.
KSTORAGE-754: KafkaMetrics Whitelist Creation and new KafkaMetricsWrapper util (#1906)
KSTORAGE-754: KafkaMetrics Whitelist Creation and new KafkaMetricsWrapper util
Revert “KC-138: Implements cluster link pausing. (#1994)”
KC-138: Implements cluster link pausing.
KC-611: Fix mirror topic create processing incorrect config.
KSTORAGE-547: state recovery injection tooling
[METRICS-1235] implement exporter level whitelist
CONFLUENT: Fix Confluent provider start future completion, add unit tests
[CNKAF-881] SBK: Don’t treat all new brokers as empty. (#2005)
SBK/CruiseControl: low utilization fix
CNAKF-833, CNKAF-842: Fix missing request rate metric and enable tele…
KC-521: Add per listener authentication audit log enable config
CNAKF-833, CNKAF-842: Fix missing request rate metric and enable telemetry reporter sampler by default
KSTORAGE-547: state recovery injection tooling
CIAM-94 Decouple scope path keys and Role scope type
RackAwareGoal should allow for uneven replica distribution based on t…
[CNKAF-881] SBK: Don’t treat all new brokers as empty.
[CNKAF-719] Add option to disable dynamic throttle setting when stati…
KC-365: Add Jmh benchmarks for Authentication Audit Events
CNKAF-788: Introduce proper API exceptions in the broker removal API
KSTORAGE-706: Durability events and database.
METRICS-1708: add telemetry version header for forward-compatibility (#1883)
Allowing user to disable tiered storage safely - Phase 1
CONFLUENT: Use undeprecated ciphers for TLSv1.3 cipher change test
Port from CC: Fix low utilization threshold computation (#92)
CNKAF-791: Change data format of failedBrokers in ZK
CNKAF-807: Remove zkclient from gradle dependencies.
CNKAF-575: Remove IOtec:ZkClient dependency.
[CNKAF-776] Switch SBK Maven publish group to “io.confluent”
[CNKAF-776] Make SBK publish to “io.confluent”
RackAwareGoal should allow for uneven replica distribution based on t…
[METRICS-1609] dynamically add telemetry reporter at runtime
CONFLUENT: Move clusterId set on AuditLogProvider Instance to AuditLogProviderFactory
KC-389: Retry segment deletion in cluster link test to fix timing issue
KC-389: Cluster link integration tests with tier storage enabled
CPKAFKA-5239 - Fix AddPartitionsToTxn audit event category
CPKAFKA-5207: Complete authorizer start future only when sufficient nodes are ready
CNKAF-843: Fix remove broker validation check
Persist the broker removal state and handle failover
CNKAF-773: Do not throw exception when there are no replicas to move on broker removal
CNKAF-428: Integration test for replica placement self-healing
Add retries to telemetry sampler integration test (#1922)
[CNKAF-803] Add a metric to track the number of pending add-broker requests
KC-72: Add topic listing for cluster links.
KC-519: Support Authentication events in audit log routing
CPKAFKA-5207: Fail fast during broker start up if multiple brokers use same MDS URL
KC-536 bug fix to make offsets/acls configs dynamic (#1893)
[CNKAF-803] Add a metric to track the number of pending add-broker requests.
CNKAF-428: Update ReplicaPlacementGoal to handle replication factor change
KC-545: Allow configuring ssl cipher suites at the cluster level
fix telemetry reporter reconfiguration
KSTORAGE-623: added disk throttling metrics for producer and follower throttle rates
KC-536 bug fix to make offsets/acls configs dynamic
[KSTORAGE-610] Add metric for tier materialization lag
update ksql and streams provider
CNKAF-649: Handle broker restart during broker removal
KC-390: Cluster link authorization tests
KC-366: Add authentication audit logging support to Selector and authentication implementations
KC-594: use BufferSupplier GroupMetadataManager.scheduleLoadGroupAndOffsets
METRICS-1708: add telemetry version header for forward-compatibility
backport [METRICS-1564] telemetry - derive interbroker client configs (#1816)
ksql and streams provider use configs for building labels
(Backport) make commit-id optional, update whitelist to include C3 metrics
make commit-id optional, update whitelist to include C3 metrics
SEC-1083 - AuditAdmin role’s scopeType = Cluster
SEC-1083 - AuditAdmin role’s scopeType = Cluster
KC-544: Alter configs from external listener should only update said listener configs
CNKAF-428: Update ReplicaPlacementGoal to handle replication factor change
KC-392: Integration tests for cluster link quotas on source and destination
KC-582: Add multi-tenant support for alterMirrors.
KSTORAGE-705-Define per broker durability metrics and its management
CNKAF-795: Ensure SBK clients have bootstrap.servers configured
[METRICS-1564] telemetry - derive interbroker client configs
CIAM-71 Handle RBAC access policies at different scopeLevels
fix checkstyle error
[KAFKA-8528] Expose Trogdor-specific JMX metrics for Tasks and Agents
Backport PR 1897 to 6.0.x
update ksql and streams provider
SEC-1083 - Add AuditAdmin role
CNKAF-794: Properly wrap broker removal exceptions in listener and improve remove broker test
Add schemas topic label to SchemaRegistryProvider

Security¶

Fix auth for SchemasResource
Fix SR security test classpath
Implement asychronous schema fetching in LruSchemaRegistryClient
SEC-1478 Update from JUnit4 to JUnit5
Implements FF-873 in a generic way by supporting wildcards *
Check authentication.roles in KSQL bearer token requests
Include Kafka dependencies to KSQL packaging
DG-540 Add batch get schema API
fix: Make rest-utils compile scope as ksqlDB no longer pulls this in

MQTT Proxy¶

Fix telemetry reporter test on 6.0.x

Metadata Service¶

SEC-1565: fix unit test based on SEC-1565 on 5.4.x
SEC-1565: Incorrect implementation of equals in ManagedRoleBindings
SEC-1566: Switch to https for packages.confluent.io
SEC-1507 cluster information migration script should validate properties file
[MMA-9298]: bump up the version for nice name feature
[SEC-1506]: cluster-information-migration-script - allow user to specify curl options
[SEC-1514]: cluster-information-migration-script - suppress progress but show errors
[SEC-1499]: Add –help option to cluster registry import/export script
CIAM-399: decouple rbac-db from rbac-api-server
SEC-1494: Update to junit5
CIAM-338 - EnvAdmin can see more that it should in cli
[SEC-1471]: Backport Friendly cluster name from master to 6.0.x
CIAM-371 Add Delete operation to Cluster for CloudClusterAdmin
CIAM-338 - Test MyRoleBindings endpoints
CIAM-325 Add ON_ERROR_STOP inside rbac_schema_postgres.sql
CIAM-318 Define cc-rbac Postgres users in SQL
SEC-1452: refactor out cluster name validation to its own validator
CIAM-121 Environment variable is named DBMIGRATE_SHARE_DIR
SEC-1419: fix script to exit with 0 after successful execution
CIAM-121 Support DB Migrations
CIAM-313 give USAGE privilege to RBAC DB users
Add implementation for newly added “describe and alter SCRAM user” API
SEC-1402 Cluster Registry Unicode Cluster Names
CIAM-257: Add db healthcheck skip option for cpd
CIAM-269 Terraform to create schema and DB roles
CIAM-295 - Verify MDS behavior with CCloud setup and ANON super.user
Restore Cluster Registry import script in master
CIAM-241 - Port v2 tests to demonstrate calls to DB
MMA-7359 - Space in Principal Breaks Control Center Authorization
CIAM-248 - Parity Suite Infrastructure in MDS
CIAM-269 Terraform for CC RBAC RDS
CIAM-261: Add metrics for all rest end points
CIAM-262 Use connection pool for RBAC DB
CIAM-222: Config support for db
CIAM-134 Add test to demonstrate /lookup/rolebindings/principal
CIAM-245: Add DbAuthCache/DbAuthWriter metrics
SEC-1382: validate groups in “super.users”
CIAM-236 Add test for removal of env describe when cluster admin removed
CIAM-221 Implement list rolebindings queries
CIAM-102: Add caching to DBAuthCache findRule method
SEC-1372 - MDS : Address V1 Visibility Testing Gap found by C3
CIAM-227: (follow-up) Add DbAuthStoreConfig and minor cleanups to DbAuthWriter/RbacOrmService
CIAM-237 rename resource_id to id
CIAM-227: Implement resource RoleBinding methods and add validations to DBAuthWriter
CIAM-223: Support add, delete, lookup rolebindings from DB
CIAM-82 Cloud RBAC Schema
CIAM-232 Add endpoint to delete all role bindings for a principal
CIAM-82 Data generator
CIAM-119 - Rbac DB Stubs
CIAM-119 - DB Auth Maven Module and Stubs
CIAM-162 Provide /lookup/rolebindings endpoint
CIAM-86: Add perf tests for RBAC control plane authorizations
CIAM-110 Fix yaml indentation
CIAM-140 Add tests for /principals endpoints
CIAM-110 Associate RBAC Roles with Namespaces
CIAM-59: Readme update
CIAM-59: Update mds-in-a-box to support v2alpha1 rbac
SEC-1089 Fix v2 yaml MdsScope
CIAM-145 Update sdk yaml to use lookup role endpoint, make bindWithRe…
Fix aquirePort typo in test helper function
CIAM-138 Test /roles endpoint
SEC-1226 - Updated audit log api code after ce-audit added a new category event authentication
CIAM-132 Don’t allow empty principal name
SEC-1229 - MINOR: Disable getUpdatedAuditRouteConfigTest and dependents because …
SEC-1205 Gracefully handle roles request for nonexistent role
CIAM-59: Update MdsWritesFilter pathMatcher for v2alpha1
CIAM-109 fixed checkstyle
CIAM-109 add hierarchy traversal for managed resource endpoint
Have ce-kafka inject the (internal) ConfluentAdmin directly, instead …
Cherry pick 297: Move MDS to KafkaHttpServlet
CIAM-96 fix unit tests based on changes in cloud cluster admin role
CIAM-126 Make /roles/{roleName} and /roleNames v2 endpoints available to mds sdk
CIAM-125 Update Roles object
SEC-1127 - Fixed a defect that AuditLogConfigSpec didn’t get updated
CIAM-59: Disable token endpoints if no user store configured
CIAM-100 Allow scopes with SR cluster but no Kafka cluster
CIAM-27 Lookup scope includes contained scope
CIAM-94 Upstream interface changes
CIAM-59: Config to skip userstore setup/basicauth fallback
CIAM-79 Update MDS for changed interfaces in CIAM-71
removed required kafka-cluster in Scope

Replicator¶

REPL-1310 fix for monitor thread failures in start stop start cycle
REP-1300: Null out licenseManager on connector stop to avoid hang on restart
REPL-1303: Setting deadline correctly when trying to resume partitions
FF-3928:Exclude test dependency javassisst from curator-test to be assembled
FF-3928: Bump the version of Guava
REPL-1263: Supporting source url configs for active active SR
REPL-1204: Enable retries on cluster id fetch for Replicator task start
REP-1193: Removed error logging for failed executable connector submission
REP-1056: Handle empty assignments
REP-1046: Improve logging on conversion error
REP-917: Executable should not log an errors when attempting to start connectors on followers
REP-762: Add a READ ACLs check in verifier tool for destination topics
REP-763: Improve WARN log to explain if the error is on source or dest location