2 Commits

Author	SHA1	Message	Date
yihao.dai	b1e74dc7cb	enhance: [cherry-pick] Decouple compaction from shard (#34157) ... This PR cherry-picks the following commits: - Implement task limit control logic in datanode. https://github.com/milvus-io/milvus/pull/32881 - Load bf from storage instead of memory during L0 compaction. https://github.com/milvus-io/milvus/pull/32913 - Remove dependencies on shards (e.g. SyncSegments, injection). https://github.com/milvus-io/milvus/pull/33138 - Rename Compaction interface to CompactionV2. https://github.com/milvus-io/milvus/pull/33858 - Remove the unused residual compaction logic. https://github.com/milvus-io/milvus/pull/33932 issue: https://github.com/milvus-io/milvus/issues/32809 pr: https://github.com/milvus-io/milvus/pull/32881, https://github.com/milvus-io/milvus/pull/32913, https://github.com/milvus-io/milvus/pull/33138, https://github.com/milvus-io/milvus/pull/33858, https://github.com/milvus-io/milvus/pull/33932 --------- Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-06-25 20:22:03 +08:00
XuanYang-cn	a33b68678d	enhance: [cherry-pick] Move compactor into sub package (#34098) ... This PR consists of the following commits: - enhance: Tidy compactor and remove dup codes (#32198) - fix: Fix l0 compactor may cause DN from OOM (#33554) - enhance: Add deltaRowCount in l0 compaction (#33997) - enhance: enable stream writer in compactions (#32612) - fix: turn on compression on stream writers (#34067) - fix: adding blob memory size in binlog serde (#33324) See also: #32451, #33547, #33998, #31679 pr: #32198, #33554, #33997, #32612 --------- Signed-off-by: yangxuan <xuan.yang@zilliz.com> Signed-off-by: Ted Xu <ted.xu@zilliz.com> Co-authored-by: Ted Xu <ted.xu@zilliz.com>	2024-06-25 11:16:02 +08:00