3 Commits

Author	SHA1	Message	Date
sthuang	d7df78a6c9	feat: Storage v2 compaction (#40667) ... - Feat: Support Mix compaction. Covering tests include compatibility and rollback ability. - Read v1 segments and compact with v2 format. - Read both v1 and v2 segments and compact with v2 format. - Read v2 segments and compact with v2 format. - Compact with duplicate primary key test. - Compact with bm25 segments. - Compact with merge sort segments. - Compact with no expiration segments. - Compact with lack binlog segments. - Compact with nullable field segments. - Feat: Support Clustering compaction. Covering tests include compatibility and rollback ability. - Read v1 segments and compact with v2 format. - Read both v1 and v2 segments and compact with v2 format. - Read v2 segments and compact with v2 format. - Compact bm25 segments with v2 format. - Compact with memory limit. - Enhance: Use serdeMap serialize in BuildRecord function to support all Milvus data types. related: #39173 Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>	2025-03-21 10:16:12 +08:00
Ted Xu	df4285c9ef	enhance: API integration with storage v2 in clustering-compactions (#40133) ... See #39173 --------- Signed-off-by: Ted Xu <ted.xu@zilliz.com>	2025-03-13 14:12:06 +08:00
XuanYang-cn	837ac295fa	enhance: Remove iterators in datanode (#40301) ... Iterators are long deprecated, but sort are still using it. This PR unifies stats task with the latest compaction common functions and remove the usage of iterators. 1. Rename `datanode/compaction` to `datanode/compactor` 2. Add `internal/compaction` and move some compaction commons into it. 3. Replace `DeltalogIterators` with `ComposeDeleteFromDeltalogs` 4. Remove `datanode/iterators` See also: #39242 Signed-off-by: yangxuan <xuan.yang@zilliz.com>	2025-03-04 12:14:00 +08:00