4 Commits

Author SHA1 Message Date
XuanYang-cn
21d76ad1ce
enhance: Use partitionID when delete by partitionKey (#38232)
When delete by partition_key, Milvus will generates L0 segments
globally. During L0 Compaction, those L0 segments will touch all
partitions collection wise. Due to the false-positive rate of segment
bloomfilters, L0 compactions will append false deltalogs to completed
irrelevant partitions, which causes *partition deletion amplification.

This PR uses partition_key to set targeted partitionID when producing
deleteMsgs into MsgStreams. This'll narrow down L0 segments scope to
partition level, and remove the false-positive influence
collection-wise.

However, due to DeleteMsg structure, we can only label one partition to
one deleteMsg, so this enhancement fails if user wants to delete over 2
partition_keys in one deletion.

pr: #38231 
See also: #34665

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-23 13:52:51 +08:00
wei liu
da026b1e28
enhance: Add depguard rules to ban deprecated proto lib (#35140) (#35818)
See also #34394 #34252
pr: #35140

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: congqixia <congqi.xia@zilliz.com>
2024-08-30 14:13:01 +08:00
XuanYang-cn
ff68eb53d7
fix: [skip e2e]unstable l0 it (#35613)
See also: #35617 
pr: #35612

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-26 18:53:04 +08:00
XuanYang-cn
4a5e6bc6f6
enhance: Add integration tests for l0 (#35430)
pr: #35429
See also: #34796

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-19 10:54:55 +08:00