milvus

mirror of https://gitee.com/milvus-io/milvus.git synced 2025-12-07 01:28:27 +08:00

History

fix: Prevent delete records loss during slow segment loading [QueryNodeV2] (#43527 )

issue: #42884
Fixes an issue where delete records for a segment are lost from the
delete buffer if `load segment` execution on the delegator is too slow,
causing `syncTargetVersion` or other cleanup operations to clear them
prematurely.

Changes include:
- Introduced `Pin` and `Unpin` methods in `DeleteBuffer` interface and
its implementations (`doubleCacheBuffer`, `listDeleteBuffer`).
- Added a `pinnedTimestamps` map to track timestamps protected from
cleanup by specific segments.
- Modified `LoadSegments` in `shardDelegator` to `Pin` relevant segment
delete records before loading and `Unpin` them afterwards.
- Added `isPinned` check in `UnRegister` and `TryDiscard` methods of
`listDeleteBuffer` to skip cleanup if corresponding timestamps are
pinned.
- Added comprehensive unit tests for `Pin`, `Unpin`, and `isPinned`
functionality, covering basic, multiple pins, concurrent, and edge
cases.

This ensures the integrity of delete records by preventing their
premature removal from the delete buffer during segment loading.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>

2025-07-24 01:00:54 +08:00

delete_buffer_test.go

fix: Fix L0 segment retention and improve delete buffer logging (#40884 )

2025-03-27 11:24:21 +08:00

delete_buffer.go

fix: Prevent delete records loss during slow segment loading [QueryNodeV2] (#43527 )

2025-07-24 01:00:54 +08:00

delete_item_test.go

enhance: Add delete buffer related quota logic (#35918 )

2024-09-05 11:39:03 +08:00

delete_item.go

enhance: Add delete buffer related quota logic (#35918 )