mirror of
https://gitee.com/milvus-io/milvus.git
synced 2025-12-28 14:35:27 +08:00
0943713481
### **User description** issue: #46466 ___ ### **PR Type** Bug fix ___ ### **Description** - Extract finished task state check into reusable helper function - Skip finished tasks during compaction recovery to prevent reprocessing - Add backward compatibility check for pre-allocated segment IDs ___ ### Diagram Walkthrough ```mermaid flowchart LR A["Compaction Task States"] -->|"Check with helper"| B["isCompactionTaskFinished()"] B -->|"Used in"| C["compactionInspector.loadMeta()"] B -->|"Used in"| D["compactionTaskMeta.reloadFromKV()"] C -->|"Skip finished tasks"| E["Recovery Process"] D -->|"Backward compatibility"| E ``` <details><summary><h3>File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table> <tr> <td> <details> <summary><strong>compaction_util.go</strong><dd><code>Add isCompactionTaskFinished helper function</code> </dd></summary> <hr> internal/datacoord/compaction_util.go <ul><li>Added new helper function <code>isCompactionTaskFinished()</code> to check if a <br>compaction task is in a terminal state<br> <li> Function checks for failed, timeout, completed, cleaned, or unknown <br>states<br> <li> Centralizes task state validation logic for reuse across multiple <br>components</ul> </details> </td> <td><a href="https://github.com/milvus-io/milvus/pull/46515/files#diff-8f2cb8d0fef37617202c5a2290ad2bdbf2df5b5983604b5b505bc73a65c7eb43">+8/-0</a> </td> </tr> </table></td></tr><tr><td><strong>Bug fix</strong></td><td><table> <tr> <td> <details> <summary><strong>compaction_inspector.go</strong><dd><code>Refactor to use finished task helper function</code> </dd></summary> <hr> internal/datacoord/compaction_inspector.go <ul><li>Replaced inline state checks with call to <code>isCompactionTaskFinished()</code> <br>helper<br> <li> Simplifies code by removing repetitive state comparison logic<br> <li> Maintains same behavior of skipping finished tasks during recovery</ul> </details> </td> <td><a href="https://github.com/milvus-io/milvus/pull/46515/files#diff-1c884001f2e84de177fea22b584f3de70a6e73695dbffa34031be9890d17da6d">+1/-5</a> </td> </tr> <tr> <td> <details> <summary><strong>compaction_task_meta.go</strong><dd><code>Add finished task check for backward compatibility</code> </dd></summary> <hr> internal/datacoord/compaction_task_meta.go <ul><li>Added check to skip finished tasks before processing pre-allocated <br>segment IDs<br> <li> Ensures backward compatibility for tasks without pre-allocated segment <br>IDs<br> <li> Prevents marking already-finished tasks as failed during reload</ul> </details> </td> <td><a href="https://github.com/milvus-io/milvus/pull/46515/files#diff-0dae7214c4c79ddf5106bd51d375b5fb2f41239d5d433798afa90708e443eca8">+1/-1</a> </td> </tr> </table></td></tr></tbody></table> </details> ___ <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved detection of finished compaction tasks to reduce false failures. * Prevented finished tasks with missing pre-allocations from being incorrectly marked as failed. * Simplified abandonment logic for completed/timeout/cleaned tasks to reduce erroneous retries and noisy logs. <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub> <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Data Coordinator
Data cooridnator(datacoord for short) is the component to organize DataNodes and segments allocations.
Dependency
- KV store: a kv store has all the meta info datacoord needs to operate. (etcd)
- Message stream: a message stream to communicate statistics information with data nodes. (Pulsar)
- Root Coordinator: timestamp, id and meta source.
- Data Node(s): could be an instance or a cluster, actual worker group handles data modification operations.