mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00

History

enhance: Forbid writing V1 format and always use StorageV2 (#46791 )

Related to #46595

Remove the EnableStorageV2 config option and enforce StorageV2 format
across all write paths including compaction, import, write buffer, and
streaming segment allocation. V1 format write tests are now skipped as
writing V1 format is no longer supported.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

2026-01-06 11:55:23 +08:00

allocator

…

broker

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

session

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

task

fix: Skip running scalar index when segment was compacted (#44690 )

2025-10-11 12:05:56 +08:00

.mockery.yaml

feat: Add force merge (#45556 )

2025-12-19 18:03:18 +08:00

analyze_inspector.go

…

analyze_meta_test.go

…

analyze_meta.go

…

build_index_policy.go

…

channel.go

enhance: remove redundant channel manager from datacoord (#44532 )

2025-10-09 11:01:57 +08:00

compaction_inspector_test.go

enhance: Set latestDeletePos from L0 segments to bound L1 selection (#46436 )

2025-12-23 11:55:19 +08:00

compaction_inspector.go

fix: Restore the compaction task correctly to ensure it can be properly cleaned up (#46577 )

2025-12-26 18:57:19 +08:00

compaction_l0_view_test.go

enhance: Set latestDeletePos from L0 segments to bound L1 selection (#46436 )

2025-12-23 11:55:19 +08:00

compaction_l0_view.go

enhance: Set latestDeletePos from L0 segments to bound L1 selection (#46436 )

2025-12-23 11:55:19 +08:00

compaction_policy_clustering_test.go

enhance: remove uncessary segment size estimation and make it configurable (#46302 )

2025-12-13 02:58:46 +08:00

compaction_policy_clustering.go

feat: Support ttl field for entity level expiration (#46342 )

2026-01-05 10:27:24 +08:00

compaction_policy_forcemerge_test.go

feat: Add force merge (#45556 )

2025-12-19 18:03:18 +08:00

compaction_policy_forcemerge.go

feat: Support ttl field for entity level expiration (#46342 )

2026-01-05 10:27:24 +08:00

compaction_policy_l0_test.go

enhance: Set latestDeletePos from L0 segments to bound L1 selection (#46436 )

2025-12-23 11:55:19 +08:00

compaction_policy_l0.go

enhance: Set latestDeletePos from L0 segments to bound L1 selection (#46436 )

2025-12-23 11:55:19 +08:00

compaction_policy_single_test.go

enhance: Move sort stats task to sort compaction (#42562 )

2025-07-08 20:22:47 +08:00

compaction_policy_single.go

feat: Support ttl field for entity level expiration (#46342 )

2026-01-05 10:27:24 +08:00

compaction_queue_test.go

…

compaction_queue.go

…

compaction_task_clustering_test.go

fix: Set task init when worker doesn't have task (#45675 )

2025-11-19 18:03:07 +08:00

compaction_task_clustering.go

enhance: integrate Storage V2 FFI interface for unified storage access (#45723 )

2025-11-24 19:57:07 +08:00

compaction_task_l0_test.go

fix: Set task init when worker doesn't have task (#45675 )

2025-11-19 18:03:07 +08:00

compaction_task_l0.go

enhance: integrate Storage V2 FFI interface for unified storage access (#45723 )

2025-11-24 19:57:07 +08:00

compaction_task_meta_test.go

…

compaction_task_meta.go

fix: Skip Finished tasks when recovery with compatibility (#46515 )

2025-12-23 18:09:18 +08:00

compaction_task_mix_test.go

enhance: remove redundant channel manager from datacoord (#44532 )

2025-10-09 11:01:57 +08:00

compaction_task_mix.go

enhance: Add log with segment size for tasks (#46118 )

2025-12-05 16:45:11 +08:00

compaction_task.go

fix: Accidentally ignored sealed segments in L0 Compaction (#45340 )

2025-11-06 16:53:38 +08:00

compaction_trigger_test.go

feat: Support ttl field for entity level expiration (#46342 )

2026-01-05 10:27:24 +08:00

compaction_trigger_v2_test.go

enhance: Set latestDeletePos from L0 segments to bound L1 selection (#46436 )

2025-12-23 11:55:19 +08:00

compaction_trigger_v2.go

enhance: Set latestDeletePos from L0 segments to bound L1 selection (#46436 )

2025-12-23 11:55:19 +08:00

compaction_trigger.go

feat: Support ttl field for entity level expiration (#46342 )

2026-01-05 10:27:24 +08:00

compaction_util.go

enhance: tidy code in import plugin context (#46374 )

2026-01-04 16:51:23 +08:00

compaction_view_forcemerge_test.go

fix: Disable VCS to allow pkg tests to run (#46501 )

2025-12-25 14:31:19 +08:00

compaction_view_forcemerge.go

fix: Disable VCS to allow pkg tests to run (#46501 )

2025-12-25 14:31:19 +08:00

compaction_view.go

feat: support mannual compact l0(#44439 ) (#44440 )

2025-09-23 12:44:07 +08:00

const.go

…

copy_segment_checker_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

copy_segment_checker.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

copy_segment_inspector_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

copy_segment_inspector.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

copy_segment_job_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

copy_segment_job.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

copy_segment_meta_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

copy_segment_meta.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

copy_segment_task_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

copy_segment_task.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

create_meta_test.go

…

ddl_callbacks_alter_index.go

enhance: support collection and index with WAL-based DDL framework (#45033 )

2025-10-30 14:24:08 +08:00

ddl_callbacks_create_index.go

enhance: support collection and index with WAL-based DDL framework (#45033 )

2025-10-30 14:24:08 +08:00

ddl_callbacks_drop_index.go

enhance: support collection and index with WAL-based DDL framework (#45033 )

2025-10-30 14:24:08 +08:00

ddl_callbacks_flushall.go

enhance: Implement new FlushAllMessage and refactor flush all (#45920 )

2025-12-10 19:27:13 +08:00

ddl_callbacks_snapshot_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

ddl_callbacks_snapshot.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

ddl_callbacks.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

errors_test.go

…

errors.go

…

external_collection_inspector.go

enhance: implement external collection update task with source change detection (#45905 )

2025-12-29 19:53:21 +08:00

external_collection_task_meta.go

enhance: implement external collection update task with source change detection (#45905 )

2025-12-29 19:53:21 +08:00

file_resource_manager_test.go

enhance: sync file resource to querynode and datanode (#44480 )

2025-12-04 16:23:11 +08:00

file_resource_manager.go

enhance: sync file resource to querynode and datanode (#44480 )

2025-12-04 16:23:11 +08:00

garbage_collector_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

garbage_collector.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

go_channel_singleton.go

…

handler_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

handler.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

import_checker_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

import_checker.go

enhance: Skip redundant failure marking for completed import jobs (#45767 )

2025-11-25 11:21:07 +08:00

import_inspector_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

import_inspector.go

enhance: Move sort stats task to sort compaction (#42562 )

2025-07-08 20:22:47 +08:00

import_job.go

enhance: Show create time for import job (#45058 )

2025-10-27 12:14:08 +08:00

import_meta_test.go

…

import_meta.go

…

import_task_import_test.go

…

import_task_import.go

enhance: support useLoonFFI flag in import workflow (#46363 )

2025-12-17 16:35:16 +08:00

import_task_preimport_test.go

…

import_task_preimport.go

fix: Fix task state is InProgress but payload is nil (#43777 )

2025-08-11 14:13:42 +08:00

import_task.go

enhance: Move sort stats task to sort compaction (#42562 )

2025-07-08 20:22:47 +08:00

import_util_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

import_util.go

enhance: Forbid writing V1 format and always use StorageV2 (#46791 )

2026-01-06 11:55:23 +08:00

index_engine_version_manager_test.go

enhance: Implement rewatch mechanism for etcd failure scenarios (#43829 )

2025-08-14 10:31:44 +08:00

index_engine_version_manager.go

enhance: Implement rewatch mechanism for etcd failure scenarios (#43829 )

2025-08-14 10:31:44 +08:00

index_inspector_test.go

enhance: override index_type while creating segment index (#45416 )

2025-11-11 07:27:36 +08:00

index_inspector.go

enhance: Estimate index task slot using field size instead of segment size (#46275 )

2025-12-23 11:23:22 +08:00

index_meta_test.go

fix: ddl framework bug patch (#45290 )

2025-11-04 22:25:33 +08:00

index_meta.go

enhance: override index_type while creating segment index (#45416 )

2025-11-11 07:27:36 +08:00

index_service_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

index_service.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

knapsack_test.go

…

knapsack.go

…

meta_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

meta_util.go

fix: make savebinlogpath idompotent at binlog level (#43615 )

2025-07-29 19:47:36 +08:00

meta.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

metrics_info_test.go

enhance: sync file resource to querynode and datanode (#44480 )

2025-12-04 16:23:11 +08:00

metrics_info.go

enhance: remove redundant channel manager from datacoord (#44532 )

2025-10-09 11:01:57 +08:00

mock_collection_topology_querier.go

feat: Add force merge (#45556 )

2025-12-19 18:03:18 +08:00

mock_compaction_inspector.go

enhance: Move sort stats task to sort compaction (#42562 )

2025-07-08 20:22:47 +08:00

mock_compaction_meta.go

enhance: Fail compaction task to prevent data loss (#43545 )

2025-07-25 16:24:54 +08:00

mock_handler.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

mock_import_meta.go

fix: Pre-check import message to prevent pipeline block indefinitely (#42415 )

2025-06-11 13:40:38 +08:00

mock_index_engine_version_manager.go

…

mock_segment_manager.go

…

mock_stats_job_manager.go

…

mock_sub_cluster.go

…

mock_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

mock_trigger_manager.go

feat: Add force merge (#45556 )

2025-12-19 18:03:18 +08:00

mock_trigger.go

…

OWNERS

…

partition_stats_meta_test.go

…

partition_stats_meta.go

…

README.md

…

segment_allocation_policy_test.go

enhance: remove uncessary segment size estimation and make it configurable (#46302 )

2025-12-13 02:58:46 +08:00

segment_allocation_policy.go

enhance: remove uncessary segment size estimation and make it configurable (#46302 )

2025-12-13 02:58:46 +08:00

segment_info_test.go

enhance: Estimate index task slot using field size instead of segment size (#46275 )

2025-12-23 11:23:22 +08:00

segment_info.go

enhance: Estimate index task slot using field size instead of segment size (#46275 )

2025-12-23 11:23:22 +08:00

segment_manager_test.go

…

segment_manager.go

…

segment_operator_test.go

…

segment_operator.go

…

server_test.go

feat: Add force merge (#45556 )

2025-12-19 18:03:18 +08:00

server.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

services_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

services.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

snapshot_manager_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

snapshot_manager.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

snapshot_meta_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

snapshot_meta.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

snapshot_test.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

snapshot.go

feat: Add comprehensive snapshot functionality for collections (#44361 )

2026-01-06 10:15:24 +08:00

stats_inspector_test.go

enhance: Move sort stats task to sort compaction (#42562 )

2025-07-08 20:22:47 +08:00

stats_inspector.go

enhance: support json stats with shredding design (#42534 )

2025-09-01 10:49:52 +08:00

stats_task_meta_test.go

enhance: Move sort stats task to sort compaction (#42562 )

2025-07-08 20:22:47 +08:00

stats_task_meta.go

enhance: Move sort stats task to sort compaction (#42562 )

2025-07-08 20:22:47 +08:00

task_analyze_test.go

feat: Add CMEK cipher plugin (#43722 )

2025-08-27 11:15:52 +08:00

task_analyze.go

feat: Add CMEK cipher plugin (#43722 )

2025-08-27 11:15:52 +08:00

task_index_test.go

…

task_index.go

enhance: Resolve issues integrating loon FFI (#45918 )

2025-11-28 18:41:08 +08:00

task_stats_test.go

enhance: Move sort stats task to sort compaction (#42562 )

2025-07-08 20:22:47 +08:00

task_stats.go

enhance: pass manifest path to stats task for storage v2 support (#46350 )

2025-12-16 11:11:16 +08:00

task_update_external_collection_test.go

enhance: implement external collection update task with source change detection (#45905 )

2025-12-29 19:53:21 +08:00

task_update_external_collection.go

fix: remove redundant log (#46695 )

2025-12-31 15:35:21 +08:00

util_test.go

fix: validate collection TTL property to prevent compaction stuck (#46717 )

2026-01-01 08:13:22 +08:00

util.go

fix: validate collection TTL property to prevent compaction stuck (#46717 )

2026-01-01 08:13:22 +08:00

README.md

Data Coordinator

Data cooridnator(datacoord for short) is the component to organize DataNodes and segments allocations.

Dependency

KV store: a kv store has all the meta info datacoord needs to operate. (etcd)
Message stream: a message stream to communicate statistics information with data nodes. (Pulsar)
Root Coordinator: timestamp, id and meta source.
Data Node(s): could be an instance or a cluster, actual worker group handles data modification operations.