mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 11:21:52 +08:00

History

enhance: move streaming coord from datacoord to rootcoord (#39007 )

issue: #38399

We want to support broadcast operation for both streaming and msgstream.
But msgstream can be only sent message from rootcoord and proxy.
So this pr move the streamingcoord to rootcoord to make easier
implementation.

---------

Signed-off-by: chyezh <chyezh@outlook.com>

2025-01-07 17:42:57 +08:00

allocator

fix: fix make generate-mockery panic with go1.22 (#36830 )

2024-10-17 12:11:31 +08:00

broker

fix: fix make generate-mockery panic with go1.22 (#36830 )

2024-10-17 12:11:31 +08:00

session

fix: SyncSegments rpc always failed (#38578 )

2024-12-19 15:40:45 +08:00

analyze_meta_test.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

analyze_meta.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

build_index_policy.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

channel_manager_factory.go

fix: drop collection failed if enable streaming service (#37444 )

2024-11-07 10:26:26 +08:00

channel_manager_test.go

fix: ChannelManager concurret Release and Watch bug (#38590 )

2024-12-19 22:50:47 +08:00

channel_manager.go

fix: ChannelManager concurret Release and Watch bug (#38590 )

2024-12-19 22:50:47 +08:00

channel_store_test.go

fix: ChannelManager concurret Release and Watch bug (#38590 )

2024-12-19 22:50:47 +08:00

channel_store.go

fix: ChannelManager concurret Release and Watch bug (#38590 )

2024-12-19 22:50:47 +08:00

channel.go

fix: ChannelManager concurret Release and Watch bug (#38590 )

2024-12-19 22:50:47 +08:00

cluster_test.go

enhance: Move datanode/indexnode manager to session pkg (#35634 )

2024-08-22 16:02:56 +08:00

cluster.go

enhance: refine exists log print with ctx (#38080 )

2024-12-14 22:36:44 +08:00

compaction_l0_view_test.go

enhance: Add deltaRowCount in l0 compaction (#33997 )

2024-06-20 17:46:01 +08:00

compaction_l0_view.go

enhance: Add deltaRowCount in l0 compaction (#33997 )

2024-06-20 17:46:01 +08:00

compaction_policy_clustering_test.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

compaction_policy_clustering.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

compaction_policy_l0_test.go

enhance: Separate allocator pkg in datacoord (#35622 )

2024-08-22 10:06:56 +08:00

compaction_policy_l0.go

enhance: tidy compaction logs (#37595 )

2024-11-12 21:18:29 +08:00

compaction_policy_single_test.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

compaction_policy_single.go

enhance: tidy compaction logs (#37595 )

2024-11-12 21:18:29 +08:00

compaction_queue_test.go

fix: Refine compactionTask to avoid data race (#36936 )

2024-10-24 09:55:28 +08:00

compaction_queue.go

fix: Refine compactionTask to avoid data race (#36936 )

2024-10-24 09:55:28 +08:00

compaction_task_clustering_test.go

enhance: clean compaction task in compactionHandler (#38170 )

2024-12-19 12:36:47 +08:00

compaction_task_clustering.go

enhance: clean compaction task in compactionHandler (#38170 )

2024-12-19 12:36:47 +08:00

compaction_task_l0_test.go

enhance: clean compaction task in compactionHandler (#38170 )

2024-12-19 12:36:47 +08:00

compaction_task_l0.go

enhance: clean compaction task in compactionHandler (#38170 )

2024-12-19 12:36:47 +08:00

compaction_task_meta_test.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

compaction_task_meta.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

compaction_task_mix_test.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

compaction_task_mix.go

enhance: clean compaction task in compactionHandler (#38170 )

2024-12-19 12:36:47 +08:00

compaction_task.go

enhance: clean compaction task in compactionHandler (#38170 )

2024-12-19 12:36:47 +08:00

compaction_test.go

enhance: clean compaction task in compactionHandler (#38170 )

2024-12-19 12:36:47 +08:00

compaction_trigger_test.go

enhance: improve mix compaction performance by removing max segment limitations (#38344 )

2024-12-11 20:38:42 +08:00

compaction_trigger_v2_test.go

enhance: Separate allocator pkg in datacoord (#35622 )

2024-08-22 10:06:56 +08:00

compaction_trigger_v2.go

enhance: refine exists log print with ctx (#38080 )

2024-12-14 22:36:44 +08:00

compaction_trigger.go

enhance: refine exists log print with ctx (#38080 )

2024-12-14 22:36:44 +08:00

compaction_view.go

enhance: Add deltaRowCount in l0 compaction (#33997 )

2024-06-20 17:46:01 +08:00

compaction.go

fix: Release compaction task lock when return function (#38856 )

2024-12-30 22:44:49 +08:00

const.go

enhance: pass partition key scalar info if enabled when build vector index (#29931 )

2024-01-24 00:04:55 +08:00

errors_test.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

errors.go

Refine DataCoord status (#27262 )

2023-09-26 17:15:27 +08:00

garbage_collector_test.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

garbage_collector.go

enhance: refine exists log print with ctx (#38080 )

2024-12-14 22:36:44 +08:00

go_channel_singleton.go

fix: Fix improper use of offset in HybridSearch (#36244 )

2024-09-13 22:05:15 +08:00

handler_test.go

fix: Support get segments from current segments view (#38512 )

2024-12-18 18:00:54 +08:00

handler.go

fix: Support get segments from current segments view (#38512 )

2024-12-18 18:00:54 +08:00

import_checker_test.go

fix: Use the correct RootPath when decompressing binlog in stats task (#38341 )

2024-12-11 16:16:42 +08:00

import_checker.go

fix: Use the correct RootPath when decompressing binlog in stats task (#38341 )

2024-12-11 16:16:42 +08:00

import_job.go

fix: Revert "enhance: Support db for bulkinsert (#37012 )" (#37420 )

2024-11-07 17:02:25 +08:00

import_meta_test.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

import_meta.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

import_scheduler_test.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

import_scheduler.go

enhance: refine exists log print with ctx (#38080 )

2024-12-14 22:36:44 +08:00

import_task.go

enhance: Use internal json(sonic) to replace std json lib (#37708 )

2024-11-18 10:46:31 +08:00

import_util_test.go

fix: Use the correct RootPath when decompressing binlog in stats task (#38341 )

2024-12-11 16:16:42 +08:00

import_util.go

fix: Use the correct RootPath when decompressing binlog in stats task (#38341 )

2024-12-11 16:16:42 +08:00

index_engine_version_manager_test.go

Optimize the codec code of session (#27360 )

2023-10-01 10:33:30 +08:00

index_engine_version_manager.go

enhance: Add go-deadlock as unittest only dependency (#33063 )

2024-05-15 16:33:34 +08:00

index_meta_test.go

enhance: Add ctx param to KV operation interfaces (#38154 )

2024-12-05 15:16:41 +08:00

index_meta.go

enhance: refine exists log print with ctx (#38080 )

2024-12-14 22:36:44 +08:00

index_service_test.go

enhance: alterindex & altercollection supports altering properties (#37437 )

2024-12-11 10:20:42 +08:00

index_service.go

enhance: refine exists log print with ctx (#38080 )

2024-12-14 22:36:44 +08:00

job_manager_test.go

feat: add segment/channel/task/slow query render (#37561 )

2024-11-12 17:44:29 +08:00

job_manager.go

fix: Use the correct RootPath when decompressing binlog in stats task (#38341 )

2024-12-11 16:16:42 +08:00

knapsack_test.go

enhance: try compact small segments first if they may compose a full segment (#37709 )

2024-12-02 16:12:38 +08:00

knapsack.go

enhance: improve mix compaction performance by removing max segment limitations (#38344 )

2024-12-11 20:38:42 +08:00

meta_test.go

enhance: Add ctx param to KV operation interfaces (#38154 )

2024-12-05 15:16:41 +08:00

meta_util.go

Make VChannelInfo read compatible with 2.0.2 (#17995 )

2022-07-07 21:58:22 +08:00

meta.go

enhance: clean compaction task in compactionHandler (#38170 )

2024-12-19 12:36:47 +08:00

metrics_info_test.go

enhance: add list index and segment index retrieval API for WebUI (#37861 )

2024-11-22 16:58:34 +08:00

metrics_info.go

enhance: add disk stats within system metrics (#38033 )

2024-12-06 16:32:41 +08:00

mock_channel_store.go

fix: ChannelManager concurret Release and Watch bug (#38590 )

2024-12-19 22:50:47 +08:00

mock_channelmanager.go

feat: add segment,pipeline, replica and resourcegroup api for WebUI (#37344 )

2024-11-07 11:52:25 +08:00

mock_cluster.go

fix: fix make generate-mockery panic with go1.22 (#36830 )

2024-10-17 12:11:31 +08:00

mock_compaction_meta.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

mock_compaction_plan_context.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

mock_handler.go

fix: Support get segments from current segments view (#38512 )

2024-12-18 18:00:54 +08:00

mock_index_engine_version_manager.go

fix: fix make generate-mockery panic with go1.22 (#36830 )

2024-10-17 12:11:31 +08:00

mock_job_manager.go

fix: Use the correct RootPath when decompressing binlog in stats task (#38341 )

2024-12-11 16:16:42 +08:00

mock_segment_manager.go

enhance: Reduce segmentManager lock granularity (#37836 )

2024-12-17 14:12:52 +08:00

mock_subcluster.go

fix: fix make generate-mockery panic with go1.22 (#36830 )

2024-10-17 12:11:31 +08:00

mock_test.go

fix: revert optimize CPU usage for CheckHealth requests (#35589 ) (#38555 )

2024-12-19 00:38:45 +08:00

mock_trigger_manager.go

fix: fix make generate-mockery panic with go1.22 (#36830 )

2024-10-17 12:11:31 +08:00

OWNERS

[skip ci]Update OWNERS files (#11898 )

2021-11-16 15:41:11 +08:00

partition_stats_meta_test.go

enhance: clean compaction task in compactionHandler (#38170 )

2024-12-19 12:36:47 +08:00

partition_stats_meta.go

enhance: clean compaction task in compactionHandler (#38170 )

2024-12-19 12:36:47 +08:00

policy_test.go

fix: Fill nil schema so that Milvus can watch channel for those upgraded from 2.2 to 2.4 #35695 (#35694 )

2024-08-27 10:36:59 +08:00

policy.go

fix: drop collection failed if enable streaming service (#37444 )

2024-11-07 10:26:26 +08:00

README.md

[skip ci]Change etcd to lowercase (#9983 )

2021-10-15 18:58:37 +08:00

segment_allocation_policy_test.go

enhance: Change the fixed value to a ratio for clustering segment size (#35076 )

2024-08-01 22:04:14 +08:00

segment_allocation_policy.go

fix: Fix large growing segment (#37388 )

2024-11-07 17:12:25 +08:00

segment_info_test.go

fix: Return all compactTo segments after support split (#36361 )

2024-09-20 14:11:11 +08:00

segment_info.go

enhance: Remove unnecessary clone in SetState (#37697 )

2024-11-15 15:06:30 +08:00

segment_manager_test.go

enhance: Reduce segmentManager lock granularity (#37836 )

2024-12-17 14:12:52 +08:00

segment_manager.go

enhance: Reduce segmentManager lock granularity (#37836 )

2024-12-17 14:12:52 +08:00

segment_operator_test.go

fix: Prevent clone when selecting segments from meta (#30928 )

2024-03-01 13:59:02 +08:00

segment_operator.go

enhance: Update stats task to optional (#35947 )

2024-09-12 20:37:08 +08:00

server_test.go

fix: revert optimize CPU usage for CheckHealth requests (#35589 ) (#38555 )

2024-12-19 00:38:45 +08:00

server.go

enhance: move streaming coord from datacoord to rootcoord (#39007 )

2025-01-07 17:42:57 +08:00

services_test.go

enhance: Reduce segmentManager lock granularity (#37836 )

2024-12-17 14:12:52 +08:00

services.go

fix: Revert "Expose metrics of stanby coordinators (#27698 )" (#38620 )

2024-12-23 11:46:57 +08:00

stats_task_meta_test.go

fix: Record the nodeID before assigning tasks (#36371 )

2024-09-28 17:21:15 +08:00

stats_task_meta.go

feat: add segment,pipeline, replica and resourcegroup api for WebUI (#37344 )

2024-11-07 11:52:25 +08:00

sync_segments_scheduler_test.go

fix: datacoord stuck at stopping progress (#36852 )

2024-10-17 12:13:37 +08:00

sync_segments_scheduler.go

fix: Support get segments from current segments view (#38512 )

2024-12-18 18:00:54 +08:00

task_analyze.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

task_index.go

fix:remove unnecessary error logs (#38245 )

2024-12-08 17:20:40 +08:00

task_scheduler_test.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

task_scheduler.go

enhance: Refine task scheduler logs (#38334 )

2024-12-11 15:00:44 +08:00

task_stats_test.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

task_stats.go

enhance: refine the datacoord meta related interfaces (#37957 )

2024-11-26 19:46:34 +08:00

types.go

fix: Record the nodeID before assigning tasks (#36371 )

2024-09-28 17:21:15 +08:00

util_test.go

enhance: Separate allocator pkg in datacoord (#35622 )

2024-08-22 10:06:56 +08:00

util.go

fix: revert optimize CPU usage for CheckHealth requests (#35589 ) (#38555 )

2024-12-19 00:38:45 +08:00

README.md

Data Coordinator

Data cooridnator(datacoord for short) is the component to organize DataNodes and segments allocations.

Dependency

KV store: a kv store has all the meta info datacoord needs to operate. (etcd)
Message stream: a message stream to communicate statistics information with data nodes. (Pulsar)
Root Coordinator: timestamp, id and meta source.
Data Node(s): could be an instance or a cluster, actual worker group handles data modification operations.