admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
milvus/internal/datacoord
History
congqixia 05b6ea1351
enhance: Refine error msg for schema & index checking (#39533) 
The error message was malformated or missing some meta info, say field
name. This PR recitfies some message format and add field name in error
message when type param check fails.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-24 09:57:06 +08:00
..
allocator
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
broker
enhance: Read metadata concurrently to accelerate recovery (#38403)
2025-01-23 14:27:27 +08:00
session
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
analyze_meta_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
analyze_meta.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
build_index_policy.go
Format the code (#27275)
2023-09-21 09:45:27 +08:00
channel_manager_factory.go
fix: drop collection failed if enable streaming service (#37444)
2024-11-07 10:26:26 +08:00
channel_manager_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
channel_manager.go
fix: Fix consume blocked due to too many consumers (#38455)
2025-01-15 21:37:01 +08:00
channel_store_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
channel_store.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
channel.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
cluster_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
cluster.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
compaction_l0_view_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
compaction_l0_view.go
enhance: Add deltaRowCount in l0 compaction (#33997)
2024-06-20 17:46:01 +08:00
compaction_policy_clustering_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
compaction_policy_clustering.go
enhance: Reduce mutex contention in datacoord meta (#38219)
2025-01-15 01:15:02 +08:00
compaction_policy_l0_test.go
fix: Record active collections for l0Policy (#39217)
2025-01-16 20:19:02 +08:00
compaction_policy_l0.go
fix: Record active collections for l0Policy (#39217)
2025-01-16 20:19:02 +08:00
compaction_policy_single_test.go
enhance: Reduce mutex contention in datacoord meta (#38219)
2025-01-15 01:15:02 +08:00
compaction_policy_single.go
enhance: Reduce mutex contention in datacoord meta (#38219)
2025-01-15 01:15:02 +08:00
compaction_queue_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
compaction_queue.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
compaction_task_clustering_test.go
enhance: Read metadata concurrently to accelerate recovery (#38403)
2025-01-23 14:27:27 +08:00
compaction_task_clustering.go
enhance: simplify compaction tasks to reduce their memory overhead (#39121)
2025-01-15 14:51:00 +08:00
compaction_task_l0_test.go
enhance: simplify compaction tasks to reduce their memory overhead (#39121)
2025-01-15 14:51:00 +08:00
compaction_task_l0.go
enhance: simplify compaction tasks to reduce their memory overhead (#39121)
2025-01-15 14:51:00 +08:00
compaction_task_meta_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
compaction_task_meta.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
compaction_task_mix_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
compaction_task_mix.go
enhance: simplify compaction tasks to reduce their memory overhead (#39121)
2025-01-15 14:51:00 +08:00
compaction_task.go
enhance: simplify compaction tasks to reduce their memory overhead (#39121)
2025-01-15 14:51:00 +08:00
compaction_test.go
fix: [skip e2e] Make test case assigner result deterministic (#39317)
2025-01-16 14:25:02 +08:00
compaction_trigger_test.go
enhance: Reduce mutex contention in datacoord meta (#38219)
2025-01-15 01:15:02 +08:00
compaction_trigger_v2_test.go
fix: Record active collections for l0Policy (#39217)
2025-01-16 20:19:02 +08:00
compaction_trigger_v2.go
fix: Record active collections for l0Policy (#39217)
2025-01-16 20:19:02 +08:00
compaction_trigger.go
enhance: Reduce mutex contention in datacoord meta (#38219)
2025-01-15 01:15:02 +08:00
compaction_view.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
compaction.go
enhance: Add configs for compaction schedule (#39010)
2025-01-22 16:31:10 +08:00
const.go
enhance: pass partition key scalar info if enabled when build vector index (#29931)
2024-01-24 00:04:55 +08:00
errors_test.go
Format the code (#27275)
2023-09-21 09:45:27 +08:00
errors.go
Refine DataCoord status (#27262)
2023-09-26 17:15:27 +08:00
garbage_collector_test.go
enhance: Read metadata concurrently to accelerate recovery (#38403)
2025-01-23 14:27:27 +08:00
garbage_collector.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
go_channel_singleton.go
fix: Fix improper use of offset in HybridSearch (#36244)
2024-09-13 22:05:15 +08:00
handler_test.go
fix: Record a map to avoid repeatedly traversing the CompactionFrom (#38925)
2025-01-15 10:02:58 +08:00
handler.go
fix: Record a map to avoid repeatedly traversing the CompactionFrom (#38925)
2025-01-15 10:02:58 +08:00
import_checker_test.go
enhance: Read metadata concurrently to accelerate recovery (#38403)
2025-01-23 14:27:27 +08:00
import_checker.go
enhance: Limit number of segments restored and promptly terminate the job (#39344)
2025-01-21 15:13:13 +08:00
import_job.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
import_meta_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
import_meta.go
enhance: refine the datacoord meta related interfaces (#37957)
2024-11-26 19:46:34 +08:00
import_scheduler_test.go
enhance: Read metadata concurrently to accelerate recovery (#38403)
2025-01-23 14:27:27 +08:00
import_scheduler.go
enhance: Limit number of segments restored and promptly terminate the job (#39344)
2025-01-21 15:13:13 +08:00
import_task.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
import_util_test.go
enhance: Read metadata concurrently to accelerate recovery (#38403)
2025-01-23 14:27:27 +08:00
import_util.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
index_engine_version_manager_test.go
fix: Add scalar index engine version for compatibility (#39204)
2025-01-15 12:25:00 +08:00
index_engine_version_manager.go
fix: Add scalar index engine version for compatibility (#39204)
2025-01-15 12:25:00 +08:00
index_meta_test.go
enhance: record memory size (uncompressed) item for index (#38770)
2025-01-14 10:33:06 +08:00
index_meta.go
enhance: Skip update index metrics if index dropped (#39458)
2025-01-22 11:19:04 +08:00
index_service_test.go
fix: deny to set the mmap param for the alter index api when enable auto index (#39518)
2025-01-23 10:01:04 +08:00
index_service.go
enhance: Refine error msg for schema & index checking (#39533)
2025-01-24 09:57:06 +08:00
job_manager_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
job_manager.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
knapsack_test.go
enhance: try compact small segments first if they may compose a full segment (#37709)
2024-12-02 16:12:38 +08:00
knapsack.go
enhance: improve mix compaction performance by removing max segment limitations (#38344)
2024-12-11 20:38:42 +08:00
meta_test.go
enhance: Read metadata concurrently to accelerate recovery (#38403)
2025-01-23 14:27:27 +08:00
meta_util.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
meta.go
enhance: Read metadata concurrently to accelerate recovery (#38403)
2025-01-23 14:27:27 +08:00
metrics_info_test.go
enhance: record memory size (uncompressed) item for index (#38770)
2025-01-14 10:33:06 +08:00
metrics_info.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
mock_channel_store.go
fix: ChannelManager concurret Release and Watch bug (#38590)
2024-12-19 22:50:47 +08:00
mock_channelmanager.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
mock_cluster.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
mock_compaction_meta.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
mock_compaction_plan_context.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
mock_handler.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
mock_index_engine_version_manager.go
fix: Add scalar index engine version for compatibility (#39204)
2025-01-15 12:25:00 +08:00
mock_job_manager.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
mock_segment_manager.go
fix: deleted the sealed segment data accidentally (#39421)
2025-01-20 15:29:02 +08:00
mock_subcluster.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
mock_test.go
enhance: Read metadata concurrently to accelerate recovery (#38403)
2025-01-23 14:27:27 +08:00
mock_trigger_manager.go
fix: deleted the sealed segment data accidentally (#39421)
2025-01-20 15:29:02 +08:00
OWNERS
[skip ci]Update OWNERS files (#11898)
2021-11-16 15:41:11 +08:00
partition_stats_meta_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
partition_stats_meta.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
policy_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
policy.go
fix: drop collection failed if enable streaming service (#37444)
2024-11-07 10:26:26 +08:00
README.md
[skip ci]Change etcd to lowercase (#9983)
2021-10-15 18:58:37 +08:00
segment_allocation_policy_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
segment_allocation_policy.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
segment_info_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
segment_info.go
enhance: Reduce mutex contention in datacoord meta (#38219)
2025-01-15 01:15:02 +08:00
segment_manager_test.go
enhance: Read metadata concurrently to accelerate recovery (#38403)
2025-01-23 14:27:27 +08:00
segment_manager.go
fix: deleted the sealed segment data accidentally (#39421)
2025-01-20 15:29:02 +08:00
segment_operator_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
segment_operator.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
server_test.go
enhance: Read metadata concurrently to accelerate recovery (#38403)
2025-01-23 14:27:27 +08:00
server.go
enhance: Read metadata concurrently to accelerate recovery (#38403)
2025-01-23 14:27:27 +08:00
services_test.go
enhance: record memory size (uncompressed) item for index (#38770)
2025-01-14 10:33:06 +08:00
services.go
enhance: Limit number of segments restored and promptly terminate the job (#39344)
2025-01-21 15:13:13 +08:00
stats_task_meta_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
stats_task_meta.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
sync_segments_scheduler_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
sync_segments_scheduler.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
task_analyze.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
task_index.go
fix: Add scalar index engine version for compatibility (#39204)
2025-01-15 12:25:00 +08:00
task_scheduler_test.go
enhance: record memory size (uncompressed) item for index (#38770)
2025-01-14 10:33:06 +08:00
task_scheduler.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
task_stats_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
task_stats.go
fix: Record active collections for l0Policy (#39217)
2025-01-16 20:19:02 +08:00
types.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
util_test.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00
util.go
enhance: make new go package to manage proto (#39114)
2025-01-10 10:49:01 +08:00

README.md

Data Coordinator

Data cooridnator(datacoord for short) is the component to organize DataNodes and segments allocations.

Dependency

  • KV store: a kv store has all the meta info datacoord needs to operate. (etcd)
  • Message stream: a message stream to communicate statistics information with data nodes. (Pulsar)
  • Root Coordinator: timestamp, id and meta source.
  • Data Node(s): could be an instance or a cluster, actual worker group handles data modification operations.