admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2025-12-28 22:45:26 +08:00
Code Issues Packages Projects Releases Wiki Activity
milvus/internal/datacoord
History
Spade A ad8aba7cb4
feat: impl ComputePhraseMatchSlop for compute min slop for phrase match query (#45892) 
issue: https://github.com/milvus-io/milvus/issues/45890

ComputePhraseMatchSlop accepts three pararms:
1. A string: query text
2. Some trings: data texts
3. Analyzer params,

Slop will be calculated for the query text with each data text in the
context of phrase match where they are tokenized with tokenizer with
analyzer params.

So two array will be returned:
1. is_match: is phrase match can sucess
2. slop: the related slop if phrase match can sucess, or -1 is cannot.

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-12-19 16:03:18 +08:00
..
allocator
enhance: Pooling for data tasks (#41256)
2025-05-20 21:06:24 +08:00
broker
enhance: Optimize FlushAll performance for multi-table scenarios (#43339)
2025-07-30 15:37:37 +08:00
session
enhance: remove redundant channel manager from datacoord (#44532)
2025-10-09 11:01:57 +08:00
task
fix: Skip running scalar index when segment was compacted (#44690)
2025-10-11 12:05:56 +08:00
.mockery.yaml
fix: Pre-check import message to prevent pipeline block indefinitely (#42415)
2025-06-11 13:40:38 +08:00
analyze_inspector.go
fix: Fix task getting stuck after recovery (#42114)
2025-05-28 12:46:28 +08:00
analyze_meta_test.go
enhance: Use v2 package name for pkg module (#39990)
2025-02-22 23:15:58 +08:00
analyze_meta.go
enhance: Pooling for data tasks (#41256)
2025-05-20 21:06:24 +08:00
build_index_policy.go
Format the code (#27275)
2023-09-21 09:45:27 +08:00
channel.go
enhance: remove redundant channel manager from datacoord (#44532)
2025-10-09 11:01:57 +08:00
compaction_inspector_test.go
enhance: remove redundant channel manager from datacoord (#44532)
2025-10-09 11:01:57 +08:00
compaction_inspector.go
enhance: remove redundant channel manager from datacoord (#44532)
2025-10-09 11:01:57 +08:00
compaction_l0_view_test.go
feat: support mannual compact l0(#44439) (#44440)
2025-09-23 12:44:07 +08:00
compaction_l0_view.go
feat: support mannual compact l0(#44439) (#44440)
2025-09-23 12:44:07 +08:00
compaction_policy_clustering_test.go
enhance: remove uncessary segment size estimation and make it configurable (#46302)
2025-12-13 02:58:46 +08:00
compaction_policy_clustering.go
enhance: remove uncessary segment size estimation and make it configurable (#46302)
2025-12-13 02:58:46 +08:00
compaction_policy_l0_test.go
feat: support mannual compact l0(#44439) (#44440)
2025-09-23 12:44:07 +08:00
compaction_policy_l0.go
feat: support mannual compact l0(#44439) (#44440)
2025-09-23 12:44:07 +08:00
compaction_policy_single_test.go
enhance: Move sort stats task to sort compaction (#42562)
2025-07-08 20:22:47 +08:00
compaction_policy_single.go
feat: Auto add namespace field data if namespace is enabled (#44933)
2025-10-24 18:40:05 +08:00
compaction_queue_test.go
enhance: Use v2 package name for pkg module (#39990)
2025-02-22 23:15:58 +08:00
compaction_queue.go
enhance: Use v2 package name for pkg module (#39990)
2025-02-22 23:15:58 +08:00
compaction_task_clustering_test.go
fix: Set task init when worker doesn't have task (#45675)
2025-11-19 18:03:07 +08:00
compaction_task_clustering.go
enhance: integrate Storage V2 FFI interface for unified storage access (#45723)
2025-11-24 19:57:07 +08:00
compaction_task_l0_test.go
fix: Set task init when worker doesn't have task (#45675)
2025-11-19 18:03:07 +08:00
compaction_task_l0.go
enhance: integrate Storage V2 FFI interface for unified storage access (#45723)
2025-11-24 19:57:07 +08:00
compaction_task_meta_test.go
enhance: Use v2 package name for pkg module (#39990)
2025-02-22 23:15:58 +08:00
compaction_task_meta.go
fix: Fix invalid pre-allocated segment IDs (#44350)
2025-09-15 10:27:58 +08:00
compaction_task_mix_test.go
enhance: remove redundant channel manager from datacoord (#44532)
2025-10-09 11:01:57 +08:00
compaction_task_mix.go
enhance: Add log with segment size for tasks (#46118)
2025-12-05 16:45:11 +08:00
compaction_task.go
fix: Accidentally ignored sealed segments in L0 Compaction (#45340)
2025-11-06 16:53:38 +08:00
compaction_trigger_test.go
enhance: remove uncessary segment size estimation and make it configurable (#46302)
2025-12-13 02:58:46 +08:00
compaction_trigger_v2_test.go
feat: support mannual compact l0(#44439) (#44440)
2025-09-23 12:44:07 +08:00
compaction_trigger_v2.go
feat: Auto add namespace field data if namespace is enabled (#44933)
2025-10-24 18:40:05 +08:00
compaction_trigger.go
enhance: remove uncessary segment size estimation and make it configurable (#46302)
2025-12-13 02:58:46 +08:00
compaction_util.go
feat: Enable to backup and reload ez (#46332)
2025-12-16 17:19:16 +08:00
compaction_view.go
feat: support mannual compact l0(#44439) (#44440)
2025-09-23 12:44:07 +08:00
const.go
enhance: pass partition key scalar info if enabled when build vector index (#29931)
2024-01-24 00:04:55 +08:00
create_meta_test.go
enhance: Pooling for data tasks (#41256)
2025-05-20 21:06:24 +08:00
ddl_callbacks_alter_index.go
enhance: support collection and index with WAL-based DDL framework (#45033)
2025-10-30 14:24:08 +08:00
ddl_callbacks_create_index.go
enhance: support collection and index with WAL-based DDL framework (#45033)
2025-10-30 14:24:08 +08:00
ddl_callbacks_drop_index.go
enhance: support collection and index with WAL-based DDL framework (#45033)
2025-10-30 14:24:08 +08:00
ddl_callbacks_flushall.go
enhance: Implement new FlushAllMessage and refactor flush all (#45920)
2025-12-10 19:27:13 +08:00
ddl_callbacks.go
enhance: Implement new FlushAllMessage and refactor flush all (#45920)
2025-12-10 19:27:13 +08:00
errors_test.go
enhance: Use v2 package name for pkg module (#39990)
2025-02-22 23:15:58 +08:00
errors.go
Refine DataCoord status (#27262)
2023-09-26 17:15:27 +08:00
external_collection_inspector.go
enhance: implement external collection update task with source change detection (#45690)
2025-11-27 15:33:08 +08:00
external_collection_task_meta.go
enhance: implement external collection update task with source change detection (#45690)
2025-11-27 15:33:08 +08:00
file_resource_manager_test.go
enhance: sync file resource to querynode and datanode (#44480)
2025-12-04 16:23:11 +08:00
file_resource_manager.go
enhance: sync file resource to querynode and datanode (#44480)
2025-12-04 16:23:11 +08:00
garbage_collector_test.go
enhance: support pause GC at collection level (#45943)
2025-12-08 19:33:15 +08:00
garbage_collector.go
fix: stack overflow when gc json or json key (#46317)
2025-12-12 20:05:15 +08:00
go_channel_singleton.go
fix: Fix improper use of offset in HybridSearch (#36244)
2024-09-13 22:05:15 +08:00
handler_test.go
enhance: support collection and index with WAL-based DDL framework (#45033)
2025-10-30 14:24:08 +08:00
handler.go
enhance: support AckSyncUp for broadcaster, and enable it in truncate api (#46313)
2025-12-17 16:55:17 +08:00
import_checker_test.go
enhance: sync file resource to querynode and datanode (#44480)
2025-12-04 16:23:11 +08:00
import_checker.go
enhance: Skip redundant failure marking for completed import jobs (#45767)
2025-11-25 11:21:07 +08:00
import_inspector_test.go
enhance: sync file resource to querynode and datanode (#44480)
2025-12-04 16:23:11 +08:00
import_inspector.go
enhance: Move sort stats task to sort compaction (#42562)
2025-07-08 20:22:47 +08:00
import_job.go
enhance: Show create time for import job (#45058)
2025-10-27 12:14:08 +08:00
import_meta_test.go
enhance: Pooling for data tasks (#41256)
2025-05-20 21:06:24 +08:00
import_meta.go
enhance: Enhance import context (#42021)
2025-05-23 12:58:27 +08:00
import_task_import_test.go
enhance: Enhance import context (#42021)
2025-05-23 12:58:27 +08:00
import_task_import.go
enhance: support useLoonFFI flag in import workflow (#46363)
2025-12-17 16:35:16 +08:00
import_task_preimport_test.go
enhance: Enhance import context (#42021)
2025-05-23 12:58:27 +08:00
import_task_preimport.go
fix: Fix task state is InProgress but payload is nil (#43777)
2025-08-11 14:13:42 +08:00
import_task.go
enhance: Move sort stats task to sort compaction (#42562)
2025-07-08 20:22:47 +08:00
import_util_test.go
enhance: sync file resource to querynode and datanode (#44480)
2025-12-04 16:23:11 +08:00
import_util.go
enhance: support useLoonFFI flag in import workflow (#46363)
2025-12-17 16:35:16 +08:00
index_engine_version_manager_test.go
enhance: Implement rewatch mechanism for etcd failure scenarios (#43829)
2025-08-14 10:31:44 +08:00
index_engine_version_manager.go
enhance: Implement rewatch mechanism for etcd failure scenarios (#43829)
2025-08-14 10:31:44 +08:00
index_inspector_test.go
enhance: override index_type while creating segment index (#45416)
2025-11-11 07:27:36 +08:00
index_inspector.go
enhance: Add log with segment size for tasks (#46118)
2025-12-05 16:45:11 +08:00
index_meta_test.go
fix: ddl framework bug patch (#45290)
2025-11-04 22:25:33 +08:00
index_meta.go
enhance: override index_type while creating segment index (#45416)
2025-11-11 07:27:36 +08:00
index_service_test.go
fix: support upgrading from 2.6.x to 2.6.5 (#45264)
2025-11-04 18:31:32 +08:00
index_service.go
enhance: override index_type while creating segment index (#45416)
2025-11-11 07:27:36 +08:00
knapsack_test.go
enhance: refine compaction trigger to reduce read/write amplifaction(#41336) (#41728)
2025-06-04 11:24:38 +08:00
knapsack.go
enhance: refine compaction trigger to reduce read/write amplifaction(#41336) (#41728)
2025-06-04 11:24:38 +08:00
meta_test.go
feat: support TruncateCollection api to clear collection data (#46167)
2025-12-12 10:31:14 +08:00
meta_util.go
fix: make savebinlogpath idompotent at binlog level (#43615)
2025-07-29 19:47:36 +08:00
meta.go
feat: support TruncateCollection api to clear collection data (#46167)
2025-12-12 10:31:14 +08:00
metrics_info_test.go
enhance: sync file resource to querynode and datanode (#44480)
2025-12-04 16:23:11 +08:00
metrics_info.go
enhance: remove redundant channel manager from datacoord (#44532)
2025-10-09 11:01:57 +08:00
mock_compaction_inspector.go
enhance: Move sort stats task to sort compaction (#42562)
2025-07-08 20:22:47 +08:00
mock_compaction_meta.go
enhance: Fail compaction task to prevent data loss (#43545)
2025-07-25 16:24:54 +08:00
mock_handler.go
feat: Support add_function/alter_function/drop_function (#44895)
2025-11-13 20:53:39 +08:00
mock_import_meta.go
fix: Pre-check import message to prevent pipeline block indefinitely (#42415)
2025-06-11 13:40:38 +08:00
mock_index_engine_version_manager.go
enhance: Pooling for data tasks (#41256)
2025-05-20 21:06:24 +08:00
mock_segment_manager.go
feat: add NotifyDropPartition in mixcoord for droppartition in dc (#42029)
2025-05-23 18:32:26 +08:00
mock_stats_job_manager.go
enhance: Use v2 package name for pkg module (#39990)
2025-02-22 23:15:58 +08:00
mock_sub_cluster.go
enhance: Pooling for data tasks (#41256)
2025-05-20 21:06:24 +08:00
mock_test.go
feat: impl ComputePhraseMatchSlop for compute min slop for phrase match query (#45892)
2025-12-19 16:03:18 +08:00
mock_trigger_manager.go
feat: support mannual compact l0(#44439) (#44440)
2025-09-23 12:44:07 +08:00
mock_trigger.go
enhance: Pooling for data tasks (#41256)
2025-05-20 21:06:24 +08:00
OWNERS
[skip ci]Update OWNERS files (#11898)
2021-11-16 15:41:11 +08:00
partition_stats_meta_test.go
enhance: Use v2 package name for pkg module (#39990)
2025-02-22 23:15:58 +08:00
partition_stats_meta.go
enhance: Use v2 package name for pkg module (#39990)
2025-02-22 23:15:58 +08:00
README.md
[skip ci]Change etcd to lowercase (#9983)
2021-10-15 18:58:37 +08:00
segment_allocation_policy_test.go
enhance: remove uncessary segment size estimation and make it configurable (#46302)
2025-12-13 02:58:46 +08:00
segment_allocation_policy.go
enhance: remove uncessary segment size estimation and make it configurable (#46302)
2025-12-13 02:58:46 +08:00
segment_info_test.go
fix: Fix GetCompactionTo return empty results when segment was GCed (#44270)
2025-09-12 18:11:58 +08:00
segment_info.go
fix: Fix GetCompactionTo return empty results when segment was GCed (#44270)
2025-09-12 18:11:58 +08:00
segment_manager_test.go
feat: add NotifyDropPartition in mixcoord for droppartition in dc (#42029)
2025-05-23 18:32:26 +08:00
segment_manager.go
feat: add NotifyDropPartition in mixcoord for droppartition in dc (#42029)
2025-05-23 18:32:26 +08:00
segment_operator_test.go
enhance: Use v2 package name for pkg module (#39990)
2025-02-22 23:15:58 +08:00
segment_operator.go
enhance: Add json key inverted index in stats for optimization (#38039)
2025-04-10 15:20:28 +08:00
server_test.go
enhance: support AckSyncUp for broadcaster, and enable it in truncate api (#46313)
2025-12-17 16:55:17 +08:00
server.go
fix: wrong context using by session of grpc client (#46183)
2025-12-08 21:47:12 +08:00
services_test.go
enhance: Return FlushAllMsg in response (#46347)
2025-12-16 10:35:16 +08:00
services.go
enhance: Return FlushAllMsg in response (#46347)
2025-12-16 10:35:16 +08:00
stats_inspector_test.go
enhance: Move sort stats task to sort compaction (#42562)
2025-07-08 20:22:47 +08:00
stats_inspector.go
enhance: support json stats with shredding design (#42534)
2025-09-01 10:49:52 +08:00
stats_task_meta_test.go
enhance: Move sort stats task to sort compaction (#42562)
2025-07-08 20:22:47 +08:00
stats_task_meta.go
enhance: Move sort stats task to sort compaction (#42562)
2025-07-08 20:22:47 +08:00
task_analyze_test.go
feat: Add CMEK cipher plugin (#43722)
2025-08-27 11:15:52 +08:00
task_analyze.go
feat: Add CMEK cipher plugin (#43722)
2025-08-27 11:15:52 +08:00
task_index_test.go
enhance: Pooling for data tasks (#41256)
2025-05-20 21:06:24 +08:00
task_index.go
enhance: Resolve issues integrating loon FFI (#45918)
2025-11-28 18:41:08 +08:00
task_stats_test.go
enhance: Move sort stats task to sort compaction (#42562)
2025-07-08 20:22:47 +08:00
task_stats.go
enhance: pass manifest path to stats task for storage v2 support (#46350)
2025-12-16 11:11:16 +08:00
task_update_external_collection.go
enhance: implement external collection update task with source change detection (#45690)
2025-11-27 15:33:08 +08:00
util_test.go
enhance: Use v2 package name for pkg module (#39990)
2025-02-22 23:15:58 +08:00
util.go
enhance: support AckSyncUp for broadcaster, and enable it in truncate api (#46313)
2025-12-17 16:55:17 +08:00

README.md

Data Coordinator

Data cooridnator(datacoord for short) is the component to organize DataNodes and segments allocations.

Dependency

  • KV store: a kv store has all the meta info datacoord needs to operate. (etcd)
  • Message stream: a message stream to communicate statistics information with data nodes. (Pulsar)
  • Root Coordinator: timestamp, id and meta source.
  • Data Node(s): could be an instance or a cluster, actual worker group handles data modification operations.