mirror of https://gitee.com/milvus-io/milvus.git synced 2025-12-06 17:18:35 +08:00

History

Patrick Weizhi Xu 52ae47c850

enhance: gather materialized view search info once per request (#31996 )

issue: #29892 

This PR:
1. Move the process of gathering materialized search info to when the
search plan is created, before it goes to each segment, to avoid
repeated work and access the plan node under multi-threaded
circumstances.
2. Enforce the supported MV type to `VARCHAR`
3. Add integration test

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>

2024-04-11 15:21:19 +08:00

broker

enhance: index meta use independent rather than global meta lock (#30869 )

2024-03-04 16:56:59 +08:00

allocator_test.go

Decoupling client and server API in types interface (#27186 )

2023-09-26 09:57:25 +08:00

allocator.go

feat: Add import scheduler and manager (#29367 )

2024-03-01 18:31:02 +08:00

build_index_policy.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

channel_checker_test.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

channel_checker.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

channel_manager_factory.go

enhance: Remove not in use policies (#29448 )

2023-12-28 10:38:46 +08:00

channel_manager_test.go

enhance: Change ChannelManager to interface (#29300 )

2023-12-25 19:24:46 +08:00

channel_manager.go

enhance:change some logs (#29579 )

2024-01-05 16:12:48 +08:00

channel_store_test.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

channel_store.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

channel.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

cluster_test.go

enhance: Remove import v1 (#31403 )

2024-03-22 15:29:09 +08:00

cluster.go

enhance: Remove import v1 (#31403 )

2024-03-22 15:29:09 +08:00

compaction_l0_view_test.go

enhance: Add force trigger (#30641 )

2024-02-19 18:40:50 +08:00

compaction_l0_view.go

fix: Trigger l0 compaction when l0 views don't change (#30729 )

2024-03-05 16:37:00 +08:00

compaction_scheduler_test.go

enhance: Add CompactionTaskNum metrics (#29518 )

2023-12-28 15:46:55 +08:00

compaction_scheduler.go

fix: Trigger l0 compaction when l0 views don't change (#30729 )

2024-03-05 16:37:00 +08:00

compaction_test.go

fix: [cherry-pick]Skip changing meta if nodeID not match with channel (#31672 )

2024-04-10 15:09:18 +08:00

compaction_trigger_test.go

enhance: Prevent the backlog of channelCP update tasks, perform batch updates of channelCPs (#30941 )

2024-03-07 20:39:02 +08:00

compaction_trigger_v2_test.go

fix: Skip to submit l0 tasks when scheduler full (#31270 )

2024-03-15 10:21:12 +08:00

compaction_trigger_v2.go

fix: Skip to submit l0 tasks when scheduler full (#31270 )

2024-03-15 10:21:12 +08:00

compaction_trigger.go

feat: Add global mmap enable configuration (#31267 )

2024-03-18 15:17:10 +08:00

compaction_view_manager_test.go

fix: Correct the last empty l0 views (#31198 )

2024-03-14 10:31:04 +08:00

compaction_view_manager.go

fix: Correct the last empty l0 views (#31198 )

2024-03-14 10:31:04 +08:00

compaction_view.go

fix: Trigger l0 compaction when l0 views don't change (#30729 )

2024-03-05 16:37:00 +08:00

compaction.go

fix: [cherry-pick]Skip changing meta if nodeID not match with channel (#31672 )

2024-04-10 15:09:18 +08:00

const.go

enhance: pass partition key scalar info if enabled when build vector index (#29931 )

2024-01-24 00:04:55 +08:00

errors_test.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

errors.go

Refine DataCoord status (#27262 )

2023-09-26 17:15:27 +08:00

garbage_collector_test.go

enhance: Remove import v1 (#31403 )

2024-03-22 15:29:09 +08:00

garbage_collector.go

fix: Use correct ticker for gc work defer (#31456 )

2024-03-20 22:17:06 +08:00

handler.go

enhance: Add AllPartitionsID const to replace InvalidPartitionID (#31438 )

2024-03-20 19:01:05 +08:00

import_checker_test.go

enhance: Ensure ImportV2 waits for the index to be built and refine some logic (#31629 )

2024-04-01 20:09:13 +08:00

import_checker.go

enhance: Ensure ImportV2 waits for the index to be built and refine some logic (#31629 )

2024-04-01 20:09:13 +08:00

import_job.go

enhance: Return more fields in import progress response (#31539 )

2024-03-24 21:57:06 +08:00

import_meta_test.go

feat: Add import scheduler and manager (#29367 )

2024-03-01 18:31:02 +08:00

import_meta.go

feat: Add import scheduler and manager (#29367 )

2024-03-01 18:31:02 +08:00

import_scheduler_test.go

enhance: Ensure ImportV2 waits for the index to be built and refine some logic (#31629 )

2024-04-01 20:09:13 +08:00

import_scheduler.go

enhance: Ensure ImportV2 waits for the index to be built and refine some logic (#31629 )

2024-04-01 20:09:13 +08:00

import_task.go

enhance: Enhance and modify the return content of ImportV2 (#31192 )

2024-03-13 19:51:03 +08:00

import_util_test.go

enhance: Ensure ImportV2 waits for the index to be built and refine some logic (#31629 )

2024-04-01 20:09:13 +08:00

import_util.go

enhance: Ensure ImportV2 waits for the index to be built and refine some logic (#31629 )

2024-04-01 20:09:13 +08:00

index_builder_test.go

enhance: gather materialized view search info once per request (#31996 )

2024-04-11 15:21:19 +08:00

index_builder.go

enhance: gather materialized view search info once per request (#31996 )

2024-04-11 15:21:19 +08:00

index_engine_version_manager_test.go

Optimize the codec code of session (#27360 )

2023-10-01 10:33:30 +08:00

index_engine_version_manager.go

feat: trigger compaction to handle index version (#28442 )

2023-11-21 09:26:22 +08:00

index_meta_test.go

enhance: Ensure ImportV2 waits for the index to be built and refine some logic (#31629 )

2024-04-01 20:09:13 +08:00

index_meta.go

enhance: Ensure ImportV2 waits for the index to be built and refine some logic (#31629 )

2024-04-01 20:09:13 +08:00

index_service_test.go

enhance: Add ListIndexes API from datacoord (#31104 )

2024-03-07 17:37:01 +08:00

index_service.go

feat: Add global mmap enable configuration (#31267 )

2024-03-18 15:17:10 +08:00

indexnode_manager_test.go

Refine state check (#27541 )

2023-10-11 21:01:35 +08:00

indexnode_manager.go

Decoupling client and server API in types interface (#27186 )

2023-09-26 09:57:25 +08:00

meta_test.go

enhance: add db label on binlog size metrics (#32003 )

2024-04-10 21:01:20 +08:00

meta_util.go

Make VChannelInfo read compatible with 2.0.2 (#17995 )

2022-07-07 21:58:22 +08:00

meta.go

enhance: add db label on binlog size metrics (#32003 )

2024-04-10 21:01:20 +08:00

metrics_info_test.go

Refine state check (#27541 )

2023-10-11 21:01:35 +08:00

metrics_info.go

enhance: add the related data size for the read apis (#31816 )

2024-04-10 15:07:17 +08:00

mock_allocator_test.go

enhance: index meta use independent rather than global meta lock (#30869 )

2024-03-04 16:56:59 +08:00

mock_channel_store.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

mock_channelmanager.go

enhance: Add CompactionTaskNum metrics (#29518 )

2023-12-28 15:46:55 +08:00

mock_cluster.go

fix: add lazy load field to mark segment load type (#31591 )

2024-03-28 11:23:10 +08:00

mock_compaction_meta.go

enhance: Refine compaction interfaces to support major compaction (#30632 )

2024-02-19 20:52:50 +08:00

mock_compaction_plan_context.go

fix: Separate schedule and check results loop (#28692 )

2023-11-29 10:50:29 +08:00

mock_handler.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

mock_index_engine_version_manager.go

feat: trigger compaction to handle index version (#28442 )

2023-11-21 09:26:22 +08:00

mock_scheduler.go

enhance: Add CompactionTaskNum metrics (#29518 )

2023-12-28 15:46:55 +08:00

mock_segment_manager.go

enhance: index meta use independent rather than global meta lock (#30869 )

2024-03-04 16:56:59 +08:00

mock_session_manager.go

fix: add lazy load field to mark segment load type (#31591 )

2024-03-28 11:23:10 +08:00

mock_test.go

enhance: Remove import v1 (#31403 )

2024-03-22 15:29:09 +08:00

mock_trigger_manager.go

feat: Add Compaction views and triggers (#27906 )

2023-11-23 17:30:25 +08:00

OWNERS

[skip ci]Update OWNERS files (#11898 )

2021-11-16 15:41:11 +08:00

policy_test.go

enhance: Remove not in use policies (#29448 )

2023-12-28 10:38:46 +08:00

policy.go

enhance: Remove not in use policies (#29448 )

2023-12-28 10:38:46 +08:00

README.md

[skip ci]Change etcd to lowercase (#9983 )

2021-10-15 18:58:37 +08:00

segment_allocation_policy_test.go

Adjust datacoord for L0 Delta (#28021 )

2023-11-06 15:26:16 +08:00

segment_allocation_policy.go

fix: Decoupling importing segment from flush process (#30402 )

2024-02-03 13:01:12 +08:00

segment_info_test.go

fix: starve lock caused by slow GetCompactionTo method when too much segments (#30963 )

2024-03-05 10:04:59 +08:00

segment_info.go

enhance: Remove import v1 (#31403 )

2024-03-22 15:29:09 +08:00

segment_manager_test.go

feat: add more operation detail info for better allocation (#30438 )

2024-03-28 06:33:11 +08:00

segment_manager.go

enhance: Ensure ImportV2 waits for the index to be built and refine some logic (#31629 )

2024-04-01 20:09:13 +08:00

segment_operator_test.go

fix: Prevent clone when selecting segments from meta (#30928 )

2024-03-01 13:59:02 +08:00

segment_operator.go

fix: Prevent clone when selecting segments from meta (#30928 )

2024-03-01 13:59:02 +08:00

server_test.go

fix: Make coordinator Register not blocked on ProcessActiveStandby (#32069 )

2024-04-10 18:49:18 +08:00

server.go

fix: Make coordinator Register not blocked on ProcessActiveStandby (#32069 )

2024-04-10 18:49:18 +08:00

services_test.go

enhance: Remove import v1 (#31403 )

2024-03-22 15:29:09 +08:00

services.go

enhance: Remove the storage info report (#31772 )

2024-04-01 20:50:59 -07:00

session_manager_test.go

feat: Add import scheduler and manager (#29367 )

2024-03-01 18:31:02 +08:00

session_manager.go

feat: add more operation detail info for better allocation (#30438 )

2024-03-28 06:33:11 +08:00

session.go

Decoupling client and server API in types interface (#27186 )

2023-09-26 09:57:25 +08:00

util_test.go

feat: Add import scheduler and manager (#29367 )

2024-03-01 18:31:02 +08:00

util.go

enhance: gather materialized view search info once per request (#31996 )

2024-04-11 15:21:19 +08:00

README.md

Data Coordinator

Data cooridnator(datacoord for short) is the component to organize DataNodes and segments allocations.

Dependency

KV store: a kv store has all the meta info datacoord needs to operate. (etcd)
Message stream: a message stream to communicate statistics information with data nodes. (Pulsar)
Root Coordinator: timestamp, id and meta source.
Data Node(s): could be an instance or a cluster, actual worker group handles data modification operations.