mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00

History

fix: Prevent clone when selecting segments from meta (#30928 )

See also #30538

Previously the `SelectSegments` changed to clone all return value
preventing possible update to returned info.

Since meta is implemented following COW rules, this shall not happen and
any update on segment shall have copy before it.

This PR:
- Remove clone for read-only Get segment info
- Add Segment Operator abstraction for changing segment
- Implemnt COW for updating MaxRowNum

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

2024-03-01 13:59:02 +08:00

broker

enhance: Move datacoord broker into separate package (#28876 )

2023-12-01 10:22:34 +08:00

allocator_test.go

Decoupling client and server API in types interface (#27186 )

2023-09-26 09:57:25 +08:00

allocator.go

enhance:Remove msgbase unnecessary assignments (#28511 )

2023-11-24 15:02:39 +08:00

build_index_policy.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

channel_checker_test.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

channel_checker.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

channel_manager_factory.go

enhance: Remove not in use policies (#29448 )

2023-12-28 10:38:46 +08:00

channel_manager_test.go

enhance: Change ChannelManager to interface (#29300 )

2023-12-25 19:24:46 +08:00

channel_manager.go

enhance:change some logs (#29579 )

2024-01-05 16:12:48 +08:00

channel_store_test.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

channel_store.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

channel.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

cluster_test.go

enhance: Change ChannelManager to interface (#29300 )

2023-12-25 19:24:46 +08:00

cluster.go

enhance: Change ChannelManager to interface (#29300 )

2023-12-25 19:24:46 +08:00

compaction_l0_view_test.go

enhance: Add force trigger (#30641 )

2024-02-19 18:40:50 +08:00

compaction_l0_view.go

enhance: Add force trigger (#30641 )

2024-02-19 18:40:50 +08:00

compaction_scheduler_test.go

enhance: Add CompactionTaskNum metrics (#29518 )

2023-12-28 15:46:55 +08:00

compaction_scheduler.go

enhance: Add CompactionTaskNum metrics (#29518 )

2023-12-28 15:46:55 +08:00

compaction_test.go

fix: Clear DN unkown compaction tasks (#30850 )

2024-03-01 11:31:00 +08:00

compaction_trigger_test.go

fix: Prevent clone when selecting segments from meta (#30928 )

2024-03-01 13:59:02 +08:00

compaction_trigger_v2_test.go

fix: Fix L0 compaction in datacoord (#28814 )

2023-12-05 18:44:37 +08:00

compaction_trigger_v2.go

fix: Fill in info in CompactionSegmentBinlogs (#30279 )

2024-01-26 10:51:02 +08:00

compaction_trigger.go

fix: Prevent clone when selecting segments from meta (#30928 )

2024-03-01 13:59:02 +08:00

compaction_view_manager_test.go

fix: Limit L0 Compaction segment size and count (#30374 )

2024-02-01 20:39:03 +08:00

compaction_view_manager.go

fix: Fix L0 compaction in datacoord (#28814 )

2023-12-05 18:44:37 +08:00

compaction_view.go

fix: Fix some left comments (#29027 )

2023-12-13 18:48:39 +08:00

compaction.go

fix: Clear DN unkown compaction tasks (#30850 )

2024-03-01 11:31:00 +08:00

const.go

enhance: pass partition key scalar info if enabled when build vector index (#29931 )

2024-01-24 00:04:55 +08:00

errors_test.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

errors.go

Refine DataCoord status (#27262 )

2023-09-26 17:15:27 +08:00

garbage_collector_test.go

enhance:don't store logPath in meta to reduce memory (#28873 )

2024-01-18 22:06:31 +08:00

garbage_collector.go

enhance: Add log when garbage collection resumed (#30535 )

2024-02-05 17:09:15 +08:00

handler.go

enhance: Explicitly pass LevelZero segment ids in vchan info (#29612 )

2024-01-04 16:46:45 +08:00

index_builder_test.go

enhance: pass partition key scalar info if enabled when build vector index (#29931 )

2024-01-24 00:04:55 +08:00

index_builder.go

fix: compatibility for diskann cache param (#30119 )

2024-02-26 16:54:55 +08:00

index_engine_version_manager_test.go

Optimize the codec code of session (#27360 )

2023-10-01 10:33:30 +08:00

index_engine_version_manager.go

feat: trigger compaction to handle index version (#28442 )

2023-11-21 09:26:22 +08:00

index_meta_test.go

feat: Support multiple vector indexes in a collection (#27700 )

2023-12-29 11:44:45 +08:00

index_meta.go

feat: Support multiple vector indexes in a collection (#27700 )

2023-12-29 11:44:45 +08:00

index_service_test.go

fix: report error if the altering index doesn't support mmap (#29832 )

2024-01-17 16:40:54 +08:00

index_service.go

fix: alter_index should return error if index not found (#30786 )

2024-02-29 15:55:01 +08:00

indexnode_manager_test.go

Refine state check (#27541 )

2023-10-11 21:01:35 +08:00

indexnode_manager.go

Decoupling client and server API in types interface (#27186 )

2023-09-26 09:57:25 +08:00

meta_test.go

fix: Prevent clone when selecting segments from meta (#30928 )

2024-03-01 13:59:02 +08:00

meta_util.go

Make VChannelInfo read compatible with 2.0.2 (#17995 )

2022-07-07 21:58:22 +08:00

meta.go

fix: Prevent clone when selecting segments from meta (#30928 )

2024-03-01 13:59:02 +08:00

metrics_info_test.go

Refine state check (#27541 )

2023-10-11 21:01:35 +08:00

metrics_info.go

disable auto balance when old node exists (#28191 )

2023-11-07 14:02:20 +08:00

mock_allocator_test.go

Organize all mockery generation commands in Makefile (#26826 )

2023-09-04 21:19:48 +08:00

mock_channel_store.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

mock_channelmanager.go

enhance: Add CompactionTaskNum metrics (#29518 )

2023-12-28 15:46:55 +08:00

mock_cluster.go

enhance: pack datacoord Cluster and SessionManager with interface and mock them (#28869 )

2023-12-11 17:52:37 +08:00

mock_compaction_meta.go

enhance: Refine compaction interfaces to support major compaction (#30632 )

2024-02-19 20:52:50 +08:00

mock_compaction_plan_context.go

fix: Separate schedule and check results loop (#28692 )

2023-11-29 10:50:29 +08:00

mock_handler.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

mock_index_engine_version_manager.go

feat: trigger compaction to handle index version (#28442 )

2023-11-21 09:26:22 +08:00

mock_scheduler.go

enhance: Add CompactionTaskNum metrics (#29518 )

2023-12-28 15:46:55 +08:00

mock_session_manager.go

fix: Clear DN unkown compaction tasks (#30850 )

2024-03-01 11:31:00 +08:00

mock_test.go

feat: Implement DescribeAlias and ListAliases interfaces (#29641 )

2024-01-11 19:12:51 +08:00

mock_trigger_manager.go

feat: Add Compaction views and triggers (#27906 )

2023-11-23 17:30:25 +08:00

OWNERS

[skip ci]Update OWNERS files (#11898 )

2021-11-16 15:41:11 +08:00

policy_test.go

enhance: Remove not in use policies (#29448 )

2023-12-28 10:38:46 +08:00

policy.go

enhance: Remove not in use policies (#29448 )

2023-12-28 10:38:46 +08:00

README.md

[skip ci]Change etcd to lowercase (#9983 )

2021-10-15 18:58:37 +08:00

segment_allocation_policy_test.go

Adjust datacoord for L0 Delta (#28021 )

2023-11-06 15:26:16 +08:00

segment_allocation_policy.go

fix: Decoupling importing segment from flush process (#30402 )

2024-02-03 13:01:12 +08:00

segment_info.go

enhance:don't store logPath in meta to reduce memory (#28873 )

2024-01-18 22:06:31 +08:00

segment_manager_test.go

fix: Decoupling importing segment from flush process (#30402 )

2024-02-03 13:01:12 +08:00

segment_manager.go

fix: Decoupling importing segment from flush process (#30402 )

2024-02-03 13:01:12 +08:00

segment_operator_test.go

fix: Prevent clone when selecting segments from meta (#30928 )

2024-03-01 13:59:02 +08:00

segment_operator.go

fix: Prevent clone when selecting segments from meta (#30928 )

2024-03-01 13:59:02 +08:00

server_test.go

fix: Decoupling importing segment from flush process (#30402 )

2024-02-03 13:01:12 +08:00

server.go

enhance: add graceful stop timeout to avoid node stop hang under extreme cases (#30317 )

2024-02-29 17:01:50 +08:00

services_test.go

fix: wrong segment binlog path cause load segment failed #30726 (#30959 )

2024-03-01 13:09:01 +08:00

services.go

fix: Prevent clone when selecting segments from meta (#30928 )

2024-03-01 13:59:02 +08:00

session_manager_test.go

enhance: Change ChannelManager to interface (#29300 )

2023-12-25 19:24:46 +08:00

session_manager.go

fix: Clear DN unkown compaction tasks (#30850 )

2024-03-01 11:31:00 +08:00

session.go

Decoupling client and server API in types interface (#27186 )

2023-09-26 09:57:25 +08:00

util_test.go

enhance: add l0 metric and fix datacoord no need drop l0 segment when flush (#28373 )

2023-11-24 15:58:24 +08:00

util.go

fix: compatibility for diskann cache param (#30119 )

2024-02-26 16:54:55 +08:00

README.md

Data Coordinator

Data cooridnator(datacoord for short) is the component to organize DataNodes and segments allocations.

Dependency

KV store: a kv store has all the meta info datacoord needs to operate. (etcd)
Message stream: a message stream to communicate statistics information with data nodes. (Pulsar)
Root Coordinator: timestamp, id and meta source.
Data Node(s): could be an instance or a cluster, actual worker group handles data modification operations.