mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-03 01:12:25 +08:00

History

fix: Donot set LogPath when executing compaction (#30537 )

Compaction would copy logPaths from comapctFrom segA to compactTo segB,
and previous code would copy the logPath directly, causing there're
full-logPaths-of-segA in compactTo segB's meta. So, for the next
compaction of segB, if segA has been GCed, Download would report error
"The sperified key not found".

This PR makes sure compactTo segment's meta contains logID only. And
this PR also refines CompleteComapctionMutation, increasing some
readability and merge two methods into one.

See also: #30496

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

2024-02-28 19:03:01 +08:00

broker

enhance: Move datacoord broker into separate package (#28876 )

2023-12-01 10:22:34 +08:00

allocator_test.go

Decoupling client and server API in types interface (#27186 )

2023-09-26 09:57:25 +08:00

allocator.go

enhance:Remove msgbase unnecessary assignments (#28511 )

2023-11-24 15:02:39 +08:00

build_index_policy.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

channel_checker_test.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

channel_checker.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

channel_manager_factory.go

enhance: Remove not in use policies (#29448 )

2023-12-28 10:38:46 +08:00

channel_manager_test.go

enhance: Change ChannelManager to interface (#29300 )

2023-12-25 19:24:46 +08:00

channel_manager.go

enhance:change some logs (#29579 )

2024-01-05 16:12:48 +08:00

channel_store_test.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

channel_store.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

channel.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

cluster_test.go

enhance: Change ChannelManager to interface (#29300 )

2023-12-25 19:24:46 +08:00

cluster.go

enhance: Change ChannelManager to interface (#29300 )

2023-12-25 19:24:46 +08:00

compaction_l0_view_test.go

enhance: Add force trigger (#30641 )

2024-02-19 18:40:50 +08:00

compaction_l0_view.go

enhance: Add force trigger (#30641 )

2024-02-19 18:40:50 +08:00

compaction_scheduler_test.go

enhance: Add CompactionTaskNum metrics (#29518 )

2023-12-28 15:46:55 +08:00

compaction_scheduler.go

enhance: Add CompactionTaskNum metrics (#29518 )

2023-12-28 15:46:55 +08:00

compaction_test.go

fix: Donot set LogPath when executing compaction (#30537 )

2024-02-28 19:03:01 +08:00

compaction_trigger_test.go

fix: donot set l0 segment as growing when savebinlogs (#29194 )

2024-02-04 10:21:06 +08:00

compaction_trigger_v2_test.go

fix: Fix L0 compaction in datacoord (#28814 )

2023-12-05 18:44:37 +08:00

compaction_trigger_v2.go

fix: Fill in info in CompactionSegmentBinlogs (#30279 )

2024-01-26 10:51:02 +08:00

compaction_trigger.go

fix: donot set l0 segment as growing when savebinlogs (#29194 )

2024-02-04 10:21:06 +08:00

compaction_view_manager_test.go

fix: Limit L0 Compaction segment size and count (#30374 )

2024-02-01 20:39:03 +08:00

compaction_view_manager.go

fix: Fix L0 compaction in datacoord (#28814 )

2023-12-05 18:44:37 +08:00

compaction_view.go

fix: Fix some left comments (#29027 )

2023-12-13 18:48:39 +08:00

compaction.go

enhance: Refine compaction interfaces to support major compaction (#30632 )

2024-02-19 20:52:50 +08:00

const.go

enhance: pass partition key scalar info if enabled when build vector index (#29931 )

2024-01-24 00:04:55 +08:00

errors_test.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

errors.go

Refine DataCoord status (#27262 )

2023-09-26 17:15:27 +08:00

garbage_collector_test.go

enhance:don't store logPath in meta to reduce memory (#28873 )

2024-01-18 22:06:31 +08:00

garbage_collector.go

enhance: Add log when garbage collection resumed (#30535 )

2024-02-05 17:09:15 +08:00

handler.go

enhance: Explicitly pass LevelZero segment ids in vchan info (#29612 )

2024-01-04 16:46:45 +08:00

index_builder_test.go

enhance: pass partition key scalar info if enabled when build vector index (#29931 )

2024-01-24 00:04:55 +08:00

index_builder.go

fix: compatibility for diskann cache param (#30119 )

2024-02-26 16:54:55 +08:00

index_engine_version_manager_test.go

Optimize the codec code of session (#27360 )

2023-10-01 10:33:30 +08:00

index_engine_version_manager.go

feat: trigger compaction to handle index version (#28442 )

2023-11-21 09:26:22 +08:00

index_meta_test.go

feat: Support multiple vector indexes in a collection (#27700 )

2023-12-29 11:44:45 +08:00

index_meta.go

feat: Support multiple vector indexes in a collection (#27700 )

2023-12-29 11:44:45 +08:00

index_service_test.go

fix: report error if the altering index doesn't support mmap (#29832 )

2024-01-17 16:40:54 +08:00

index_service.go

enhance: pass partition key scalar info if enabled when build vector index (#29931 )

2024-01-24 00:04:55 +08:00

indexnode_manager_test.go

Refine state check (#27541 )

2023-10-11 21:01:35 +08:00

indexnode_manager.go

Decoupling client and server API in types interface (#27186 )

2023-09-26 09:57:25 +08:00

meta_test.go

fix: Donot set LogPath when executing compaction (#30537 )

2024-02-28 19:03:01 +08:00

meta_util.go

Make VChannelInfo read compatible with 2.0.2 (#17995 )

2022-07-07 21:58:22 +08:00

meta.go

fix: Donot set LogPath when executing compaction (#30537 )

2024-02-28 19:03:01 +08:00

metrics_info_test.go

Refine state check (#27541 )

2023-10-11 21:01:35 +08:00

metrics_info.go

disable auto balance when old node exists (#28191 )

2023-11-07 14:02:20 +08:00

mock_allocator_test.go

Organize all mockery generation commands in Makefile (#26826 )

2023-09-04 21:19:48 +08:00

mock_channel_store.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

mock_channelmanager.go

enhance: Add CompactionTaskNum metrics (#29518 )

2023-12-28 15:46:55 +08:00

mock_cluster.go

enhance: pack datacoord Cluster and SessionManager with interface and mock them (#28869 )

2023-12-11 17:52:37 +08:00

mock_compaction_meta.go

enhance: Refine compaction interfaces to support major compaction (#30632 )

2024-02-19 20:52:50 +08:00

mock_compaction_plan_context.go

fix: Separate schedule and check results loop (#28692 )

2023-11-29 10:50:29 +08:00

mock_handler.go

Change channel to Interface (#27839 )

2023-11-13 11:16:18 +08:00

mock_index_engine_version_manager.go

feat: trigger compaction to handle index version (#28442 )

2023-11-21 09:26:22 +08:00

mock_scheduler.go

enhance: Add CompactionTaskNum metrics (#29518 )

2023-12-28 15:46:55 +08:00

mock_session_manager.go

enhance: Add open telemetry tracing for compaction (#30168 )

2024-01-23 10:37:00 +08:00

mock_test.go

feat: Implement DescribeAlias and ListAliases interfaces (#29641 )

2024-01-11 19:12:51 +08:00

mock_trigger_manager.go

feat: Add Compaction views and triggers (#27906 )

2023-11-23 17:30:25 +08:00

OWNERS

[skip ci]Update OWNERS files (#11898 )

2021-11-16 15:41:11 +08:00

policy_test.go

enhance: Remove not in use policies (#29448 )

2023-12-28 10:38:46 +08:00

policy.go

enhance: Remove not in use policies (#29448 )

2023-12-28 10:38:46 +08:00

README.md

[skip ci]Change etcd to lowercase (#9983 )

2021-10-15 18:58:37 +08:00

segment_allocation_policy_test.go

Adjust datacoord for L0 Delta (#28021 )

2023-11-06 15:26:16 +08:00

segment_allocation_policy.go

fix: Decoupling importing segment from flush process (#30402 )

2024-02-03 13:01:12 +08:00

segment_info.go

enhance:don't store logPath in meta to reduce memory (#28873 )

2024-01-18 22:06:31 +08:00

segment_manager_test.go

fix: Decoupling importing segment from flush process (#30402 )

2024-02-03 13:01:12 +08:00

segment_manager.go

fix: Decoupling importing segment from flush process (#30402 )

2024-02-03 13:01:12 +08:00

server_test.go

fix: Decoupling importing segment from flush process (#30402 )

2024-02-03 13:01:12 +08:00

server.go

fix: Skip filling segmentID in indexBuildCh to prevent flush blocked (#30747 )

2024-02-22 20:40:53 +08:00

services_test.go

fix: [skip-e2e] Fix unstable SaveBinlogPath ut (#30508 )

2024-02-05 16:00:59 +08:00

services.go

fix: Donot set LogPath when executing compaction (#30537 )

2024-02-28 19:03:01 +08:00

session_manager_test.go

enhance: Change ChannelManager to interface (#29300 )

2023-12-25 19:24:46 +08:00

session_manager.go

enhance: Add open telemetry tracing for compaction (#30168 )

2024-01-23 10:37:00 +08:00

session.go

Decoupling client and server API in types interface (#27186 )

2023-09-26 09:57:25 +08:00

util_test.go

enhance: add l0 metric and fix datacoord no need drop l0 segment when flush (#28373 )

2023-11-24 15:58:24 +08:00

util.go

fix: compatibility for diskann cache param (#30119 )

2024-02-26 16:54:55 +08:00

README.md

Data Coordinator

Data cooridnator(datacoord for short) is the component to organize DataNodes and segments allocations.

Dependency

KV store: a kv store has all the meta info datacoord needs to operate. (etcd)
Message stream: a message stream to communicate statistics information with data nodes. (Pulsar)
Root Coordinator: timestamp, id and meta source.
Data Node(s): could be an instance or a cluster, actual worker group handles data modification operations.