mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-03 01:12:25 +08:00

History

enhance: Reduce get segments scan during l0 compaction (#30408 )

See also #27606

Previously l0 linear compaction will scan all target segment id from
metacache for each line of delta entry, which is not needed since
compaction target segments shall be all immutable.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

2024-02-01 10:59:03 +08:00

allocator

Replace manual composed grpc call with Broker methods (#27676 )

2023-10-13 09:55:34 +08:00

broker

enhance:don't store logPath in meta to reduce memory (#28873 )

2024-01-18 22:06:31 +08:00

importv2

feat: Add import executor and manager for datanode (#29438 )

2024-01-31 20:45:04 +08:00

enhance: Make l0 compactor download files in parallel (#30309 )

2024-01-30 10:07:09 +08:00

iterators

Add float16 approve for multi-type part (#28427 )

2024-01-11 15:48:51 +08:00

metacache

enhance: Remove current stats after RollStats action (#30391 )

2024-01-31 18:55:04 +08:00

syncmgr

feat: Add import executor and manager for datanode (#29438 )

2024-01-31 20:45:04 +08:00

writebuffer

enhance: Add MemoryHighSyncPolicy back to write buffer manager (#29997 )

2024-01-31 19:03:04 +08:00

binlog_io_test.go

enhance: make compactor use actual buffer size to decide when to sync (#29945 )

2024-01-13 01:32:52 +08:00

binlog_io.go

enhance: Add open telemetry tracing for compaction (#30168 )

2024-01-23 10:37:00 +08:00

cache_test.go

Make duplicate flush recalls success and other logic update (#16755 )

2022-05-06 17:49:51 +08:00

cache.go

Use typeutil.ConcurrentMap instead of sync.Map (#25846 )

2023-07-24 10:23:01 +08:00

channel_checkpoint_updater.go

feat: Introduce channelCheckpointUpdater to reduce goroutine use in ttNode (#28570 )

2023-12-12 13:48:42 +08:00

channel_manager_test.go

enhance: Change DN channelmanger into interface (#29307 )

2023-12-27 16:00:48 +08:00

channel_manager.go

enhance: Change DN channelmanger into interface (#29307 )

2023-12-27 16:00:48 +08:00

compaction_executor_test.go

fix: Separate schedule and check results loop (#28692 )

2023-11-29 10:50:29 +08:00

compaction_executor.go

fix: Add more throughput in related metrics (#30038 )

2024-01-19 11:34:54 +08:00

compactor_test.go

fix: the delete msg disorder issue (#29915 )

2024-01-14 10:26:52 +08:00

compactor.go

enhance: Make compaction log has traceID (#30338 )

2024-01-30 10:09:03 +08:00

data_node_test.go

enhance: Add FlowgraphManager interface (#28852 )

2023-11-30 18:42:32 +08:00

data_node.go

feat: Add import executor and manager for datanode (#29438 )

2024-01-31 20:45:04 +08:00

data_sync_service_test.go

fix: Use channel cp as the dml&start position for import segments (#30107 )

2024-01-22 14:36:55 +08:00

data_sync_service.go

enhance:don't store logPath in meta to reduce memory (#28873 )

2024-01-18 22:06:31 +08:00

event_manager_test.go

enhance: Add FlowgraphManager interface (#28852 )

2023-11-30 18:42:32 +08:00

event_manager.go

fix: panic with datanode negetive wait group counter (#30135 )

2024-01-30 18:15:04 +08:00

flow_graph_dd_node_test.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

flow_graph_dd_node.go

fix: Add more throughput in related metrics (#30038 )

2024-01-19 11:34:54 +08:00

flow_graph_dmstream_input_node_test.go

Support to replicate the mq message (#27240 )

2023-10-20 14:26:09 +08:00

flow_graph_dmstream_input_node.go

fix: add back existing datanode metrics (#29360 )

2023-12-22 14:20:43 +08:00

flow_graph_manager_test.go

enhance: Add FlowgraphManager interface (#28852 )

2023-11-30 18:42:32 +08:00

flow_graph_manager.go

enhance: Add FlowgraphManager interface (#28852 )

2023-11-30 18:42:32 +08:00

flow_graph_message_test.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

flow_graph_message.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

flow_graph_node.go

Make flow graph retry longer (#24355 )

2023-05-25 09:53:30 +08:00

flow_graph_time_tick_node.go

fix: Skip updating checkpoint after dropcollection (#29220 )

2023-12-15 16:04:45 +08:00

flow_graph_time_ticker.go

enhance: create goroutine only once (#28594 )

2023-11-22 10:26:28 +08:00

flow_graph_write_node.go

enhance: Remove commented code and fix naming issue (#28450 )

2023-11-16 00:22:20 +08:00

flush_task_counter_test.go

Fix flushManager.isFull is too slow (#28141 )

2023-11-03 14:42:17 +08:00

flush_task_counter.go

Fix flushManager.isFull is too slow (#28141 )

2023-11-03 14:42:17 +08:00

io_pool_test.go

Fix copylocks linter errors (#26217 )

2023-08-08 20:41:15 +08:00

io_pool.go

Fix getting incorrect CPU num (#28146 )

2023-11-06 06:02:16 +08:00

l0_compactor_test.go

enhance: Reduce get segments scan during l0 compaction (#30408 )

2024-02-01 10:59:03 +08:00

l0_compactor.go

enhance: Reduce get segments scan during l0 compaction (#30408 )

2024-02-01 10:59:03 +08:00

meta_service_test.go

Replace manual composed grpc call with Broker methods (#27676 )

2023-10-13 09:55:34 +08:00

meta_service.go

Replace manual composed grpc call with Broker methods (#27676 )

2023-10-13 09:55:34 +08:00

meta_util.go

Use go-api/v2 for milvus-proto (#24770 )

2023-06-09 01:28:37 +08:00

metrics_info.go

enhance: Add FlowgraphManager interface (#28852 )

2023-11-30 18:42:32 +08:00

mock_channelmanager.go

enhance: Change DN channelmanger into interface (#29307 )

2023-12-27 16:00:48 +08:00

mock_fgmanager.go

enhance: separate serializer logic from sync task (#29413 )

2023-12-26 10:40:47 +08:00

mock_test.go

enhance: make compactor use actual buffer size to decide when to sync (#29945 )

2024-01-13 01:32:52 +08:00

OWNERS

[skip ci]Update OWNERS files (#11898 )

2021-11-16 15:41:11 +08:00

rate_collector_test.go

Move some modules from internal to public package (#22572 )

2023-04-06 19:14:32 +08:00

rate_collector.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

README.md

[skip ci]Change uppercase of MinIO (#9539 )

2021-10-09 11:22:58 +08:00

services_test.go

fix: Use channel cp as the dml&start position for import segments (#30107 )

2024-01-22 14:36:55 +08:00

services.go

feat: Add import executor and manager for datanode (#29438 )

2024-01-31 20:45:04 +08:00

stats_updater_test.go

Use writebuffer, sync manager refactory in datanode (#28320 )

2023-11-15 15:24:18 +08:00

stats_updater.go

enhance:Remove msgbase unnecessary assignments (#28511 )

2023-11-24 15:02:39 +08:00

timetick_sender_test.go

Refine datanode Timetick Sender (#28393 )

2023-11-13 21:46:20 +08:00

timetick_sender.go

Refine datanode Timetick Sender (#28393 )

2023-11-13 21:46:20 +08:00

util.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

README.md

Data Node

DataNode is the component to write insert and delete messages into persistent blob storage, for example MinIO or S3.

Dependency

KV store: a kv store that persists messages into blob storage.
Message stream: receive messages and publish imformation
Root Coordinator: get the latest unique IDs.
Data Coordinator: get the flush information and which message stream to subscribe.