mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00

History

enhance: [2.4] Make timeout work for each GetSegmentInfo req (#38132 )

Cherry pick from master
pr: #36026
Relate: #36025

Fix datanode watch channel timeout when segment number is too large

Previous timeout apply for whole process for batching fetch segment
info, when segment number is large one rpc timeout does not work well
for multiple round rpc case

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>

2024-12-02 20:14:39 +08:00

allocator

Replace manual composed grpc call with Broker methods (#27676 )

2023-10-13 09:55:34 +08:00

broker

enhance: [2.4] Make timeout work for each GetSegmentInfo req (#38132 )

2024-12-02 20:14:39 +08:00

compaction

enhance: compaction performance by remove paramtable get (#37882 )

2024-11-28 10:38:37 +08:00

importv2

enhance: add search params in search request in restful(#36304 ) (#37673 )

2024-11-15 17:54:30 +08:00

enhance: [cherry-pick] Optimize clustering compaction (#34313 ) (#34398 )

2024-07-04 09:52:09 +08:00

iterators

enhance: [2.4] Avoid merging insert data when buffering insert msgs (#34205 )

2024-06-27 10:14:05 +08:00

metacache

enhance: Use Blocked Bloom Filter instead of basic bloom fitler impl (#34377 )

2024-07-05 17:04:10 +08:00

syncmgr

fix: [cp24]Change memoryCheck write lock to read lock (#37526 )

2024-11-15 14:42:31 +08:00

util

enhance: [cherry-pick] Move compactor into sub package (#34098 )

2024-06-25 11:16:02 +08:00

writebuffer

fix: [24]Replace outer lock with concurrent map (#37817 ) (#37897 )

2024-11-29 14:26:43 +08:00

cache_test.go

Make duplicate flush recalls success and other logic update (#16755 )

2022-05-06 17:49:51 +08:00

cache.go

Use typeutil.ConcurrentMap instead of sync.Map (#25846 )

2023-07-24 10:23:01 +08:00

channel_checkpoint_updater_test.go

enhance: Prevent the backlog of channelCP update tasks, perform batch updates of channelCPs (#30941 )

2024-03-07 20:39:02 +08:00

channel_checkpoint_updater.go

enhance: [cherry-pick] Batch pick PRs related to compaction (#34315 )

2024-07-02 09:54:07 +08:00

channel_manager_test.go

fix: [2.4]channel manager's goroutine run order (#33121 )

2024-05-21 14:31:39 +08:00

channel_manager.go

fix: watch channel stuck due to misuse of timer.Reset (#37433 ) (#37542 )

2024-11-08 18:46:27 +08:00

data_node_test.go

enhance: Remove import v1 (#31403 )

2024-03-22 15:29:09 +08:00

data_node.go

enhance: [2.4] Graceful stop flowgraph manager when stopping datanode (#36229 ) (#36358 )

2024-09-19 17:39:12 +08:00

data_sync_service_test.go

enhance: Update protobuf-go to protobuf-go v2 (#34394 ) (#35555 )

2024-08-21 18:50:58 +08:00

data_sync_service.go

enhance: Remove bf from datanode (#36367 ) (#37027 )

2024-10-22 11:15:28 +08:00

event_manager_test.go

enhance: Update protobuf-go to protobuf-go v2 (#34394 ) (#35555 )

2024-08-21 18:50:58 +08:00

event_manager.go

enhance: Update protobuf-go to protobuf-go v2 (#34394 ) (#35555 )

2024-08-21 18:50:58 +08:00

flow_graph_dd_node_test.go

enhance: Update protobuf-go to protobuf-go v2 (#34394 ) (#35555 )

2024-08-21 18:50:58 +08:00

flow_graph_dd_node.go

enhance: Update protobuf-go to protobuf-go v2 (#34394 ) (#35555 )

2024-08-21 18:50:58 +08:00

flow_graph_dmstream_input_node_test.go

enhance: move rocksmq from internal to pkg (#34165 )

2024-06-26 13:36:05 +08:00

flow_graph_dmstream_input_node.go

enhance: [cherry-pick] optimize datanode cpu usage and correct the update logic of ttchecker (#34383 )

2024-07-04 16:34:17 +08:00

flow_graph_manager_test.go

enhance: Remove rootcoord from datanode broker (#32818 )

2024-05-14 10:03:32 +08:00

flow_graph_manager.go

enhance: Make datanode exit and case TestProxy faster (#32218 )

2024-04-16 10:49:20 +08:00

flow_graph_message_test.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

flow_graph_message.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

flow_graph_node.go

Make flow graph retry longer (#24355 )

2023-05-25 09:53:30 +08:00

flow_graph_time_tick_node.go

enhance: [cherry-pick] Batch pick PRs related to compaction (#34315 )

2024-07-02 09:54:07 +08:00

flow_graph_time_ticker.go

enhance: create goroutine only once (#28594 )

2023-11-22 10:26:28 +08:00

flow_graph_write_node.go

enhance: Update protobuf-go to protobuf-go v2 (#34394 ) (#35555 )

2024-08-21 18:50:58 +08:00

flush_task_counter_test.go

Fix flushManager.isFull is too slow (#28141 )

2023-11-03 14:42:17 +08:00

flush_task_counter.go

Fix flushManager.isFull is too slow (#28141 )

2023-11-03 14:42:17 +08:00

meta_util.go

Use go-api/v2 for milvus-proto (#24770 )

2023-06-09 01:28:37 +08:00

metrics_info.go

enhance: Use struct{} for sync task future result (#32673 )

2024-04-29 10:59:26 +08:00

mock_channelmanager.go

enhance: Change DN channelmanger into interface (#29307 )

2023-12-27 16:00:48 +08:00

mock_fgmanager.go

enhance: Use struct{} for sync task future result (#32673 )

2024-04-29 10:59:26 +08:00

mock_test.go

enhance: Update protobuf-go to protobuf-go v2 (#34394 ) (#35555 )

2024-08-21 18:50:58 +08:00

OWNERS

[skip ci]Update OWNERS files (#11898 )

2021-11-16 15:41:11 +08:00

rate_collector_test.go

Move some modules from internal to public package (#22572 )

2023-04-06 19:14:32 +08:00

rate_collector.go

feat: support rate limiter based on db and partition levels (#31070 )

2024-04-12 16:01:19 +08:00

README.md

[skip ci]Change uppercase of MinIO (#9539 )

2021-10-09 11:22:58 +08:00

services_test.go

enhance: [2.4] Graceful stop flowgraph manager when stopping datanode (#36229 ) (#36358 )

2024-09-19 17:39:12 +08:00

services.go

enhance:[cherry-pick] Add compaction task slot usage logic (#34625 )

2024-07-18 09:55:43 +08:00

stats_updater_test.go

Use writebuffer, sync manager refactory in datanode (#28320 )

2023-11-15 15:24:18 +08:00

stats_updater.go

enhance: Update protobuf-go to protobuf-go v2 (#34394 ) (#35555 )

2024-08-21 18:50:58 +08:00

timetick_sender_test.go

enhance: Reduce the memory usage of the timeTickSender (#30968 )

2024-03-02 10:13:01 +08:00

timetick_sender.go

enhance: [cherry-pick] optimize datanode cpu usage and correct the update logic of ttchecker (#34383 )

2024-07-04 16:34:17 +08:00

util.go

Format the code (#27275 )

2023-09-21 09:45:27 +08:00

README.md

Data Node

DataNode is the component to write insert and delete messages into persistent blob storage, for example MinIO or S3.

Dependency

KV store: a kv store that persists messages into blob storage.
Message stream: receive messages and publish imformation
Root Coordinator: get the latest unique IDs.
Data Coordinator: get the flush information and which message stream to subscribe.