admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
milvus/internal/datanode
History
chyezh 77e123762f
enhance: add graceful stop timeout to avoid node stop hang under extreme cases (#30320) 
1. add coordinator and proxy graceful stop timeout to 5s.
3. add other work node graceful stop timeout to 900s, and we should
potentially change this to 600s when graceful stop is smooth
4. change the order of datacoord component while stop.
5. `LivenessCheck` do not perform graceful shutdown now. 

issue: https://github.com/milvus-io/milvus/issues/30310
pr: #30317
also see: https://github.com/milvus-io/milvus/pull/30306

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-01-27 08:45:02 +08:00
..
allocator
Replace manual composed grpc call with Broker methods (#27676)
2023-10-13 09:55:34 +08:00
broker
Not convert legacy error code to new merr (#28232) (#28274)
2023-11-08 19:18:20 +08:00
iterators
Add iterators (#27643)
2023-10-18 19:34:08 +08:00
metacache
Add meta cache to datanode for L0 Delta (#27768)
2023-10-23 19:42:10 +08:00
binlog_io_test.go
enhance: [Cherry-pick] make compactor use actual buffer size to decide when to sync(#29945) (#29971)
2024-01-16 12:22:52 +08:00
binlog_io.go
Add revive sub-lints and fix existing problems (#27495)
2023-10-07 20:53:38 +08:00
buffer_test.go
Format the code (#27275)
2023-09-21 09:45:27 +08:00
buffer.go
Format the code (#27275)
2023-09-21 09:45:27 +08:00
cache_test.go
Make duplicate flush recalls success and other logic update (#16755)
2022-05-06 17:49:51 +08:00
cache.go
Use typeutil.ConcurrentMap instead of sync.Map (#25846)
2023-07-24 10:23:01 +08:00
channel_checkpoint_updater.go
feat: Introduce channelCheckpointUpdater to reduce goroutine in ttNode (#29107)
2023-12-12 13:48:42 +08:00
channel_manager_test.go
[skip e2e] Fix unstable ut (#27618)
2023-10-11 07:21:32 +08:00
channel_manager.go
Refine state check (#27541)
2023-10-11 21:01:35 +08:00
channel_meta_test.go
enhance: add param for bloomfilter(#29388) (#29614)
2024-01-02 18:58:47 +08:00
channel_meta.go
fix: frequent flush cause minio rate limit (#28625)
2023-12-20 15:02:55 +08:00
compaction_executor_test.go
Format the code (#27275)
2023-09-21 09:45:27 +08:00
compaction_executor.go
Use typeutil.ConcurrentMap instead of sync.Map (#25846)
2023-07-24 10:23:01 +08:00
compactor_test.go
enhance: [Cherry-pick] make compactor use actual buffer size to decide when to sync(#29945) (#29971)
2024-01-16 12:22:52 +08:00
compactor.go
enhance: [Cherry-pick] make compactor use actual buffer size to decide when to sync(#29945) (#29971)
2024-01-16 12:22:52 +08:00
data_node_test.go
Replace manual composed grpc call with Broker methods (#27676)
2023-10-13 09:55:34 +08:00
data_node.go
enhance: add graceful stop timeout to avoid node stop hang under extreme cases (#30320)
2024-01-27 08:45:02 +08:00
data_sync_service_test.go
fix: Use channel cp as the dml&start position for import segments (#30107) (#30133)
2024-01-22 13:32:55 +08:00
data_sync_service.go
feat: Introduce channelCheckpointUpdater to reduce goroutine in ttNode (#29107)
2023-12-12 13:48:42 +08:00
event_manager_test.go
[2.3] Refine datanode Timetick Sender (#28393) (#28430)
2023-11-15 10:08:26 +08:00
event_manager.go
fix: [cherry-pick]Save lite WatchInfo into etcd in DataNode (#29751)
2024-01-10 20:48:50 +08:00
flow_graph_dd_node_test.go
Format the code (#27275)
2023-09-21 09:45:27 +08:00
flow_graph_dd_node.go
fix: [cherry-pick]Skip updating checkpoint after dropcollection (#29221)
2023-12-15 16:04:45 +08:00
flow_graph_delete_node_test.go
Format the code (#27275)
2023-09-21 09:45:27 +08:00
flow_graph_delete_node.go
fix: Print number of pks instead of delete pk val (#29653)
2024-01-04 10:02:46 +08:00
flow_graph_dmstream_input_node_test.go
Support to replicate the mq message (#27240)
2023-10-20 14:26:09 +08:00
flow_graph_dmstream_input_node.go
Enhance newDataSyncService (#27277)
2023-09-27 11:07:25 +08:00
flow_graph_insert_buffer_node_test.go
enhance: add param for bloomfilter(#29388) (#29614)
2024-01-02 18:58:47 +08:00
flow_graph_insert_buffer_node.go
enhance: Use single instance for mergedTimeTickerSender (#27730) (#28546)
2023-11-21 16:04:23 +08:00
flow_graph_manager_test.go
Not convert legacy error code to new merr (#28232) (#28274)
2023-11-08 19:18:20 +08:00
flow_graph_manager.go
Remove recollect segment stats during starting datacoord (#27410)
2023-10-16 10:26:09 +08:00
flow_graph_message_test.go
Format the code (#27275)
2023-09-21 09:45:27 +08:00
flow_graph_message.go
Format the code (#27275)
2023-09-21 09:45:27 +08:00
flow_graph_node.go
Make flow graph retry longer (#24355)
2023-05-25 09:53:30 +08:00
flow_graph_time_tick_node.go
fix: flushTs is never reset in channelMeta (#29244)
2023-12-16 14:00:38 +08:00
flow_graph_time_ticker.go
enhance: create goroutine only once(#28594) (#28609)
2023-11-22 10:26:28 +08:00
flush_manager_test.go
Not convert legacy error code to new merr (#28232) (#28274)
2023-11-08 19:18:20 +08:00
flush_manager.go
enhance: move confusing warning log to error branch (#29891)
2024-01-12 15:50:52 +08:00
flush_task_test.go
Format the code (#27275)
2023-09-21 09:45:27 +08:00
flush_task.go
Support to replicate the mq message (#27240)
2023-10-20 14:26:09 +08:00
io_pool_test.go
Fix copylocks linter errors (#26217)
2023-08-08 20:41:15 +08:00
io_pool.go
Fix getting incorrect CPU num (#28178)
2023-11-07 11:52:22 +08:00
meta_service_test.go
Replace manual composed grpc call with Broker methods (#27676)
2023-10-13 09:55:34 +08:00
meta_service.go
Replace manual composed grpc call with Broker methods (#27676)
2023-10-13 09:55:34 +08:00
meta_util.go
Use go-api/v2 for milvus-proto (#24770)
2023-06-09 01:28:37 +08:00
metrics_info.go
Refine state check (#27541)
2023-10-11 21:01:35 +08:00
mock_test.go
enhance: [Cherry-pick] make compactor use actual buffer size to decide when to sync(#29945) (#29971)
2024-01-16 12:22:52 +08:00
OWNERS
[skip ci]Update OWNERS files (#11898)
2021-11-16 15:41:11 +08:00
rate_collector_test.go
Move some modules from internal to public package (#22572)
2023-04-06 19:14:32 +08:00
rate_collector.go
Format the code (#27275)
2023-09-21 09:45:27 +08:00
README.md
…
segment_sync_policy_test.go
fix: frequent flush cause minio rate limit (#28625)
2023-12-20 15:02:55 +08:00
segment_sync_policy.go
fix: frequent flush cause minio rate limit (#28625)
2023-12-20 15:02:55 +08:00
segment_test.go
Format the code (#27275)
2023-09-21 09:45:27 +08:00
segment.go
enhance: add param for bloomfilter(#29388) (#29614)
2024-01-02 18:58:47 +08:00
services_test.go
fix: Use channel cp as the dml&start position for import segments (#30107) (#30133)
2024-01-22 13:32:55 +08:00
services.go
fix: Use channel cp as the dml&start position for import segments (#30107) (#30133)
2024-01-22 13:32:55 +08:00
timetick_sender_test.go
[2.3] Refine datanode Timetick Sender (#28393) (#28430)
2023-11-15 10:08:26 +08:00
timetick_sender.go
[2.3] Refine datanode Timetick Sender (#28393) (#28430)
2023-11-15 10:08:26 +08:00
util.go
Format the code (#27275)
2023-09-21 09:45:27 +08:00

README.md

Data Node

DataNode is the component to write insert and delete messages into persistent blob storage, for example MinIO or S3.

Dependency

  • KV store: a kv store that persists messages into blob storage.
  • Message stream: receive messages and publish imformation
  • Root Coordinator: get the latest unique IDs.
  • Data Coordinator: get the flush information and which message stream to subscribe.