chyezh
dd957cf9e3
enhance: add configurable memory index load predict memory usage factor ( #30561 )
...
related pr: https://github.com/milvus-io/milvus/pull/30475
Signed-off-by: chyezh <chyezh@outlook.com>
2024-03-01 15:23:00 +08:00
chyezh
0c7474d7e8
enhance: add graceful stop timeout to avoid node stop hang under extreme cases ( #30317 )
...
1. add coordinator graceful stop timeout to 5s
2. change the order of datacoord component while stop
3. change querynode grace stop timeout to 900s, and we should
potentially change this to 600s when graceful stop is smooth
issue: #30310
also see pr: #30306
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-29 17:01:50 +08:00
yihao.dai
c5918290e6
feat: Add import executor and manager for datanode ( #29438 )
...
This PR introduces novel importv2 roles for datanode:
1. Executor: To execute tasks, a import task will be divided into the
following steps: read data -> hash data -> sync data;
2. Manager: To manage all the tasks;
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-01-31 20:45:04 +08:00
yihao.dai
c02fb64ad6
enhance: Allows proactive warming up of chunk cache ( #30182 )
...
Allows proactive warming up of chunk cache. Original vector data will be
asynchronously loaded into the chunk cache during the load process. It
has the potential to significantly reduce query/search latency for a
certain duration after the load, albeit with a concurrent increase in
disk usage.
issue: https://github.com/milvus-io/milvus/issues/30181
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-01-25 19:55:39 +08:00
xige-16
0a70e8b601
enhance: Remove multiple vector field limit ( #27827 )
...
issue: https://github.com/milvus-io/milvus/issues/25639
/kind improvement
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-12-28 16:40:46 +08:00
wei liu
839a72129e
fix: Auto balance param can't be updated by dynamic ( #29501 )
...
This PR fixed that auto balance param can't be updated by dynamic
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-27 14:30:53 +08:00
MrPresent-Han
7c7003bff6
enhance:refine the range of chunk size config value( #29388 ) ( #29389 )
...
related: #29388
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-12-26 17:36:46 +08:00
wei liu
820ee692fc
enhance: Add config for querycoord auto balance channel ( #29231 )
...
issue: #23726
This PR add control config to querycoord's background auto balance
channel operation
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-18 10:00:40 +08:00
wayblink
51f870da7e
feat: Introduce channelCheckpointUpdater to reduce goroutine use in ttNode ( #28570 )
...
/kind enhancement
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-12-12 13:48:42 +08:00
wei liu
69a71e11d1
enhance: Add config for auto upgrade segment index ( #29112 )
...
This PR add config for auto upgrade segment index to index engine's
version.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-12 11:56:37 +08:00
wayblink
6736f65345
feat: skip some empty ttMsg in Datanode flowgraph ( #28756 )
...
/kind feature
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-12-07 01:00:37 +08:00
wei liu
4b8d3eaa41
enhance: Increase heartbeat timeout ( #28648 )
...
it's easy to trigger heartbeat timeout after 100ms when standalone cpu
usage reach 100%.
This PR increase the heartbeat timeout param to 2000ms
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-23 19:50:24 +08:00
wei liu
5b45a138b1
disable auto balance when old node exists ( #28191 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-07 14:02:20 +08:00
yah01
90e2c63d9e
Fix getting incorrect CPU num ( #28146 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-11-06 06:02:16 +08:00
cqy123456
4fbe3c9142
replace loaded binlog with binlog index for search performance ( #27673 )
...
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-11-01 02:20:15 +08:00
smellthemoon
4b0ec156b3
Set channel work pool size in datanode ( #27728 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-10-19 08:28:08 +08:00
yihao.dai
106c17f304
Make read ahead policy in ChunkCache configurable ( #27291 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-28 15:47:27 +08:00
wei liu
4071132f6a
reload loading collection when qc recover ( #27300 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-09-27 11:43:28 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam ( #26725 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-05 10:31:48 +08:00
wei liu
1097776477
stop heartbeat if reach heartbeat limit ( #26728 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-09-04 17:51:48 +08:00
XuanYang-cn
b2e7cbdf4b
Remove TimeTravel in compactor ( #26785 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-04 17:41:48 +08:00
congqixia
d343888f3c
Fix pkg codecov & static-check script ( #26720 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-31 09:59:01 +08:00
congqixia
ee7aef9272
Make pulsar request timeout configurable ( #26525 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-23 20:46:23 +08:00
wei liu
79ccf06cf6
refine proxy to querynode heartbeat interval ( #26426 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-18 14:26:19 +08:00
wei liu
f33a89387f
enable config shard level cost metrics whether contains worker's cost ( #26132 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-09 11:37:15 +08:00
congqixia
767955ec6b
Reduce MQ buffer length and flowgraph wait queue length to 16 ( #26179 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-09 10:05:14 +08:00
xige-16
d1d0169fa3
Delete useless config ( #26173 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-08 10:23:08 +08:00
SimFG
d2649b63db
Delete the user-role mapping info when deleting the user ( #25988 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-04 18:37:08 +08:00
wei liu
518b6310a2
refine retry times on replica ( #26043 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-03 15:55:09 +08:00
wei liu
302897f866
refine look aside balance logic ( #25837 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-25 18:51:01 +08:00
jaime
8e5d6193f9
Add a timeout config for bulkinsert request ( #25789 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-07-24 14:23:00 +08:00
congqixia
1045c88102
Support replace indexed field in QueryCoord ( #25747 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-19 21:22:58 +08:00
wei liu
6534396b3d
enable config different interval for different checker ( #25514 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-19 16:50:57 +08:00
foxspy
31173727b2
growing segment index memory opt & get vector bugfix ( #25272 )
...
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-07-05 00:04:25 +08:00
smellthemoon
b30517d303
Enlarge timeout to prevent health check failure ( #25173 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-06-29 17:10:23 +08:00
wei liu
7b999b42bd
enable config policy on replica selection ( #25067 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-25 19:46:44 +08:00
yihao.dai
c73219a54d
Limit the number of concurrent sync tasks and allow only one sync task for the same segment ( #24881 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-16 14:14:39 +08:00
MrPresent-Han
b517bc9e6a
refine balance mechanism including:( #23454 ) ( #23763 ) ( #23791 )
...
1. balance granuity to replica to avoid influence unrelated replicas
2. avoid balance back and forth
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-05-04 12:22:40 +08:00
yah01
eab94489ba
Refine the merge algorithm ( #23767 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-27 18:26:35 +08:00
foxspy
6f4ed517de
add growing segment index ( #23615 )
...
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-04-26 10:14:41 +08:00
wei liu
4336ed8609
fix node up ( #23415 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-20 09:52:31 +08:00
cqy123456
7ff6a3a246
Set query node thread pool size in milvus.yaml ( #23286 )
...
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-04-17 10:44:29 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
