55 Commits

Author SHA1 Message Date
foxspy
560e167214
fix: add score compute consistency config for knowhere (#32584)
issue: #32583 
/kind branch-feature

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-04-25 14:07:25 +08:00
jaime
5ddb0b435f
fix: revoke session may be ignored due to server context cancellation in advance (#31213)
issue: #31219
pr: #31220

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-03-14 19:05:04 +08:00
chyezh
1c8d9fa686
fix: wrong context passing into NewClient, error handling lost in session_util (#30818)
issue: #30799
pr: #30817

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-28 10:41:00 +08:00
chyezh
77e123762f
enhance: add graceful stop timeout to avoid node stop hang under extreme cases (#30320)
1. add coordinator and proxy graceful stop timeout to 5s.
3. add other work node graceful stop timeout to 900s, and we should
potentially change this to 600s when graceful stop is smooth
4. change the order of datacoord component while stop.
5. `LivenessCheck` do not perform graceful shutdown now. 

issue: https://github.com/milvus-io/milvus/issues/30310
pr: #30317
also see: https://github.com/milvus-io/milvus/pull/30306

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-01-27 08:45:02 +08:00
congqixia
dd52a674aa
enhance: [cherry-pick] add ctx for HandleCStatus and callers (#29517) (#29546)
Cherry-pick from master
pr: #29517 
See also #29516

Make `HandleCStatus` print trace id for better logging

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-28 10:20:47 +08:00
wei liu
514da535e5
enhance: add metrics for stopping querynode balance progress (#29201) (#29390)
pr: #29201
This PR add three metrics to track the stopping balance progress.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-26 10:02:46 +08:00
SimFG
74e72ce27e
enhance: [2.3] Support to get the param value in the runtime (#29298)
pr: #29297
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-12-21 20:36:43 +08:00
MrPresent-Han
5f4ac437b2
enhance: [Cherry-pick] Moving etcd client into session (#27069) (#28996)
relate: #26694
pr: https://github.com/milvus-io/milvus/pull/27069

Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
Co-authored-by: Filip Haltmayer <81822489+filip-halt@users.noreply.github.com>
2023-12-07 16:22:34 +08:00
cqy123456
8fd38c8eea
enhance:[cherry-pick] Use binlog index for better search performance (#29012)
this pr is cherry-pick from master:
pr: https://github.com/milvus-io/milvus/pull/28528
pr: https://github.com/milvus-io/milvus/pull/27673
related issue:
issue: https://github.com/milvus-io/milvus/issues/27678

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-12-07 09:52:34 +08:00
SimFG
598788e6b8
Delay the cancellation of ctx when stopping the node (#28249)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-08 01:46:20 +08:00
congqixia
bcbe98aba1
Add querynode client wrapper and avoid grpc in standalone mode (#27781)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-19 11:10:07 +08:00
jaime
ec1fe3549e
Add a stop hook to clean session (#27564)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-16 10:24:10 +08:00
yihao.dai
7d0dd0047d
Use path.Join when init ChunkCache path (#27433)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-30 17:05:27 +08:00
yihao.dai
106c17f304
Make read ahead policy in ChunkCache configurable (#27291)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-28 15:47:27 +08:00
congqixia
258e1ccd66
Refine querynode scheduler lifetime (#26915)
This PR refines scheduler lifetime control:
- Move private tri-state into lifetime package
- Make scheduler block incoming "Add" task
- Make scheduler Stop wait until all previously accepted task done

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-28 10:21:26 +08:00
congqixia
8c59dba329
Refine queryHook mockery (#27394)
This PR move `QueryHook` interface to `optimizers` pkg
Update all mockery generated files to latest
Add makefile entry for `QueryHook`

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-28 10:01:26 +08:00
foxspy
5db4a0489e
dynamic index version control (#27335)
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-09-25 21:39:27 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
yihao.dai
bb6711f28c
Add ChunkCache: support get vector from storage (#26142)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-15 10:21:20 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam (#26725)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-05 10:31:48 +08:00
congqixia
4b58c71908
Add ctx parameter for organizeTask and GetWorker method (#26835)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-05 10:05:48 +08:00
Gao
7e36f819b4
Make autoindex config clean (#26732)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-08-31 10:25:01 +08:00
wei liu
7af0f7d90c
avoid concurrent sub/unsub on same channel (#26454)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-23 10:10:22 +08:00
cqy123456
fd37860e57
init knowhere build/search thread pool; (#26449)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-08-18 14:00:19 +08:00
yihao.dai
63b86b32a6
Add server id validation interceptor (#26395)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-17 20:20:20 +08:00
Enwei Jiao
7d61355ab0
Refactor log for Query (#26310)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-14 18:57:32 +08:00
xige-16
1971d98897
Add disk metric info (#25675)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-11 10:35:42 +08:00
yah01
300fef446b
Enable mmap for vector index (#25877)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-10 13:59:15 +08:00
MrPresent-Han
5634ba777d
add new threadpool with various priority to avoid deadlock(#25781) (#26028)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-03 09:31:07 +08:00
zhagnlu
833674c1cb
add glog configurable function and redirect aws log to segcore log (#25664)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-07-27 19:49:02 +08:00
yah01
361db51337
Create dir if mmap enabled for validating path (#25750)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-19 18:48:58 +08:00
congqixia
f07c934558
Fix querynodev2 grace stop condition reversed (#25409)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-07 17:28:25 +08:00
yah01
4c0e36d28c
Stop blocking until all channels & segments released (#25328)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-06 10:04:25 +08:00
chyezh
d7d61f529c
[Feature|Pick] enable scheduler policy and add user-task-polling policy (#24839)
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-07-03 18:24:25 +08:00
Gao
de6e4817a2
Support dynamic tuning config (#25152)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-07-03 15:18:24 +08:00
yah01
7b5013664d
Support graceful stop (#25226)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-30 17:28:23 +08:00
xige-16
33c2012675
Add more metrics (#25081)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-26 17:52:44 +08:00
xige-16
04082b3de2
Migrate the ability to upload and download binlog to cpp (#22984)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-25 14:38:44 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
congqixia
54306a10b3
Fix confusing log while plugin is not enabled (#24741)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-08 15:40:35 +08:00
yiwangdr
37c02c9927
clean up meta interface (#24729)
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-06-08 14:18:35 +08:00
xige-16
fe18109aab
Clean tmp disk data when start milvus (#24404)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-05-26 15:07:26 +08:00
congqixia
eb27a8a8c0
Fix integration test not wait for index built (#24037)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-12 09:51:25 +08:00
yah01
400364483d
Improve predict the resource usage (#23968)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-11 15:33:24 +08:00
foxspy
6f4ed517de
add growing segment index (#23615)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-04-26 10:14:41 +08:00
congqixia
4fe363c4b2
Add KNN cgo pool (#23526)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-19 23:56:31 +08:00
congqixia
eb690ef033
Fix querynodev2 stop logic (#23487)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-18 17:56:30 +08:00
cqy123456
7ff6a3a246
Set query node thread pool size in milvus.yaml (#23286)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-04-17 10:44:29 +08:00
yah01
dc0eb433ae
Add IoPoolSize config (#23360)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-13 14:50:28 +08:00
cai.zhang
43a9e175a3
Exit component process when session key is deleted (#21658) (#22164)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-04-12 20:12:28 +08:00