255 Commits

Author SHA1 Message Date
congqixia
36ddeae2cc
Refine log level when request resource fail for loading segments (#28004)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-30 14:20:14 +08:00
Filip Haltmayer
6b1a106a31
Moving etcd client into session (#27069)
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
2023-10-27 07:36:12 +08:00
congqixia
13877a07ff
Add ctx parameter for tsafe pkg & NewDelegator method (#27877)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-26 19:14:10 +08:00
wei liu
8041fc3c75
avoid add empty growing segment to delegator distribution (#27930)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-26 10:10:10 +08:00
Gao
c8415453a1
Support prepare config for index (#27920)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-10-25 19:54:12 +08:00
aoiasd
9ba79a97e4
Add some log for delete task when it need query (#27317)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-10-24 14:54:12 +08:00
wei liu
eadd48ef7c
fix grpc error judge logic (#27800)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-20 04:28:09 +08:00
MrPresent-Han
af61a80f73
connect traceID for retrieve (#27804)
/kind improvement

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-10-19 19:22:10 +08:00
zhenshan.cao
020ad9a6bc
Rectify wrong exception messages associated with Array datatype (#27769)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-10-19 17:24:07 +08:00
congqixia
bcbe98aba1
Add querynode client wrapper and avoid grpc in standalone mode (#27781)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-19 11:10:07 +08:00
Enwei Jiao
e98e56f75d
Fix SIGSEGV if run within gdb (#27736)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-10-18 02:16:11 +08:00
congqixia
1f2a76d04d
Handle legacy querynode Delete Unimplemented (#27749)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-17 19:06:06 +08:00
jaime
ec1fe3549e
Add a stop hook to clean session (#27564)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-16 10:24:10 +08:00
yah01
50672f0d60
Skip empty segment while query/search (#27711)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-15 16:10:04 +08:00
yihao.dai
d9431266bb
Fix panic at loadSegment during rolling upgrade (#27671)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-13 16:31:35 +08:00
yah01
be980fbc38
Refine state check (#27541)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
wei liu
0695c88844
refine_log_of_update_target_version (#27572)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-11 06:51:33 +08:00
yihao.dai
7d0dd0047d
Use path.Join when init ChunkCache path (#27433)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-30 17:05:27 +08:00
zhenshan.cao
dbdb9e15d8
Update Knowhere version (#27445)
Signed-off-by: Li Liu <li.liu@zilliz.com>
Co-authored-by: Li Liu <li.liu@zilliz.com>
2023-09-29 14:23:28 +08:00
Jiquan Long
370fdaf50d
Record engine version for segment index (#27384)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-09-28 18:03:28 +08:00
congqixia
6ff4ffa6c2
Fix scheduler may panic during close (#27431)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-28 16:53:26 +08:00
yihao.dai
106c17f304
Make read ahead policy in ChunkCache configurable (#27291)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-28 15:47:27 +08:00
congqixia
258e1ccd66
Refine querynode scheduler lifetime (#26915)
This PR refines scheduler lifetime control:
- Move private tri-state into lifetime package
- Make scheduler block incoming "Add" task
- Make scheduler Stop wait until all previously accepted task done

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-28 10:21:26 +08:00
congqixia
8c59dba329
Refine queryHook mockery (#27394)
This PR move `QueryHook` interface to `optimizers` pkg
Update all mockery generated files to latest
Add makefile entry for `QueryHook`

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-28 10:01:26 +08:00
yah01
2d6a968233
Refine QueryNode errors (#27380)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-26 22:09:28 +08:00
yah01
6539a5ae2c
Refine DataCoord status (#27262)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-26 17:15:27 +08:00
cai.zhang
30a6c6072e
Fix bug for loading array data panic (#27343)
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-09-26 12:23:26 +08:00
jaime
7f7c71ea7d
Decoupling client and server API in types interface (#27186)
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
foxspy
5db4a0489e
dynamic index version control (#27335)
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-09-25 21:39:27 +08:00
xige-16
fa06265756
Fix cgo pointer param error (#27313)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-09-22 18:49:26 +08:00
yihao.dai
8f4aaa2da8
Remove ValidateIndexedFieldsData (#27254)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-22 18:31:25 +08:00
foxspy
370b6fde58
milvus support multi index engine (#27178)
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-09-22 09:59:26 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
yah01
0a750408d0
Skip delta logs have been applied (#26971)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-19 16:21:23 +08:00
MrPresent-Han
33e3e78937
add metrics for query reduce (#27201)
/kind improvement
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-09-19 14:53:22 +08:00
congqixia
cc9974979f
Add staticcheck linter and fix existing problems (#27174)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-19 10:05:22 +08:00
MrPresent-Han
7939f0e7d5
enable ctx traceId for assignsegment on dc(#26972) (#27108) (#27030)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-09-18 11:39:20 +08:00
Xiaofan
e51cb739eb
Remove some logs in querynode delegator (#27156)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-09-18 10:57:20 +08:00
xige-16
488b423e1b
Fix high cpu usage caused by proto.size (#27054)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-09-17 20:55:21 +08:00
yihao.dai
bb6711f28c
Add ChunkCache: support get vector from storage (#26142)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-15 10:21:20 +08:00
yah01
168e82ee10
Fix panic while handling with the nil status (#27040)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-15 10:09:21 +08:00
congqixia
394c898b4c
Discard SyncDistribution set action from legacy querycoord (#27027)
Since Milvus in lower version (< 2.3.0), there is no load info in set action
which may corrupt data integrity and cause panicking

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-13 09:35:18 +08:00
yah01
00c65fa0d7
Refine QueryNode errors (#27013)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-12 16:07:18 +08:00
aoiasd
e107d0794c
support complex delete expression (#25752)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-09-12 10:19:17 +08:00
congqixia
ac45af585b
Make segment loaded successful put in manager even ctx done (#26992)
Leave segment loaded in manager even wait other segment failed
See also #26908
Fix error case in distributed scenario

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-11 21:33:18 +08:00
congqixia
2a5d574a0d
Fix querynodev2 concurrent load logic (#26959)
Fix logic error from #26926
function `waitSegmentLoadDone` shall return error when context is done

Make delegator control concurrency for each same segment
Related to #26908

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-10 07:41:18 +08:00
congqixia
c6116d1819
Remove segment to LocalSegment type assertion (#26931)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-09 10:35:16 +08:00
congqixia
c8f9f22c4a
Fix segment loader return false success (#26926)
`waitSegmentLoadDone` did not check waitCh result is success or failure
after load return without error, delegator will assume all segments are loaded

This PR changes waitCh to loadResult with `sync.Cond` with `atomic.Int32` to represent status

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-08 16:41:16 +08:00
Xu Tong
9166011c4a
Add float16 vector (#25852)
Signed-off-by: Writer-X <1256866856@qq.com>
2023-09-08 10:03:16 +08:00
XuanYang-cn
7f1ae35e72
Add timeout in dispatcher, AsConsumer and Seek (#26686)
See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-08 09:51:17 +08:00