1308 Commits

Author SHA1 Message Date
yihao.dai
4434d001fd
Assign metric type in search request when user has not explicitly assigned (#24583)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-01 13:44:35 +08:00
Gao
8da5b6a614
Disable search params rewrite while searching with filter (#24556)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-05-31 19:52:33 +08:00
Bingyi Sun
b341f43e68
Add default panic handler for ants pool (#24536)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-05-31 14:43:39 +08:00
xige-16
793864c12d
Clean tmp disk data when start milvus (#24400)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-05-26 14:13:27 +08:00
yihao.dai
bbb4d5d463
Revert "Retrieve segments concurrently (#24358)" (#24383)
This reverts commit a6b82733ad60bf3b0068af18c138650ad7e6239b.

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-25 19:45:28 +08:00
yihao.dai
a6b82733ad
Retrieve segments concurrently (#24358)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-25 09:57:27 +08:00
wei liu
266bf418d4
fix real time task num metrics (#24349)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-24 14:27:26 +08:00
yihao.dai
a9c05d1eb0
Support rate limit based on growing segments size (#24157)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-16 22:03:24 +08:00
chyezh
20054dc42b
Support configurable policy for query node, Add user level schedule policy (#23718)
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-05-16 10:55:22 +08:00
yah01
62cabbfce2
Support JSON data type (#23839)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-09 22:11:24 +08:00
cqy123456
f7159189fd
modify diskann memory estimation method (#23892)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-05-09 20:29:09 +08:00
congqixia
bf8e6376db
Make querynode lazy load statslog if SkipBFLoad is true (#23904)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-08 14:20:41 +08:00
Gao
21762d9c23
Seperate reading build and load params for diskann (#23722)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-04-27 10:54:36 +08:00
yah01
1a773b8db1
Refine the merge algorithm (#23721)
For given two search tasks t1, t2:
- same topk: merge them if nq1 + nq2 <= MaxNQ
- otherwises: merge them if the merge cost doesn't exceeded merge ratio

the merge cost is defined as:
total cost - min(cost1, cost2)

Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-27 09:52:36 +08:00
congqixia
5aad58d318
Fix shard detector rewatch always returns closed channel (#23734)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-26 19:26:39 +08:00
congqixia
75b5751117
Re-calculate segment entry num when binlog field is zero (#23696)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-26 10:24:35 +08:00
congqixia
4b3a598a1e
Limit delete cgo call concurrency (#23680)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-25 19:18:34 +08:00
Gao
e820479b56
Add querynode plugin (#23599)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-04-25 10:40:34 +08:00
MrPresent-Han
8d5852494d
refine balance mechanism by pruning code and support urgent stopping balance(#23454) (#23455)
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-04-19 17:50:31 +08:00
Xiaofan
52fb48a33c
Add more log when search result is empty (#23421)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-04-17 17:14:31 +08:00
Xiaofan
6a4c2058d9
Fix Dead lock in shard manager (#23445)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-04-17 16:54:31 +08:00
congqixia
a66b77e408
Fix watcher loop quit and channel shouldDrop logic (#23394)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-14 07:30:30 +08:00
congqixia
27808850f1
Check leader ready before transfer load/release (#23190)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-12 16:22:29 +08:00
MrPresent-Han
d82ad03b19
fix false memory toleration calculation when loading segments on qnode(#23262) (#23263)
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-04-10 17:50:30 +08:00
cqy123456
dccaf5a5a6
set query node thread pool size in milvus.yaml (#23119)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-04-07 11:22:28 +08:00
wei liu
0bd812e334
fix dml channel metrics (#23137)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-31 15:08:23 +08:00
wei liu
315d468b49
fix fg metrics (#23139)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-31 14:12:23 +08:00
zhenshan.cao
f513fae4ca
Refine usage of TimeRecorder.Record (#23143)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-03-30 21:04:23 +08:00
Xiaofan
6f1eb73c77
Add config for disable BF load (#22997)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-03-28 20:06:08 +08:00
xige-16
ac6b65bacc
Migrate the ability to upload and download binlog to cpp (#22382)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-03-24 17:49:59 +08:00
congqixia
c13684c5e5
[2.2] Fix unsub channel alway removes queryshard (#22961)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-23 20:25:59 +08:00
congqixia
65f5400062
Use distribution.Peek for GetDataDistribution in case of blocked by release (#22752)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-14 19:03:55 +08:00
SimFG
b2ece6a569
Fix the nil point about the session (#22696)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-03-10 21:43:54 +08:00
yah01
07357e9051
Fix deadlock when releasing segments and removing nodes concurrently (#22584)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-07 11:21:51 +08:00
jaime
6605db2a24
Add NotFoundTSafer and NoReplicaAvailable to retryable error code (#22505)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-03-03 16:37:49 +08:00
smellthemoon
bd5b263c76
Fix get index mode error (#22364) (#22440)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-02-28 20:13:48 +08:00
yah01
3e560841e8
Fix ignoring error while subscribing channel (#22429) (#22452)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-02-28 14:35:48 +08:00
congqixia
a4149ee7fc
Prevent search/release on same segment (#22414)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-02-27 17:57:48 +08:00
cai.zhang
7aab677023
Gracefully stop session (#22386)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-02-27 16:05:48 +08:00
aoiasd
e89ee8d547
add search option to ignore ignore segment (#21994) (#22353)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-02-24 18:01:48 +08:00
SimFG
8123161738
Add the retry op for all server (#22274)
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
2023-02-24 17:51:47 +08:00
bigsheeper
ec20fec166
Skip forwarding query req if segments distribution is empty (#22341)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-02-22 21:25:45 +08:00
congqixia
a0e1cc4da8
Check cluster serviceable before get GetSegmentInfos for distribution (#22317)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-02-22 10:16:28 +08:00
xige-16
faf62cfa55
Fix metric of querynode's channel num (#22252)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-02-20 19:08:26 +08:00
yah01
787322f50c
Improve search task merge policy (#22006) (#22287)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-02-20 13:56:26 +08:00
congqixia
bc531b9e7a
Fix offline calculation not correct (#22225)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-02-20 10:08:26 +08:00
cai.zhang
dd63c71bd0
Fix ut about disconnect etcd (#22238)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-02-16 23:50:37 +08:00
congqixia
f63a00af6f
Fix concurrent segment alloc changes caused LeaderView error (#22154)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-02-15 14:08:34 +08:00
xige-16
590498d531
Fix metrics not registered (#22098)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-02-09 21:18:32 +08:00
xige-16
50739e6c6a
Add more metrics (#21975)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-02-09 16:06:33 +08:00