16080 Commits

Author SHA1 Message Date
yah01
c5f215da67
Refactor task scheduler and executor (#20828) (#20900)
Make the performance able to scale out

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-30 16:37:17 +08:00
Cai Yudong
225ced2183
Add param check for BruteForceSearch in segcore (#20903)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-11-30 16:15:15 +08:00
yah01
7d426ca744
Refine TimeRecorder output (#20897) (#20899)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-30 15:35:16 +08:00
cai.zhang
b81cd10dc3
Reduce some logs level which is frequent (#20901)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-11-30 15:27:16 +08:00
Jiquan Long
2bc5b61f62
Fix metric ProxyApplyPrimaryKeyLatency (#20499)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-11-30 14:21:16 +08:00
congqixia
3d7b83d4e5
Fix delete operation scope to vchannel (#20866) (#20881)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-11-30 09:55:16 +08:00
Xiaofan
61baf7dada
Remove some logs from debug to info (#20699)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-11-29 21:09:15 +08:00
NicoYuan1986
e2dd7a41ad
Modify test cases by making create index after insert (#20778)
Signed-off-by: nico <cheng.yuan@zilliz.com>

Signed-off-by: nico <cheng.yuan@zilliz.com>
2022-11-29 20:09:14 +08:00
yah01
8c353bf099
Fix asserting PK slice equal fails due to the same distances (#20858) (#20887)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-29 19:01:17 +08:00
yah01
84f88dd16c
Fix IndexNode panic while failed to create index (#20861) (#20883)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-29 16:53:15 +08:00
Jiquan Long
770918eaff
Fix term filtering failed on indexed scalar field (#20840)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-11-29 15:25:15 +08:00
congqixia
5c81fca3d1
Fix LRU cache may panic on double close (#20854) (#20872)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-11-29 14:15:14 +08:00
congqixia
711c91eb9a
Add log for transfer segment operation (#20829) (#20847)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-11-29 11:53:15 +08:00
congqixia
ae42fd5a6e
Add collection delta channel in mutex protection (#20736) (#20770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-11-29 11:51:14 +08:00
yah01
dd10e571cf
Avoid allocating and coping when fill column with default values (#20721) (#20788)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-26 18:59:13 +08:00
wei liu
5e63b84ff1
support pulsar tenant and auth (#19727) (#20762)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2022-11-25 17:11:12 +08:00
yah01
f19740fdac
Check whether CodecIndex is nil to avoid panic (#20816) (#20826)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-25 14:55:12 +08:00
shaoyue
0c49a1fe87
[restapi] Fix insert request lost precision for int64 fields (#20827)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-11-25 13:57:12 +08:00
XuanYang-cn
f85c7eb02e
Remove mockTestKv in metatable test (#20690)
See also: #19289

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-11-25 11:09:13 +08:00
jingkl
d2d72c1642
[test]modify the diskann testcase (#20782)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>

Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2022-11-24 16:55:14 +08:00
SimFG
925fa9d41d
Make sure that the segment path matches the segment id (#20696)
Signed-off-by: SimFG <bang.fu@zilliz.com>
(cherry picked from commit 764f17890072d206831ac40c12d6f51830251c90)
2022-11-24 10:51:12 +08:00
Bennu
a69425c2b8
[skip e2e] Delete windows code-checker (#20783) (#20785)
Signed-off-by: Bennu-Li <yunmei.li@zilliz.com>

Signed-off-by: Bennu-Li <yunmei.li@zilliz.com>
2022-11-23 16:49:12 +08:00
jingkl
2966710b9f
[test]modify the diskann testcase (#20728)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>

Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2022-11-23 09:53:21 +08:00
XuanYang-cn
89bdb34c98
Enable to gen compaction plan for stale segments (#20742)
See also: #20502

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-11-22 17:33:16 +08:00
yah01
004b443288
Change LRU to a generic type (#20744) (#20759)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-22 16:13:11 +08:00
congqixia
5594d754f1
Use embed etcd for allocator ut (#20749) (#20754)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-11-22 11:53:11 +08:00
Jenny Li
15c096b3b2
Make docker base separate from milvus image (#20473) (#20737)
Signed-off-by: Jenny Li <jing.li@zilliz.com>

Signed-off-by: Jenny Li <jing.li@zilliz.com>

Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-11-22 10:11:11 +08:00
smellthemoon
64e9cb3061
Only balance segement in targets (#20709) (#20750)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-11-22 09:49:11 +08:00
wei liu
efa1cf7f23
fix mannual balance with empty segment list (#20739)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2022-11-21 19:31:12 +08:00
Jenny Li
94407a9d2a
Show milvus config after install milvus (#20712) (#20722)
Signed-off-by: Jenny Li <jing.li@zilliz.com>

Signed-off-by: Jenny Li <jing.li@zilliz.com>

Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-11-21 14:31:11 +08:00
groot
e1e2323fdd
Fix a regression of local storage (#20631)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-21 10:31:12 +08:00
groot
e7429f88af
Change import task timeout value (#20716)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
v2.2.0
2022-11-18 18:59:09 +08:00
wayblink
b494b564e3
Update datanode.segment.insertBufSize default value to 16777216 (#20714)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-11-18 18:35:10 +08:00
ThreadDao
44d1df91d4
[test] Fix case compact while insert (#20662)
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2022-11-18 17:19:09 +08:00
jingkl
afb351d196
modfiy the testcase (#20702)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>

Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2022-11-18 13:25:09 +08:00
jingkl
b4af8703e3 [test]Add diskann testcase with search,query
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2022-11-17 23:20:10 +08:00
jaime
a2b383b523
Fix alter collection hang (#20686)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-11-17 22:07:09 +08:00
Bingyi Sun
c65306bc9b
Fix load uses compacted segments' binlogs (#20655) (#20695)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-11-17 22:01:09 +08:00
Jenny Li
3a42122b63
Task/Action won't finish util the RPC returned (#20669) (#20691)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
2022-11-17 20:41:09 +08:00
bigsheeper
dbcbe4d8f9
Disable tt delay protection (#20658)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-11-17 20:21:14 +08:00
wei liu
8b682fdeac
fix duplicate watch channel task (#20670)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2022-11-17 15:31:09 +08:00
cai.zhang
38845d9669
Reduce log frequency in IndexCoord (#20638)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-11-17 15:13:08 +08:00
yah01
15ae68faa5
Refresh targets atomically (#20666)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-17 14:15:08 +08:00
XuanYang-cn
2917856793
Reduce unnecessary logs of get compaction results (#20643)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-11-17 10:09:12 +08:00
Xiaofan
279cf6f5e5
Revert Index Codec (#20652)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-11-16 19:47:08 +08:00
yah01
6b1bb85426
Only balance segments in targets (#20636)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-16 14:33:11 +08:00
Xiaofan
d01092a119
Fix Index Codec memory and storage usage (#20622)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-11-16 11:45:08 +08:00
congqixia
31ee7098b5
Limit concurrency of decoded index file and do force gc (#20624)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-11-15 23:55:09 +08:00
cai.zhang
0ea2ff7858
Use GetSegmentUnsafe when get unhealthy segment (#20620)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-11-15 20:55:08 +08:00
congqixia
2880a75dfc
Fix querynode panics when watch/unsub runs concurrently (#20606) (#20619)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-11-15 20:03:06 +08:00