387 Commits

Author SHA1 Message Date
zhagnlu
0cb3f88e73
Fix unit test case of thread pool (#26501)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-08-21 21:50:22 +08:00
zhagnlu
79ca04b995
fix unstable test case for thread pool (#26465)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-08-18 22:42:20 +08:00
cai.zhang
c073aa0dc3
Fix bug for json_contains_all has multiple array elements (#26446)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-08-18 22:36:19 +08:00
cqy123456
fd37860e57
init knowhere build/search thread pool; (#26449)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-08-18 14:00:19 +08:00
zhagnlu
bdc8c507ea
Refactor thread pool to dynamic threads mode (#26407)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-08-18 11:58:19 +08:00
xige-16
50a77ef1f7
Fix serialize string index failed when greater than 2G (#26392)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-17 21:16:19 +08:00
zhagnlu
709352f96c
Repeat multi times for performace compare test because of cpu cache (#26394)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-08-17 15:18:17 +08:00
Jiquan Long
95bcf3e900
Fix error message of proto parser not clear (#26365)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-16 20:20:17 +08:00
Patrick Weizhi Xu
09da953a19
Use Knowhere AIO Context Init Defalut Value and Panic when Fail (#26286)
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2023-08-13 20:53:30 +08:00
xige-16
5b8d716cbc
Add ut for growing segment load binlog (#26268)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-13 20:41:31 +08:00
cai.zhang
a0198ce8ae
Support json contains feature (#25384)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-08-11 17:09:30 +08:00
zhagnlu
9489e14000
Optimize multi logical exprs performance when meet some situations (#26265)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-08-11 15:31:29 +08:00
zhagnlu
411f9ac823
Upgrade minio-go and add region and virtual host config for segcore chunk manager (#26194)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-08-11 10:37:36 +08:00
Enwei Jiao
ca1349708b
Remove time travel ralted testcase (#26119)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-10 18:53:17 +08:00
xige-16
1055c90456
Add default retrieve limit (#24782)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-10 14:11:15 +08:00
yah01
300fef446b
Enable mmap for vector index (#25877)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-10 13:59:15 +08:00
Jiquan Long
6578a27fe7
Support empty expression (#26153)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-10 08:37:15 +08:00
Jiquan Long
bafb183a2b
Optimize bitset usage (#26096)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-03 15:25:09 +08:00
MrPresent-Han
5634ba777d
add new threadpool with various priority to avoid deadlock(#25781) (#26028)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-03 09:31:07 +08:00
Jiquan Long
5c1f79dc54
Push down the limit operator to segcore (#25959)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-01 20:29:05 +08:00
yah01
9618bd9b42
Set channel capacity before consuming it (#25895)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-26 17:35:01 +08:00
foxspy
5437fcce8e
fix range search (#25880)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-07-25 10:07:01 +08:00
yah01
53c3bf053e
Fix unstable sealed segment bruteforce unittest (#25867)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-07-25 09:05:00 +08:00
Cai Yudong
9a4761dcc7
Remove binary metrics TANIMOTO/SUPERSTRUCTURE/SUBSTRUCTURE (#25708)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-07-19 16:16:58 +08:00
zhagnlu
65cb52d06b
Support dynamic simd framework and using term expr as example (#25260)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-07-13 16:22:30 +08:00
yah01
dd5f896dc8
Load batch by batch (#25212)
This will significantly reduce the memory usage while loading
- 1x memory usage and MBs overhead for buffer (memory mode)
- only MBs overhead for buffer (mmap mode)

Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-06 13:58:27 +08:00
Jiquan Long
c2a2ece551
Fix expression on integer overflow case (#25320)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-07-05 10:18:24 +08:00
foxspy
31173727b2
growing segment index memory opt & get vector bugfix (#25272)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-07-05 00:04:25 +08:00
Enwei Jiao
816158e4af
Remove outdated searchplan (#25282)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-04 18:30:25 +08:00
xige-16
04082b3de2
Migrate the ability to upload and download binlog to cpp (#22984)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-25 14:38:44 +08:00
PowderLi
3f4356df10
fix the spelling of field (#25008)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-06-21 14:00:42 +08:00
yah01
a413842e38
Fix deleted data is still visible (#24849)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-16 17:16:41 +08:00
zhagnlu
c5b1533fdc
Change segcore search_id from travelling all bits to select true bits(#24659) (#24800)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-06-16 16:48:44 +08:00
cqy123456
a519213316
Update knowhere version, update diskann api and generate cache nodes in build process (#24898)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-06-16 14:20:39 +08:00
Jiquan Long
26a6e1b946
Fix BinaryRange expression on integer overflow case (#24920)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-06-16 09:56:39 +08:00
zhagnlu
f60b839127
Support element in json array in segcore part(#24677) (#24829)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-06-14 14:38:37 +08:00
Jiquan Long
c6a6998ba7
Fix term expression on interger overflow case (#24867)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-06-14 11:44:38 +08:00
Cai Yudong
1b3c4b26f1
Optimize range search result sort in segcore (#24837)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-06-13 19:22:38 +08:00
yah01
ceda0ed598
Optimize the performance of filter by JSON field (#24268)
- Construct JSON pointer only once
- Avoid copying nested path for each row

Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-22 00:47:25 +08:00
yah01
c75e7a5d05
Fix failed to compare int value with double value (#24229)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-05-19 12:57:23 +08:00
zhagnlu
113f9a0ebc
Support SIMD of several Expr (#23715) (#23717)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-05-12 14:11:20 +08:00
cai.zhang
9715a850fa
Support expr with json field (#23804)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-10 10:19:19 +08:00
yah01
62eea5286f
Support to filter with json expr (#23739)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-30 20:36:39 +08:00
yah01
60fdd7e4f4
Introduce simdjson (#23644)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-26 10:30:34 +08:00
foxspy
6f4ed517de
add growing segment index (#23615)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-04-26 10:14:41 +08:00
yihao.dai
092d743917
Add support for getting vectors by ids (#23450)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-23 09:00:32 +08:00
yah01
546080dcdd
Support to retrieve json (#23563)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-21 11:46:32 +08:00
Cai Yudong
5f4673fd16
Optimize unittest to save runtime (#23248)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-04-07 14:20:29 +08:00
Cai Yudong
ef63e64ded
Remove ANNOY index type (#23189)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-04-04 16:30:27 +08:00
Enwei Jiao
50dab2d394
Add tracing for retrieve (#23033)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-03-29 18:10:02 +08:00