275 Commits

Author SHA1 Message Date
foxspy
31173727b2
growing segment index memory opt & get vector bugfix (#25272)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-07-05 00:04:25 +08:00
Enwei Jiao
816158e4af
Remove outdated searchplan (#25282)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-04 18:30:25 +08:00
yah01
73b7cf5df4
Fix bool unary expression sometimes returns no data (#25016)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-20 16:38:41 +08:00
Jiquan Long
8885fa2bc7
Fix BinaryArithOpEvalRange integer overflow case (#24943)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-06-16 17:10:39 +08:00
zhagnlu
c5b1533fdc
Change segcore search_id from travelling all bits to select true bits(#24659) (#24800)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-06-16 16:48:44 +08:00
Jiquan Long
26a6e1b946
Fix BinaryRange expression on integer overflow case (#24920)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-06-16 09:56:39 +08:00
zhagnlu
f3f3f8a849
Segcore retrieve by pk optimazation (#24659) (#24660)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-06-15 15:04:39 +08:00
Jiquan Long
7faf934f91
Fix UnaryRange expression on integer overflow case (#24893)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-06-15 14:06:39 +08:00
zhagnlu
f60b839127
Support element in json array in segcore part(#24677) (#24829)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-06-14 14:38:37 +08:00
Jiquan Long
c6a6998ba7
Fix term expression on interger overflow case (#24867)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-06-14 11:44:38 +08:00
Cai Yudong
1b3c4b26f1
Optimize range search result sort in segcore (#24837)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-06-13 19:22:38 +08:00
yah01
852e5ce904
Convert double to int if possible with term expression (#24530)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-05-30 22:03:28 +08:00
yah01
99b7a6882c
Fix the nested path lost when reuse the expression (#24417)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-05-25 21:35:27 +08:00
yah01
7721cc73f0
Perf: Avoid copying string/json when filter on them (#24381)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-25 13:13:26 +08:00
presburger
152acb3a76
fix fmt lib conflict when compile with raft suppport (#24266)
Signed-off-by: Yusheng.Ma <Yusheng.Ma@zilliz.com>
2023-05-24 13:05:26 +08:00
yah01
6da8853756
Fix the JSON parser always parse all documents (#24290)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-22 15:49:27 +08:00
yah01
ceda0ed598
Optimize the performance of filter by JSON field (#24268)
- Construct JSON pointer only once
- Avoid copying nested path for each row

Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-22 00:47:25 +08:00
yihao.dai
45aa9779e0
Retrieve segments concurrently (#24245)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-19 18:19:24 +08:00
yah01
c75e7a5d05
Fix failed to compare int value with double value (#24229)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-05-19 12:57:23 +08:00
zhagnlu
113f9a0ebc
Support SIMD of several Expr (#23715) (#23717)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-05-12 14:11:20 +08:00
cai.zhang
9715a850fa
Support expr with json field (#23804)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-10 10:19:19 +08:00
yah01
62eea5286f
Support to filter with json expr (#23739)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-30 20:36:39 +08:00
yah01
60fdd7e4f4
Introduce simdjson (#23644)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-26 10:30:34 +08:00
foxspy
6f4ed517de
add growing segment index (#23615)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-04-26 10:14:41 +08:00
yah01
546080dcdd
Support to retrieve json (#23563)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-21 11:46:32 +08:00
Cai Yudong
2725d38b9e
Add COSINE metric type (#23350)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-04-20 10:20:31 +08:00
Cai Yudong
ef63e64ded
Remove ANNOY index type (#23189)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-04-04 16:30:27 +08:00
yah01
3202eb0d9f
Optimize rounding distances, avoid promoting to double (#22846)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-20 14:09:56 +08:00
Jiquan Long
8139106b51
Feat: count entities by expression (#22765)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-03-16 19:31:55 +08:00
yah01
a4031da634
Refine string parameters, avoid coping or deref (#22708)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-13 17:53:53 +08:00
yah01
bdd6bc7695
Re-format cpp code (#22513)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-02 15:55:49 +08:00
yah01
7478e44911
Support using mmap to load data (#22052)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-01 18:07:49 +08:00
smellthemoon
cf999b68dd
Fix range search can not get status from knowhere (#22384)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-02-24 17:59:45 +08:00
smellthemoon
7a4dfcc72b
Add status error output (#22325)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-02-22 15:51:44 +08:00
smellthemoon
9e0ec15436
Support range search (#21652)
Signed-off-by: smellthemoon <xinguo.li@zilliz.com>
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
2023-02-21 09:48:32 +08:00
presburger
9950cacd10
support knowhere 2.0 (#21857)
Signed-off-by: Yusheng.Ma <Yusheng.Ma@zilliz.com>
2023-02-10 14:24:32 +08:00
yah01
bdcff5b921
Avoid counting all bits to improve query performance (#21909)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-02-01 18:49:51 +08:00
Cai Yudong
e3d1624e97
Handle distance Inf correctly (#21828)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2023-02-01 14:59:51 +08:00
yah01
e8e2696430
avoid checking search params many times (#21516)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-06 13:15:35 +08:00
jaime
58b79eb74c
Add based on timetravel GC for snapshot KV (#21417)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-04 21:37:35 +08:00
Cai Yudong
4c5ffc832c
Add param check for BruteForceSearch in segcore (#20838)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-11-30 14:53:15 +08:00
Jiquan Long
8d78d9943a
Fix term filtering failed on indexed scalar field (#20840) (#20882)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-11-30 10:05:15 +08:00
smellthemoon
f5ab719f21
timestamp decided if the pks were the same (#20166)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-11-21 10:55:10 +08:00
liliu-z
174310a14e
Use all() api for bitset check (#20462)
Signed-off-by: Li Liu <liliu@LideMacBook-Pro.local>

Signed-off-by: Li Liu <liliu@LideMacBook-Pro.local>
Co-authored-by: Li Liu <liliu@LideMacBook-Pro.local>
2022-11-10 16:15:05 +08:00
XuanYang-cn
c24244240e
Fix insert search concurrency problem (#20065)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-10-26 18:57:36 +08:00
Enwei Jiao
4e5d1c2e5f
Add log to debug bitset (#19982)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-10-24 10:09:28 +08:00
Cai Yudong
87d78a4a85
Ignore cases when comparing metric type in segcore (#19437)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-09-26 17:58:52 +08:00
aoiasd
2b58bd5c0a
Optimize large memory usage of InsertRecord by using vector instead of unordered_map if InsertRecord used in sealed segment (#19245)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-09-23 17:08:51 +08:00
xige-16
428840178c
Support diskann index for vector field (#19093)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-21 20:16:51 +08:00
Cai Yudong
686b0ce796
Upgrade to knowhere-v1.3.0, remove following index support: (#18935)
- IVF_SQ8H
- RHNSW_FLAT/RHNSW_PQ/RHNSW_SQ
- NGT
- NSG
- SPTAG

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-09-05 10:41:11 +08:00