249 Commits

Author SHA1 Message Date
Cai Yudong
ef63e64ded
Remove ANNOY index type (#23189)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-04-04 16:30:27 +08:00
yah01
3202eb0d9f
Optimize rounding distances, avoid promoting to double (#22846)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-20 14:09:56 +08:00
Jiquan Long
8139106b51
Feat: count entities by expression (#22765)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-03-16 19:31:55 +08:00
yah01
a4031da634
Refine string parameters, avoid coping or deref (#22708)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-13 17:53:53 +08:00
yah01
bdd6bc7695
Re-format cpp code (#22513)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-02 15:55:49 +08:00
yah01
7478e44911
Support using mmap to load data (#22052)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-01 18:07:49 +08:00
smellthemoon
cf999b68dd
Fix range search can not get status from knowhere (#22384)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-02-24 17:59:45 +08:00
smellthemoon
7a4dfcc72b
Add status error output (#22325)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-02-22 15:51:44 +08:00
smellthemoon
9e0ec15436
Support range search (#21652)
Signed-off-by: smellthemoon <xinguo.li@zilliz.com>
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
2023-02-21 09:48:32 +08:00
presburger
9950cacd10
support knowhere 2.0 (#21857)
Signed-off-by: Yusheng.Ma <Yusheng.Ma@zilliz.com>
2023-02-10 14:24:32 +08:00
yah01
bdcff5b921
Avoid counting all bits to improve query performance (#21909)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-02-01 18:49:51 +08:00
Cai Yudong
e3d1624e97
Handle distance Inf correctly (#21828)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2023-02-01 14:59:51 +08:00
yah01
e8e2696430
avoid checking search params many times (#21516)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-06 13:15:35 +08:00
jaime
58b79eb74c
Add based on timetravel GC for snapshot KV (#21417)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-04 21:37:35 +08:00
Cai Yudong
4c5ffc832c
Add param check for BruteForceSearch in segcore (#20838)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-11-30 14:53:15 +08:00
Jiquan Long
8d78d9943a
Fix term filtering failed on indexed scalar field (#20840) (#20882)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-11-30 10:05:15 +08:00
smellthemoon
f5ab719f21
timestamp decided if the pks were the same (#20166)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-11-21 10:55:10 +08:00
liliu-z
174310a14e
Use all() api for bitset check (#20462)
Signed-off-by: Li Liu <liliu@LideMacBook-Pro.local>

Signed-off-by: Li Liu <liliu@LideMacBook-Pro.local>
Co-authored-by: Li Liu <liliu@LideMacBook-Pro.local>
2022-11-10 16:15:05 +08:00
XuanYang-cn
c24244240e
Fix insert search concurrency problem (#20065)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-10-26 18:57:36 +08:00
Enwei Jiao
4e5d1c2e5f
Add log to debug bitset (#19982)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-10-24 10:09:28 +08:00
Cai Yudong
87d78a4a85
Ignore cases when comparing metric type in segcore (#19437)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-09-26 17:58:52 +08:00
aoiasd
2b58bd5c0a
Optimize large memory usage of InsertRecord by using vector instead of unordered_map if InsertRecord used in sealed segment (#19245)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-09-23 17:08:51 +08:00
xige-16
428840178c
Support diskann index for vector field (#19093)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-21 20:16:51 +08:00
Cai Yudong
686b0ce796
Upgrade to knowhere-v1.3.0, remove following index support: (#18935)
- IVF_SQ8H
- RHNSW_FLAT/RHNSW_PQ/RHNSW_SQ
- NGT
- NSG
- SPTAG

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-09-05 10:41:11 +08:00
Letian Jiang
ef7742507d
Remove redundant code (#18832)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-08-25 19:48:54 +08:00
Cai Yudong
305601ad25
Add API SearchOnSealedChunk() to search on sealed segment without index (#18830)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-08-25 18:16:55 +08:00
Cai Yudong
dcf45df029
Optimize API vector_search parameter in segcore (#18827)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-08-25 16:16:54 +08:00
Cai Yudong
94ffa8a275
Upgrade to knowhere-v1.2.0 (#18746)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-08-23 10:20:58 +08:00
Cai Yudong
7cd37fc6dd
Merge FloatSearch() and BinarySearch() into SearchOnGrowing() (#18498)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-08-03 11:56:36 +08:00
zhenshan.cao
a287a2b3fd
Return empty result in advance if all data filtered out (#18329) (#18438)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-28 12:36:30 +08:00
Jeng.Gwan
a926a9eb66
Fix redundant string copy of PlaceholderGroup (#18236)
Signed-off-by: xaxys <zheng.guan@zilliz.com>
2022-07-12 15:48:25 +08:00
Cai Yudong
015a2f0866
Let growing segment call knowhere brute search API (#18227)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-07-12 11:58:25 +08:00
bigsheeper
22508f36d3
Get search field id from search plan and log if loaded index when search segments (#18183)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-08 20:18:22 +08:00
Cai Yudong
5edeeb5fcd
Define new SubSearchResult constructor to avoid memory copy for growing (#18000)
segment search

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-07-05 17:14:20 +08:00
Jiquan Long
6954a5ba3e
Fix search successfully with invalid metric type (#17977)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-07-01 22:28:23 +08:00
Cai Yudong
a001412e12
Replace faiss::MetricType with knowhere::MetricType (#17891)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-06-29 14:20:19 +08:00
Cai Yudong
7385770014
Upgrade to knowhere-v1.1.12 (#17692)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-06-24 10:34:18 +08:00
Cai Yudong
e78269f450
Optimize search related interface in segcore (#17568)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-06-23 18:16:13 +08:00
Jiquan Long
e751663fdc
Fix err msg when the dimension of query vectors mismatch field (#17553)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-15 10:54:09 +08:00
xige-16
36ad989590
Fix segOffset grater than insert barrier when mark delete (#17444)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-10 20:02:08 +08:00
bigsheeper
f38637c227
Pass PlaceholderGroup pointer to prevent memory copy in SegCore (#17389)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-06 21:34:05 +08:00
bigsheeper
9f786dd752
Remove unnecessary bitset flip in Search and Query (#17387)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-06 17:38:05 +08:00
xige-16
56778787be
Reverse data from scalar index (#17145)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-26 14:58:01 +08:00
xige-16
a8829554eb
Merge multiple sets of expr's optypes in diff dirs (#17192)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-24 21:56:00 +08:00
zhenshan.cao
08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-23 16:41:58 +08:00
Jiquan Long
77b0f290a8
Fix prefix match on sealed segment (#17050)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-05-17 20:15:55 +08:00
xige-16
515d0369de
Support string type in segcore (#16546)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
2022-04-29 13:35:49 +08:00
Aivin V. Solatorio
04fffb08ef
Support arithmetic operations on numerical fields for scalar filtering (#16520)
Signed-off-by: Aivin V. Solatorio <avsolatorio@gmail.com>
2022-04-24 16:43:45 +08:00
yah01
3f5733b738
Return the local variable directly, make RVO enabled (#16201)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-22 17:43:42 +08:00
zhenshan.cao
58ea38142f
Use boost dynamic_bitset in segcore (#16476)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-04-14 22:37:34 +08:00