222 Commits

Author SHA1 Message Date
yah01
dc89730a50
Support collection-level mmap control (#26901)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-02 23:52:16 +08:00
Enwei Jiao
8ae9c947ae
Use OpenDAL to access object store (#25642)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-11-01 09:00:14 +08:00
yah01
2af46d7333
Increase the ChunkManager request timeout (#28015)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-31 09:06:13 +08:00
zhagnlu
6060dd7ea8
Add chunk manager request timeout (#27692)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-10-23 20:08:08 +08:00
Enwei Jiao
e98e56f75d
Fix SIGSEGV if run within gdb (#27736)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-10-18 02:16:11 +08:00
Enwei Jiao
b80a3e19d3
Add code for PanicInfo (#27364)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-27 12:01:28 +08:00
foxspy
5db4a0489e
dynamic index version control (#27335)
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-09-25 21:39:27 +08:00
foxspy
fa033e586a
disable growing index for flat (#27309)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-09-22 14:19:24 +08:00
foxspy
370b6fde58
milvus support multi index engine (#27178)
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-09-22 09:59:26 +08:00
cai.zhang
a362bb1457
Support array datatype (#26369)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-09-19 14:23:23 +08:00
PowderLi
4feb3fa7c6
support azure (#26398)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-09-19 10:01:23 +08:00
yihao.dai
bb6711f28c
Add ChunkCache: support get vector from storage (#26142)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-15 10:21:20 +08:00
Enwei Jiao
0afdfdb9af
Remove other Exceptions, keeps SegcoreError only (#27017)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-14 14:05:20 +08:00
Enwei Jiao
c3f15c6b95
Refactor duplicate error class into one place (#26985)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-11 20:43:17 +08:00
Xu Tong
9166011c4a
Add float16 vector (#25852)
Signed-off-by: Writer-X <1256866856@qq.com>
2023-09-08 10:03:16 +08:00
MrPresent-Han
a34a9d606c
fix panic due to empty traceID(#26754) (#26808)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-09-02 16:13:01 +08:00
MrPresent-Han
7d5a4b2994
add more event for segcore search(#26277) (#26688)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-30 14:15:01 +08:00
MrPresent-Han
d30a920226
add log trace for segcore(#26277) (#26339)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-16 11:41:33 +08:00
yah01
c0870d3c62
Set default thread pool size to be smaller (#26328)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-14 16:33:35 +08:00
cai.zhang
a0198ce8ae
Support json contains feature (#25384)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-08-11 17:09:30 +08:00
zhagnlu
411f9ac823
Upgrade minio-go and add region and virtual host config for segcore chunk manager (#26194)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-08-11 10:37:36 +08:00
xige-16
1055c90456
Add default retrieve limit (#24782)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-10 14:11:15 +08:00
Enwei Jiao
f97d5a7d08
Fix compile failed with llvm16 (#26112)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-03 17:29:07 +08:00
MrPresent-Han
5634ba777d
add new threadpool with various priority to avoid deadlock(#25781) (#26028)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-03 09:31:07 +08:00
zhagnlu
833674c1cb
add glog configurable function and redirect aws log to segcore log (#25664)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-07-27 19:49:02 +08:00
Cai Yudong
9a4761dcc7
Remove binary metrics TANIMOTO/SUPERSTRUCTURE/SUBSTRUCTURE (#25708)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-07-19 16:16:58 +08:00
yah01
227d2c8b3a
Reduce loading index memory usage (#25698)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-19 14:02:57 +08:00
zhagnlu
65cb52d06b
Support dynamic simd framework and using term expr as example (#25260)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-07-13 16:22:30 +08:00
yah01
cb721781f3
Improve error message throwed from knowhere (#25473)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-11 17:26:29 +08:00
yah01
dd5f896dc8
Load batch by batch (#25212)
This will significantly reduce the memory usage while loading
- 1x memory usage and MBs overhead for buffer (memory mode)
- only MBs overhead for buffer (mmap mode)

Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-06 13:58:27 +08:00
congqixia
549c4a0e1b
Enlarge integration test timeout (#25283)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-03 17:40:25 +08:00
yah01
cb4b88d5cf
Refactor the column type (#25147)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-27 19:50:45 +08:00
Cai Yudong
23492fed99
Correct range search param check error msg (#25066)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-06-25 15:34:44 +08:00
xige-16
04082b3de2
Migrate the ability to upload and download binlog to cpp (#22984)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-25 14:38:44 +08:00
Cai Yudong
1b3c4b26f1
Optimize range search result sort in segcore (#24837)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-06-13 19:22:38 +08:00
yah01
f40e22c78c
Add array_at method to get array from JSON (#24671)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-06 14:10:36 +08:00
yah01
7721cc73f0
Perf: Avoid copying string/json when filter on them (#24381)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-25 13:13:26 +08:00
presburger
152acb3a76
fix fmt lib conflict when compile with raft suppport (#24266)
Signed-off-by: Yusheng.Ma <Yusheng.Ma@zilliz.com>
2023-05-24 13:05:26 +08:00
yah01
6da8853756
Fix the JSON parser always parse all documents (#24290)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-22 15:49:27 +08:00
yah01
ceda0ed598
Optimize the performance of filter by JSON field (#24268)
- Construct JSON pointer only once
- Avoid copying nested path for each row

Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-22 00:47:25 +08:00
Cai Yudong
b1afd3ea2f
Update knowhere commit to fix BIN_IVF_FLAT upgrade (#24187)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-05-18 17:57:23 +08:00
zhagnlu
113f9a0ebc
Support SIMD of several Expr (#23715) (#23717)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-05-12 14:11:20 +08:00
yah01
62eea5286f
Support to filter with json expr (#23739)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-30 20:36:39 +08:00
yah01
0d3fcd5d16
Add method to access nested field at once (#23720)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-26 14:16:34 +08:00
yah01
60fdd7e4f4
Introduce simdjson (#23644)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-26 10:30:34 +08:00
foxspy
6f4ed517de
add growing segment index (#23615)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-04-26 10:14:41 +08:00
yihao.dai
092d743917
Add support for getting vectors by ids (#23450)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-23 09:00:32 +08:00
yah01
546080dcdd
Support to retrieve json (#23563)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-21 11:46:32 +08:00
yah01
aa2985490c
Retrieve page size by getpagesize() (#23561)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-20 12:36:30 +08:00
Enwei Jiao
967a97b9bd
Support json & array types (#23408)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
2023-04-20 11:32:31 +08:00