admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-03 17:31:58 +08:00
Code Issues Packages Projects Releases Wiki Activity
18,778 Commits 138 Branches 188 Tags
Commit Graph

312 Commits

Author SHA1 Message Date
MrPresent-Han
9e2e7157e9
feat: support search_group_by for milvus(#25324) (#28983) 
related: #25324

Search GroupBy function, used to aggregate result entities based on a
specific scalar column.
several points to mention:

1. Temporarliy, the whole groupby is implemented separated from
iterative expr framework **for the first period**
2. In the long term, the groupBy operation will be incorporated into the
iterative expr framework:https://github.com/milvus-io/milvus/pull/28166
3. This pr includes some unrelated mocked interface regarding alterIndex
due to some unworth-to-mention reasons. All these un-associated content
will be removed before the final pr is merged. This version of pr is
only for review
4. All other related details were commented in the files comparison

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-01-05 15:50:47 +08:00
zhagnlu
79c417b14e
fix: pass active count to query context instead of timestamp (#29541) 
#29319

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-12-31 16:08:48 +08:00
yah01
aef483806d
enhance: improve the segcore logs (#29372) 
- remove the streaming logging
- refine existing logs

fix #29366

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-23 21:52:43 +08:00
zhagnlu
1cbe3cd5fc
fix: fix memory leak when cancel segcore task (#29431) 
#29430

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-12-22 20:28:43 +08:00
zhagnlu
a6eb7e5f9a
enhance: skip segment when using pk in (..) expr (#29394) 
#29293

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-12-21 20:06:42 +08:00
Gao
9b52cb6417
enhance: improve reducing results when many segments are filtered (#29073) 
Do not fill the invalid ids for the empty results, it will incur useless
memory overhead and reduce overhead when nq and topk is large.

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-12-20 12:56:42 +08:00
yah01
8f89e9cf75
enhance: remove all unnecessary string formatting (#29323) 
done by two regex expressions:
- `PanicInfo\((.+),[. \n]+fmt::format\(([.\s\S]+?)\)\)`
- `AssertInfo\((.+),[. \n]+fmt::format\(([.\s\S]+?)\)\)`

related: #28811

---------

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-20 10:04:43 +08:00
MrPresent-Han
bfca0a7926
fix: refine skipIndex to resolve cyclic dependcy(#29132) (#29189) 
related: #29132

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-12-19 10:26:40 +08:00
zhagnlu
a602171d06
enhance: Refactor runtime and expr framework (#28166) 
#28165

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-12-18 12:04:42 +08:00
cai.zhang
fb089cda8b
enhance: Load raw data while scalar index doesn't have raw data (#28888) 
issue: #28886

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-06 20:36:36 +08:00
Bingyi Sun
e5ce385ffd
enhance: remove -inl.h files (#28674) 
issue: https://github.com/milvus-io/milvus/issues/28673
Move template implementations from -inl.h to .cpp file and make explicit
instantiation

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-11-23 17:20:25 +08:00
MrPresent-Han
836f300536
support skip-index based on chunk-metrics to accelerate expr filter(#27925) (#28297) 
related: #27925

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-11-15 11:20:19 +08:00
cai.zhang
01c5484817
No hit when the index exceeds the array length (#28301) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-10 10:04:19 +08:00
Enwei Jiao
b80a3e19d3
Add code for PanicInfo (#27364) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-27 12:01:28 +08:00
cai.zhang
a362bb1457
Support array datatype (#26369) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-09-19 14:23:23 +08:00
Enwei Jiao
0afdfdb9af
Remove other Exceptions, keeps SegcoreError only (#27017) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-14 14:05:20 +08:00
Enwei Jiao
c3f15c6b95
Refactor duplicate error class into one place (#26985) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-11 20:43:17 +08:00
Xu Tong
9166011c4a
Add float16 vector (#25852) 
Signed-off-by: Writer-X <1256866856@qq.com>
 2023-09-08 10:03:16 +08:00
Gao
41d37409e9
Remove unused search conf (#26689) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-08-31 10:23:07 +08:00
MrPresent-Han
7d5a4b2994
add more event for segcore search(#26277) (#26688) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-08-30 14:15:01 +08:00
cai.zhang
d6473028d1
Fix bug for erase while traversing (#26540) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-08-23 09:34:22 +08:00
cai.zhang
c073aa0dc3
Fix bug for json_contains_all has multiple array elements (#26446) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-08-18 22:36:19 +08:00
Jiquan Long
95bcf3e900
Fix error message of proto parser not clear (#26365) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-08-16 20:20:17 +08:00
Enwei Jiao
9cb5943e71
Fix compile error on Arm (#26319) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-08-14 12:43:31 +08:00
cai.zhang
a0198ce8ae
Support json contains feature (#25384) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-08-11 17:09:30 +08:00
zhagnlu
9489e14000
Optimize multi logical exprs performance when meet some situations (#26265) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-08-11 15:31:29 +08:00
yah01
300fef446b
Enable mmap for vector index (#25877) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-10 13:59:15 +08:00
Jiquan Long
6578a27fe7
Support empty expression (#26153) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-08-10 08:37:15 +08:00
Jiquan Long
bafb183a2b
Optimize bitset usage (#26096) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-08-03 15:25:09 +08:00
Jiquan Long
5c1f79dc54
Push down the limit operator to segcore (#25959) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-08-01 20:29:05 +08:00
Cai Yudong
9a4761dcc7
Remove binary metrics TANIMOTO/SUPERSTRUCTURE/SUBSTRUCTURE (#25708) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-07-19 16:16:58 +08:00
Jiquan Long
96141fa998
Fix coping segment offsets twice (#25729) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-07-19 14:24:57 +08:00
zhagnlu
65cb52d06b
Support dynamic simd framework and using term expr as example (#25260) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-07-13 16:22:30 +08:00
yah01
cb721781f3
Improve error message throwed from knowhere (#25473) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-07-11 17:26:29 +08:00
yihao.dai
2d1ed6af45
Use PK index for string data type (#25390) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-07-07 15:26:26 +08:00
Jiquan Long
823ede3a51
Fix BinaryRange expression on indexed chunk (#25372) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-07-06 21:10:25 +08:00
Jiquan Long
c2a2ece551
Fix expression on integer overflow case (#25320) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-07-05 10:18:24 +08:00
foxspy
31173727b2
growing segment index memory opt & get vector bugfix (#25272) 
Signed-off-by: xianliang <xianliang.li@zilliz.com>
 2023-07-05 00:04:25 +08:00
Enwei Jiao
816158e4af
Remove outdated searchplan (#25282) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-07-04 18:30:25 +08:00
yah01
73b7cf5df4
Fix bool unary expression sometimes returns no data (#25016) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-06-20 16:38:41 +08:00
Jiquan Long
8885fa2bc7
Fix BinaryArithOpEvalRange integer overflow case (#24943) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-06-16 17:10:39 +08:00
zhagnlu
c5b1533fdc
Change segcore search_id from travelling all bits to select true bits(#24659) (#24800) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-06-16 16:48:44 +08:00
Jiquan Long
26a6e1b946
Fix BinaryRange expression on integer overflow case (#24920) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-06-16 09:56:39 +08:00
zhagnlu
f3f3f8a849
Segcore retrieve by pk optimazation (#24659) (#24660) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-06-15 15:04:39 +08:00
Jiquan Long
7faf934f91
Fix UnaryRange expression on integer overflow case (#24893) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-06-15 14:06:39 +08:00
zhagnlu
f60b839127
Support element in json array in segcore part(#24677) (#24829) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-06-14 14:38:37 +08:00
Jiquan Long
c6a6998ba7
Fix term expression on interger overflow case (#24867) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-06-14 11:44:38 +08:00
Cai Yudong
1b3c4b26f1
Optimize range search result sort in segcore (#24837) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-06-13 19:22:38 +08:00
yah01
852e5ce904
Convert double to int if possible with term expression (#24530) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-05-30 22:03:28 +08:00
yah01
99b7a6882c
Fix the nested path lost when reuse the expression (#24417) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-05-25 21:35:27 +08:00