admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2025-12-07 01:28:27 +08:00
Code Issues Packages Projects Releases Wiki Activity
22,347 Commits 133 Branches 183 Tags
Commit Graph

39 Commits

Author SHA1 Message Date
Ted Xu
ae32203d3a
fix: support group by with nullable grouping keys (#41797) 
See #36264

In this PR:
- Enhanced error handling in parse of grouping field.
- Fixed null handling in reduce tasks in proxy nodes. 
- Updated tests to reflect changes in error handling and data processing
logic.

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-05-17 20:54:22 +08:00
junjiejiangjjj
f23df95a77
feat : Support decay rerank (#41223) 
https://github.com/milvus-io/milvus/issues/35856
#41312

Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com>
 2025-04-23 20:48:39 +08:00
SimFG
91d40fa558
fix: Update logging context and upgrade dependencies (#41318) 
- issue: #41291

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2025-04-23 10:52:38 +08:00
Ted Xu
d50781c8cc
enhance: support nullable group by keys (#41313) 
See #36264

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-04-18 10:08:34 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990) 
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-22 23:15:58 +08:00
Chun Han
ed31a5a4bf
enhance: fix inconsistenty of alias and db for query iterator(#39045) (#39216) 
related: #39045

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-01-15 09:48:59 +08:00
Zhen Ye
bb8d1ab3bf
enhance: make new go package to manage proto (#39114) 
issue: #39095

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-01-10 10:49:01 +08:00
Patrick Weizhi Xu
85f462be1a
enhance: speed up search iterator stage 1 (#37947) 
issue: #37548

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-12-26 10:32:49 +08:00
cai.zhang
2356425530
fix: Support hybrid search with expression template (#38623) 
issue: #38616

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-12-23 14:32:51 +08:00
XuanYang-cn
ca7ec23198
enhance: Use partitionID when delete by partitionKey (#38231) 
When delete by partition_key, Milvus will generates L0 segments
globally. During L0 Compaction, those L0 segments will touch all
partitions collection wise. Due to the false-positive rate of segment
bloomfilters, L0 compactions will append false deltalogs to completed
irrelevant partitions, which causes *partition deletion amplification.

This PR uses partition_key to set targeted partitionID when producing
deleteMsgs into MsgStreams. This'll narrow down L0 segments scope to
partition level, and remove the false-positive influence
collection-wise.

However, due to DeleteMsg structure, we can only label one partition to
one deleteMsg, so this enhancement fails if user wants to delete over 2
partition_keys in one deletion.

See also: #34665

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-12-20 11:18:46 +08:00
Gao
994fc544e7
enhance: support iterative filter execution (#37363) 
issue: #37360

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-12-11 11:32:44 +08:00
Chun Han
2d29dcd30c
enhance:refine group_strict_size parameter(#37482) (#37483) 
related: #37482

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-11-12 09:56:28 +08:00
smellthemoon
86b9c3ef4a
fix: to just check null in group by field only (#37191) 
#37187

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-10-29 15:38:30 +08:00
Chun Han
eccc326e8b
enhance: report err when group_size is wrong(#36146) (#36908) 
related: #36146

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-10-17 16:05:29 +08:00
Chun Han
903450f5c6
enhance: add ts support for iterator(#22718) (#36572) 
related: #22718

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-10-16 18:51:23 +08:00
Chun Han
eb23e23cd2
enhance: refine parameter relationship for hybridsearch_group_by(#35096) (#36289) 
related: #35096

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-09-20 14:55:11 +08:00
zhenshan.cao
dcd904d2fa
fix: keep inner topK to avoid exceeding efSearch (#36284) 
issue: #https://github.com/milvus-io/milvus/issues/36243

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-09-16 11:01:25 +08:00
Chun Han
b8b4aea4f5
enhance: restrict max group size(#33544) (#36223) 
related: #33544

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-09-14 15:45:08 +08:00
zhenshan.cao
9d8d332c88
fix: Fix improper use of offset in HybridSearch (#36244) 
issue :https://github.com/milvus-io/milvus/issues/36243

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-09-13 22:05:15 +08:00
Chun Han
e480b103bd
feat: supporing hybrid search group_by (#35982) 
related: #35096

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-09-08 17:09:04 +08:00
smellthemoon
21b135c7c2
fix: not append valid data when transfer to insert record (#36027) 
fix not append valid data when transfer to insert record and add a tiny
check when in groupBy field.
#35924

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-09-06 14:53:04 +08:00
Chun Han
4641fd9195
enhance: make search groupby stop when reaching topk groups (#35814) 
related: #33544

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-09-02 18:25:03 +08:00
Chun Han
bfd9d86fe9
feat: support groupby size on go-layer(#33544) (#33845) 
related: #33544

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-27 14:21:00 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394) 
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-29 11:31:51 +08:00
jaime
21fc5f5d46
enhance: Remove datanode reporting TT based on MQ implementation (#34421) 
issue: #34420

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-05 15:48:09 +08:00
Chun Han
10b3ce24ba
enhance: use configed max topk for iterator when input topk exceeds (#34290) 
related: #34290

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-07-04 14:24:19 +08:00
jaime
9c2eeff427
enhance: refine error message for invalid limit parameter (#34178) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-06-26 22:14:04 +08:00
smellthemoon
365e50b63e
fix: revert add range search params check in proxy (#32366) 
no need to check params in empty segment.
#30365

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-04-23 17:41:23 +08:00
zhenshan.cao
089c805e0a
enhance:Refactor hybrid search (#32020) 
issue: https://github.com/milvus-io/milvus/issues/25639
https://github.com/milvus-io/milvus/issues/31368

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-04-09 14:21:18 +08:00
congqixia
1af2ee903c
fix: Check parse QueryInfo error before using it (#31836) 
See also milvus-io/milvus-sdk-go#693 #31735

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-03 11:29:14 +08:00
Chun Han
92c84ccc3f
enhance: ban groupby on hybrid-search(#29968) (#31810) 
related: #29968

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-04-02 15:47:18 +08:00
Chun Han
b99c46246c
enhance: ban groupby on binary vector(#31134) (#31659) 
related: #31134

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-03-28 15:19:10 +08:00
Chun Han
c3264ca3e3
feat: support segment pruner (#31003) 
related: #30376
 2024-03-22 13:57:06 +08:00
Patrick Weizhi Xu
982dd2834b
enhance: add materialized view search info (#30888) 
issue: #29892 

This PR
1. Pass Materialized View (MV) search information obtained from the
expression parsing planning procedure to Knowhere. It only performs when
MV is enabled and the partition key is involved in the expression. The
search information includes:
1. Touched field_id and the count of related categories in the
expression. E.g., `color == red && color == blue` yields `field_id ->
2`.
2. Whether the expression only includes AND (&&) logical operator,
default `true`.
    3. Whether the expression has NOT (!) operator, default `false`.
4. Store if turning on MV on the proxy to eliminate reading from
paramtable for every search request.
5. Renames to MV.

## Rebuttals
1. Did not write in `ExtractInfoPlanNodeVisitor` since the new scalar
framework was introduced and this part might be removed in the future.
2. Currently only interested in `==` and `in` expression, `string` data
type, anything else is a bonus.
3. Leave handling expressions like `F == A || F == A` for future works
of the optimizer.

## Detailed MV Info

![image](https://github.com/milvus-io/milvus/assets/6563846/b27c08a0-9fd3-4474-8897-30a3d6d6b36f)

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-03-21 11:19:07 +08:00
MrPresent-Han
d0eeea4b44
fix: reduce incorrectly for group-by with offset(#30828) (#30882) 
related: #30828

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-03-06 16:47:00 +08:00
congqixia
9b3005f1be
enhance: Avoid create schema helper for each read task (#30981) 
See also #30806

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-04 19:39:00 +08:00
MrPresent-Han
77eb6defb1
feat: support groupby on growing and non-indexed sealed egment(#30307) (#30644) 
related: #30308

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-02-21 14:02:53 +08:00
xige-16
0a78b38bb8
fix: fix requery without partitionIDs in hybrid search (#30444) 
issue: #30412 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2024-02-02 16:47:13 +08:00
xige-16
060c8603a3
fix: Support mvcc with hybrid serach (#30114) 
issue: https://github.com/milvus-io/milvus/issues/29656
/kind bug

Signed-off-by: xige-16 <xi.ge@zilliz.com>

---------

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2024-02-01 16:03:03 +08:00