3154 Commits

Author SHA1 Message Date
sthuang
4a47f760b3
fix: [2.5] rbac custom group privilege level check (#39164) (#39224)
cherry-pick from master: https://github.com/milvus-io/milvus/pull/39164
related: https://github.com/milvus-io/milvus/issues/39086

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-01-14 16:50:59 +08:00
zhenshan.cao
99a8274326
enhance: Add primary field name in SearchResult and QueryResults (#39222)
pr: https://github.com/milvus-io/milvus/pull/39220
issue: https://github.com/milvus-io/milvus/issues/39219

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2025-01-14 16:10:58 +08:00
Zhen Ye
54036bcafd
enhance: add broadcast operation for msgstream (#39119)
issue: #38399
pr: #39040

- make broadcast service available for msgstream by reusing the
architecture streaming service

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-14 15:11:00 +08:00
congqixia
ac730be5aa
enhance: Bump milvus & proto version to v2.5.3 (#39198)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-13 12:56:57 +08:00
sthuang
3059fae40d
fix: [2.5] fix restore rbac empty meta crash (#39143)
cp from master: https://github.com/milvus-io/milvus/pull/39141
related: https://github.com/milvus-io/milvus/issues/38985

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-01-13 12:04:58 +08:00
zhuwenxing
95df38e24f
test: add text match negative test case (#38982)
pr: https://github.com/milvus-io/milvus/pull/38981

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-01-11 14:21:03 +08:00
Zhen Ye
95809ca767
enhance: make new go package to manage proto (#39128)
issue: #39095
pr: #39114

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-10 10:53:01 +08:00
ThreadDao
c2532def02
test: update cases for search params support and groupby issue fix (#38876)
pr: #38852
- update cases for WithSearchParam
- remove skip for groupby search issue
- update output query dynamic case for server changes

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2025-01-09 19:12:56 +08:00
yanliang567
76c66f07be
test: [cp2.5]Add search with hints for different expressions (#39062)
issue: https://github.com/milvus-io/milvus/issues/38877
pr: #39048

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-01-08 16:24:57 +08:00
cai.zhang
e6dd3e5a57
fix: [2.5]Remove valid expressions from invalid expressions (#38999)
issue: #39014 

master pr: #38957 
master pr: #39012

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-01-06 18:02:55 +08:00
yanliang567
aceb028e3a
test: [cp2.5]Refactor pymilvus client v2 testbase and add a test for search hint (#39006)
issue: https://github.com/milvus-io/milvus/issues/38877

    refactor pymilvus client v2 testcasebase
    add a test for search hint
    update pymilvus to 2.6

pr: #38939

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-01-06 17:02:55 +08:00
nico
498b984607
enhance: delete one pod pipeline in nightly and open mmap in standalone(2.5) (#38888)
pr: #38887

Signed-off-by: nico <cheng.yuan@zilliz.com>
2025-01-03 16:38:55 +08:00
congqixia
6433f76c61
enhance: [2.5] Bump golang/x/net to v0.33 fixing security alarm (#38850) (#38932)
Cherry-pick from master
pr: #38850
Related to https://github.com/milvus-io/milvus/security/dependabot/106

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-03 14:50:54 +08:00
Patrick Weizhi Xu
1d55ad6a1d
enhance: [2.5] add param for tuning max VARCHAR length and restore limit to 65535 (#38883)
issue: #38882
pr: https://github.com/milvus-io/milvus/pull/38884

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit a64a7377ea8258efe36c74bd1cf9db556726ba2d)
2025-01-02 13:56:54 +08:00
ThreadDao
cf148f9bd2
test: [2.5] add cases for async milvus client (#38699) (#38853)
pr: #38699
issue: #38697

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-12-31 11:30:52 +08:00
yanliang567
e6c3d40d90
test: [cp2.5]Add alter API tests (#38843)
related issue: https://github.com/milvus-io/milvus/issues/38471
related pr: #38659 

Also add a test for hybrid search with expr templates

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-12-30 16:52:52 +08:00
foxspy
3d723676ac
enhance: [2.5]update knowhere version (#38785)
/kind branch-feature
release note: https://github.com/zilliztech/knowhere/releases/tag/v2.5.1

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-12-27 10:24:50 +08:00
congqixia
a67b57275c
enhance: Bump milvus & proto version to v2.5.1 (#38774)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-26 16:36:50 +08:00
sthuang
a4901ef7ec
fix: [2.5] fix privilege group list and list collections (#38738)
cherry-pick from: https://github.com/milvus-io/milvus/pull/38684
related: https://github.com/milvus-io/milvus/issues/37031

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-25 18:06:50 +08:00
wei liu
b16d04d7cc
fix: Fix update loading collection's load config doesn't work (#38737)
issue: #38594 
pr: #38595

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-25 15:02:50 +08:00
Xianhui Lin
3cf9b6bae5
enhance: [2.5]field stringtype maxlength raise to 1M (#38614)
enhance: field stringtype maxlength raise to 1M
issue: https://github.com/milvus-io/milvus/issues/37436
master pr: https://github.com/milvus-io/milvus/pull/38592

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-20 17:08:45 +08:00
congqixia
b132438c03
enhance: Bump milvus & proto verison to v2.5.0 (#38610)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-20 14:30:46 +08:00
congqixia
c39db11509
enhance: [GoSDK] Sync API names and add missing APIs (#38603)
Related to #31293

- Rename `UsingDatabase` to `UseDatabase`
- Uncomment default value methods
- Add missing RBAC APIs
- Add some resource group APIs

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-20 11:52:46 +08:00
XuanYang-cn
ca7ec23198
enhance: Use partitionID when delete by partitionKey (#38231)
When delete by partition_key, Milvus will generates L0 segments
globally. During L0 Compaction, those L0 segments will touch all
partitions collection wise. Due to the false-positive rate of segment
bloomfilters, L0 compactions will append false deltalogs to completed
irrelevant partitions, which causes *partition deletion amplification.

This PR uses partition_key to set targeted partitionID when producing
deleteMsgs into MsgStreams. This'll narrow down L0 segments scope to
partition level, and remove the false-positive influence
collection-wise.

However, due to DeleteMsg structure, we can only label one partition to
one deleteMsg, so this enhancement fails if user wants to delete over 2
partition_keys in one deletion.

See also: #34665

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-20 11:18:46 +08:00
Bingyi Sun
894c203f27
enhance: allow hyphen in partition name (#38494)
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-18 15:04:45 +08:00
nico
fb0e689617
test: update sdk version and add cases (#38478)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-12-18 10:10:57 +08:00
zhuwenxing
b694e06488
test: add lib for supporting higher python version (#38475)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-12-17 14:48:50 +08:00
Bingyi Sun
3e2a2f278b
enhance: Handle rust error in c++ (#38113)
https://github.com/milvus-io/milvus/issues/37930

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-16 19:40:45 +08:00
ThreadDao
8794ec966e
test: add go case for groupby search (#38411)
issue: #33419

---------

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-12-16 10:44:43 +08:00
tinswzy
27229f7907
enhance: refine exists log print with ctx (#38080)
issue: #35917 
Refines exists log print with ctx

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-14 22:36:44 +08:00
congqixia
fe79babdb3
enhance: Bump golang/x/crypto for all packages (#38445)
Related to #38446
See also https://github.com/milvus-io/milvus/security/dependabot/103

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-13 19:12:42 +08:00
Xianhui Lin
f2d79b3d3a
enhance: altercollection validation logic (#38419)
altercollection validation logic
issue: https://github.com/milvus-io/milvus/issues/37436

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-12 21:42:42 +08:00
ThreadDao
59234a3350
test: add log level for go sdk test and update part cases (#38385)
- add log level for go sdk cases
- update cases for issue #33460 & #37853

---------

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-12-12 16:58:43 +08:00
Xianhui Lin
74c6e0e94b
enhance: alterindex & altercollection supports altering properties updata pymilvus (#37437) (#38362)
enhance : 

1. alterindex delete properties
We have introduced a new parameter deleteKeys to the alterindex
functionality, which allows for the deletion of properties within an
index. This enhancement provides users with the flexibility to manage
index properties more effectively by removing specific keys as needed.
2. altercollection delete properties
We have introduced a new parameter deleteKeys to the altercollection
functionality, which allows for the deletion of properties within an
collection. This enhancement provides users with the flexibility to
manage collection properties more effectively by removing specific keys
as needed.

3.support altercollectionfield
We currently support modifying the fieldparams of a field in a
collection using altercollectionfield, which only allows changes to the
max-length attribute.
Key Points:
- New Parameter - deleteKeys: This new parameter enables the deletion of
specified properties from an index. By passing a list of keys to
deleteKeys, users can remove the corresponding properties from the
index.

- Mutual Exclusivity: The deleteKeys parameter cannot be used in
conjunction with the extraParams parameter. Users must choose one
parameter to pass based on their requirement. If deleteKeys is provided,
it indicates an intent to delete properties; if extraParams is provided,
it signifies the addition or update of properties.

issue: https://github.com/milvus-io/milvus/issues/37436

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-11 13:00:44 +08:00
wei liu
a118ca14a7
fix: Fix role be dropped when grant still exist. (#38342)
issue: #38325
the old impl only to check grant in default db before drop role, which
may cause role be dropped when grant still exist.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-11 11:24:42 +08:00
nico
d9a151e9e4
test: update test cases (#38330)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-12-10 18:46:42 +08:00
nico
224c2c8e6d
enhance: update pymilvus version (#38225)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-12-09 19:52:47 +08:00
zhuwenxing
8d2eefb264
test: add restful testcases for database api (#38281)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-12-07 14:42:45 +08:00
cai.zhang
73aa95f596
fix: Add version to the proxy cache to resolve concurrency issues (#38067)
issue: #36989

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-04 18:06:39 +08:00
shaoyue
1f66b9ebfb
feat: add config field to set internal tls sni (#38124)
/cc @xiaofan-luan @jaime0815 @nish112022

part of https://github.com/milvus-io/milvus/issues/36864

Signed-off-by: haorenfsa <haorenfsa@gmail.com>
2024-12-04 14:56:47 +08:00
sthuang
a5e0a56a8e
fix: move grant/revoke v2 params check from rootcoord to proxy (#38130)
related issue: https://github.com/milvus-io/milvus/issues/37031

fixed issues #38042: The interface "grant_v2" does not support empty
collectionName while the error says it does

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-02 19:48:37 +08:00
binbin
4c623cebeb
enhance: [skip e2e] Update test plugin version (#38106)
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-12-02 12:00:37 +08:00
sthuang
23dc313c44
fix: fix grant/revoke v2 meta and unclear error messages (#38110)
related issue: https://github.com/milvus-io/milvus/issues/37031

fixed issues:
#37974: better error messages for grant v2 interface
#37903: fix meta built-in privilege group object name
#37843: better error messages for custom privilege group interface 
#38002: fix built-in privilege group meta to pass proxy interceptor
check
#38008: fix revoke v2 to support revoking v1 granted privileges

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-02 11:36:39 +08:00
wt
ec119f5f14
test: update check search result (#38099)
issue: #37271

Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-12-01 17:50:37 +08:00
qixuan
bec54998e4
test: update resource group case about error msg and update_resource_group (#38094)
Signed-off-by: qixuan <673771573@qq.com>
2024-11-30 13:32:36 +08:00
Yinzuo Jiang
1ce609834a
feat: [GoSDK] fp32 <-> fp16/bf16 vector conversion (#37978)
Add the following methods for convenient fp32 vector <-> fp16/bf16
vector conversion

fp32 <-> fp16/bf16 vector conversion:

- `func (fv FloatVector) ToFloat16Vector() Float16Vector`
- `func (fv FloatVector) ToBFloat16Vector() BFloat16Vector`
- `func (fv Float16Vector) ToFloat32Vector() FloatVector`
- `func (fv BFloat16Vector) ToFloat32Vector() FloatVector`

`columnBasedDataOption`:

- `func (opt *columnBasedDataOption) WithFloat16VectorColumn(colName
string, dim int, data [][]float32) *columnBasedDataOption`
- `func (opt *columnBasedDataOption) WithBFloat16VectorColumn(colName
string, dim int, data [][]float32) *columnBasedDataOption`

`ColumnFloat16Vector`/`ColumnBFloat16Vector`:

- `func NewColumnFloat16VectorFromFp32Vector(fieldName string, dim int,
data [][]float32) *ColumnFloat16Vector`
- `func NewColumnBFloat16VectorFromFp32Vector(fieldName string, dim int,
data [][]float32) *ColumnBFloat16Vector`
- support []float32 or `entity.FloatVector` in
    - `func (c *ColumnFloat16Vector) AppendValue(i interface{}) error`
    - `func (c *ColumnFloat16Vector) AppendValue(i interface{}) error`

issue: #37448

Signed-off-by: Yinzuo Jiang <yinzuo.jiang@zilliz.com>
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-11-29 08:00:37 +08:00
zhuwenxing
5e807c3649
test:[skip e2e] update pulsar labelSelectors to v3 in chaos test (#38049)
After upgrading pulsar from v2 to v3, the labels of the helm deployed
pulsar were changed from app=pulsarv to app=pulsarv3.
Therefore, it is necessary to modify this part of the labelSelector so
that chaos mesh can correctly inject faults

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-28 14:18:35 +08:00
zhuwenxing
864ed1a3be
test: update python requirements in test dir (#38047)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-27 19:34:35 +08:00
zhuwenxing
e5775a71af
test: add bulk import testcases for full text search (#37197)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-27 19:32:42 +08:00
zhuwenxing
8188e1472d
test: add custom analyzer testcases (#37781)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-27 16:08:36 +08:00