3134 Commits

Author SHA1 Message Date
laurazhao0611
82d85e8c57
test: modify part of milvus client test cases with load func (#38654)
/kind improvment
/cc @yanliang567 @binbinlv

Signed-off-by: laurazhao0611 <laura.zhao@zilliz.com>
2024-12-23 17:16:48 +08:00
Xianhui Lin
90de37e60a
enhance: field stringtype maxlength raise to 1M (#38592)
enhance: field stringtype maxlength raise to 1M
issue: https://github.com/milvus-io/milvus/issues/37436

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-20 14:32:45 +08:00
congqixia
c39db11509
enhance: [GoSDK] Sync API names and add missing APIs (#38603)
Related to #31293

- Rename `UsingDatabase` to `UseDatabase`
- Uncomment default value methods
- Add missing RBAC APIs
- Add some resource group APIs

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-20 11:52:46 +08:00
XuanYang-cn
ca7ec23198
enhance: Use partitionID when delete by partitionKey (#38231)
When delete by partition_key, Milvus will generates L0 segments
globally. During L0 Compaction, those L0 segments will touch all
partitions collection wise. Due to the false-positive rate of segment
bloomfilters, L0 compactions will append false deltalogs to completed
irrelevant partitions, which causes *partition deletion amplification.

This PR uses partition_key to set targeted partitionID when producing
deleteMsgs into MsgStreams. This'll narrow down L0 segments scope to
partition level, and remove the false-positive influence
collection-wise.

However, due to DeleteMsg structure, we can only label one partition to
one deleteMsg, so this enhancement fails if user wants to delete over 2
partition_keys in one deletion.

See also: #34665

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-20 11:18:46 +08:00
Bingyi Sun
894c203f27
enhance: allow hyphen in partition name (#38494)
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-18 15:04:45 +08:00
nico
fb0e689617
test: update sdk version and add cases (#38478)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-12-18 10:10:57 +08:00
zhuwenxing
b694e06488
test: add lib for supporting higher python version (#38475)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-12-17 14:48:50 +08:00
Bingyi Sun
3e2a2f278b
enhance: Handle rust error in c++ (#38113)
https://github.com/milvus-io/milvus/issues/37930

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-16 19:40:45 +08:00
ThreadDao
8794ec966e
test: add go case for groupby search (#38411)
issue: #33419

---------

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-12-16 10:44:43 +08:00
tinswzy
27229f7907
enhance: refine exists log print with ctx (#38080)
issue: #35917 
Refines exists log print with ctx

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-14 22:36:44 +08:00
congqixia
fe79babdb3
enhance: Bump golang/x/crypto for all packages (#38445)
Related to #38446
See also https://github.com/milvus-io/milvus/security/dependabot/103

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-13 19:12:42 +08:00
Xianhui Lin
f2d79b3d3a
enhance: altercollection validation logic (#38419)
altercollection validation logic
issue: https://github.com/milvus-io/milvus/issues/37436

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-12 21:42:42 +08:00
ThreadDao
59234a3350
test: add log level for go sdk test and update part cases (#38385)
- add log level for go sdk cases
- update cases for issue #33460 & #37853

---------

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-12-12 16:58:43 +08:00
Xianhui Lin
74c6e0e94b
enhance: alterindex & altercollection supports altering properties updata pymilvus (#37437) (#38362)
enhance : 

1. alterindex delete properties
We have introduced a new parameter deleteKeys to the alterindex
functionality, which allows for the deletion of properties within an
index. This enhancement provides users with the flexibility to manage
index properties more effectively by removing specific keys as needed.
2. altercollection delete properties
We have introduced a new parameter deleteKeys to the altercollection
functionality, which allows for the deletion of properties within an
collection. This enhancement provides users with the flexibility to
manage collection properties more effectively by removing specific keys
as needed.

3.support altercollectionfield
We currently support modifying the fieldparams of a field in a
collection using altercollectionfield, which only allows changes to the
max-length attribute.
Key Points:
- New Parameter - deleteKeys: This new parameter enables the deletion of
specified properties from an index. By passing a list of keys to
deleteKeys, users can remove the corresponding properties from the
index.

- Mutual Exclusivity: The deleteKeys parameter cannot be used in
conjunction with the extraParams parameter. Users must choose one
parameter to pass based on their requirement. If deleteKeys is provided,
it indicates an intent to delete properties; if extraParams is provided,
it signifies the addition or update of properties.

issue: https://github.com/milvus-io/milvus/issues/37436

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-11 13:00:44 +08:00
wei liu
a118ca14a7
fix: Fix role be dropped when grant still exist. (#38342)
issue: #38325
the old impl only to check grant in default db before drop role, which
may cause role be dropped when grant still exist.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-11 11:24:42 +08:00
nico
d9a151e9e4
test: update test cases (#38330)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-12-10 18:46:42 +08:00
nico
224c2c8e6d
enhance: update pymilvus version (#38225)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-12-09 19:52:47 +08:00
zhuwenxing
8d2eefb264
test: add restful testcases for database api (#38281)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-12-07 14:42:45 +08:00
cai.zhang
73aa95f596
fix: Add version to the proxy cache to resolve concurrency issues (#38067)
issue: #36989

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-04 18:06:39 +08:00
shaoyue
1f66b9ebfb
feat: add config field to set internal tls sni (#38124)
/cc @xiaofan-luan @jaime0815 @nish112022

part of https://github.com/milvus-io/milvus/issues/36864

Signed-off-by: haorenfsa <haorenfsa@gmail.com>
2024-12-04 14:56:47 +08:00
sthuang
a5e0a56a8e
fix: move grant/revoke v2 params check from rootcoord to proxy (#38130)
related issue: https://github.com/milvus-io/milvus/issues/37031

fixed issues #38042: The interface "grant_v2" does not support empty
collectionName while the error says it does

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-02 19:48:37 +08:00
binbin
4c623cebeb
enhance: [skip e2e] Update test plugin version (#38106)
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-12-02 12:00:37 +08:00
sthuang
23dc313c44
fix: fix grant/revoke v2 meta and unclear error messages (#38110)
related issue: https://github.com/milvus-io/milvus/issues/37031

fixed issues:
#37974: better error messages for grant v2 interface
#37903: fix meta built-in privilege group object name
#37843: better error messages for custom privilege group interface 
#38002: fix built-in privilege group meta to pass proxy interceptor
check
#38008: fix revoke v2 to support revoking v1 granted privileges

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-02 11:36:39 +08:00
wt
ec119f5f14
test: update check search result (#38099)
issue: #37271

Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-12-01 17:50:37 +08:00
qixuan
bec54998e4
test: update resource group case about error msg and update_resource_group (#38094)
Signed-off-by: qixuan <673771573@qq.com>
2024-11-30 13:32:36 +08:00
Yinzuo Jiang
1ce609834a
feat: [GoSDK] fp32 <-> fp16/bf16 vector conversion (#37978)
Add the following methods for convenient fp32 vector <-> fp16/bf16
vector conversion

fp32 <-> fp16/bf16 vector conversion:

- `func (fv FloatVector) ToFloat16Vector() Float16Vector`
- `func (fv FloatVector) ToBFloat16Vector() BFloat16Vector`
- `func (fv Float16Vector) ToFloat32Vector() FloatVector`
- `func (fv BFloat16Vector) ToFloat32Vector() FloatVector`

`columnBasedDataOption`:

- `func (opt *columnBasedDataOption) WithFloat16VectorColumn(colName
string, dim int, data [][]float32) *columnBasedDataOption`
- `func (opt *columnBasedDataOption) WithBFloat16VectorColumn(colName
string, dim int, data [][]float32) *columnBasedDataOption`

`ColumnFloat16Vector`/`ColumnBFloat16Vector`:

- `func NewColumnFloat16VectorFromFp32Vector(fieldName string, dim int,
data [][]float32) *ColumnFloat16Vector`
- `func NewColumnBFloat16VectorFromFp32Vector(fieldName string, dim int,
data [][]float32) *ColumnBFloat16Vector`
- support []float32 or `entity.FloatVector` in
    - `func (c *ColumnFloat16Vector) AppendValue(i interface{}) error`
    - `func (c *ColumnFloat16Vector) AppendValue(i interface{}) error`

issue: #37448

Signed-off-by: Yinzuo Jiang <yinzuo.jiang@zilliz.com>
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-11-29 08:00:37 +08:00
zhuwenxing
5e807c3649
test:[skip e2e] update pulsar labelSelectors to v3 in chaos test (#38049)
After upgrading pulsar from v2 to v3, the labels of the helm deployed
pulsar were changed from app=pulsarv to app=pulsarv3.
Therefore, it is necessary to modify this part of the labelSelector so
that chaos mesh can correctly inject faults

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-28 14:18:35 +08:00
zhuwenxing
864ed1a3be
test: update python requirements in test dir (#38047)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-27 19:34:35 +08:00
zhuwenxing
e5775a71af
test: add bulk import testcases for full text search (#37197)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-27 19:32:42 +08:00
zhuwenxing
8188e1472d
test: add custom analyzer testcases (#37781)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-27 16:08:36 +08:00
ThreadDao
c5327ff150
test: add cases for gosdk v2 partial load (#37924)
issue: #33419

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-11-26 15:08:34 +08:00
congqixia
6b23e668b0
test: Update go_client version & update latest changes (#37793)
Related to #37768

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-26 10:26:40 +08:00
zhuwenxing
27c22d1166
test: remove sparse embedding in output fields for fts cases (#37980)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-25 19:14:33 +08:00
qixuan
4cd79323bf
test: Add cases for restful about default and none (#37943)
issue: #36129

Signed-off-by: qixuan <673771573@qq.com>
2024-11-25 14:06:33 +08:00
zhuwenxing
0b9edb62a9
test: add testcases contain growing segments (#37262)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-25 10:40:34 +08:00
Yinzuo Jiang
5a06faca39
feat: fp32 vector to fp16/bf16 vector conversion for RESTful API (#37556)
RESTful API. The influenced API are as follows:

- Handler. insert
- HandlerV1. insert/upsert
- HandlerV2. insert/upsert/search

We do not modify search API in Handler/HandlerV1 because they do not
support fp16/bf16 vectors.

module github.com/milvus-io/milvus/pkg:

Add `Float32ArrayToBFloat16Bytes()`, `Float32ArrayToFloat16Bytes()` and
`Float32ArrayToBytes()`. These method will be used in GoSDK in the
future.

issue: #37448

Signed-off-by: Yinzuo Jiang <yinzuo.jiang@zilliz.com>
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-11-24 17:46:33 +08:00
ThreadDao
bc131a911c
test: add cases for gosdk v2 load release (#37831)
issue: #33419

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-11-22 10:54:32 +08:00
sthuang
19572f5b06
enhance: RBAC new grant/revoke privilege (#37785)
issue: https://github.com/milvus-io/milvus/issues/37031
also fix issues: https://github.com/milvus-io/milvus/issues/37843,
https://github.com/milvus-io/milvus/issues/37842,
https://github.com/milvus-io/milvus/issues/37887

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-21 22:20:34 +08:00
Buqian Zheng
511edd29fd
enhance: disallow get raw vector data of a BM25 Function output field (#37800)
issue: https://github.com/milvus-io/milvus/issues/35853

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-11-20 14:22:30 +08:00
nish112022
484c6b5c44
feat: Added code for Internal-tls (#36865)
issue : https://github.com/milvus-io/milvus/issues/36864

I have a few questions regarding my approach.I will consolidate them
here for feedback and review.Thanks

---------

Signed-off-by: Nischay Yadav <nischay.yadav@ibm.com>
Signed-off-by: Nischay <Nischay.Yadav@ibm.com>
2024-11-20 06:00:32 +08:00
nico
866d512f6f
test: update test cases and sdk version (#37774)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-11-19 14:14:32 +08:00
cai.zhang
dae4160466
enhance: Whether to enable mergeSort mode when performing mixCompaction (#37664)
issue: #37579

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-19 11:28:31 +08:00
sthuang
2d72ad33f2
enhance: RBAC built in privilege groups (#37720)
issue: #37031

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-18 20:38:39 +08:00
zhuwenxing
3cdb485022
test: update timeout of load in ci cases (#37596)
action for
https://github.com/milvus-io/milvus/issues/37166#issuecomment-2469502955

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-16 16:00:31 +08:00
zhuwenxing
3f7352f3cf
test: remove xfail of fts test cases after fix (#37724)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-16 11:14:30 +08:00
wei liu
a1b6be1253
fix: Delegator stuck at unserviceable status (#37694)
issue: #37679

pr #36549 introduce the logic error which update current target when
only parts of channel is ready.

This PR fix the logic error and let dist handler keep pull distribution
on querynode until all delegator becomes serviceable.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-15 10:20:31 +08:00
foxspy
5ae347aba0
enhance: update knowhere version (#37688)
issue: #37665 #37631 #37620 #37587 #36906 
knowhere has add default nlist value, so some invalid param test ut with
no nlist param will be valid.

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-11-15 10:10:31 +08:00
nico
405c12f7d1
test: update test cases (#37660)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-11-14 17:46:39 +08:00
yanliang567
af433ffd68
test: Add expression template tests and use error msg only as assertion (#37618)
related issue: https://github.com/milvus-io/milvus/issues/37451

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-11-13 17:14:30 +08:00
smellthemoon
a654487995
fix: some error in restful (#37487)
#37370

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-13 17:12:39 +08:00