204 Commits

Author SHA1 Message Date
Zhen Ye
5551d99425
enhance: remove old arch non-streaming arch code (#43651)
issue: #41609

- remove all dml dead code at proxy
- remove dead code at l0_write_buffer
- remove msgstream dependency at proxy
- remove timetick reporter from proxy
- remove replicate stream implementation

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-08-06 14:41:40 +08:00
Spade A
faeb7fd410
feat: impl StructArray -- create schema, insert, and retrieve data (#42855)
Ref https://github.com/milvus-io/milvus/issues/42148

https://github.com/milvus-io/milvus/pull/42406 impls the segcore part of
storage for handling with VectorArray.
This PR:
1. impls the go part of storage for VectorArray
2. impls the collection creation with StructArrayField and VectorArray
3. insert and retrieve data from the collection.

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
2025-07-27 01:30:55 +08:00
Jean-Francois Weber-Marx
1bd66b09e3
enhance: allow '.' and '-' characters in usernames (#42417) (#42588)
related: #42417

- update the isValidUsername function to accept dots and hyphens in
addition to letters, digits, and underscores
- this change improves compatibility with common username formats and
addresses feedback in issue #42417

Signed-off-by: Jean-Francois Weber-Marx <jfwm@hotmail.com>
Signed-off-by: Jean-Francois Weber-Marx <jf.webermarx@criteo.com>
2025-07-24 09:54:54 +08:00
foxspy
ed57650b52
fix: remove invalid restrictions on dim for int8 vector (#43469)
issue: #43466

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2025-07-23 20:22:54 +08:00
congqixia
684f027496
fix: Remove trimming space logic when validating collection name (#43064)
Related to #43031

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-07-04 11:00:45 +08:00
congqixia
74ea57bac1
enhance: Remove unused load field check from proxy (#42816)
Related to #42489

Since load list works as hint after cachelayer implemented, the related
check logic could be removed to keep code logic clean.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-06-19 19:34:47 +08:00
junjiejiangjjj
f1a4526bac
enhance: refactor rrf and weighted rerank (#42154)
https://github.com/milvus-io/milvus/issues/35856

Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com>
2025-06-10 18:08:35 +08:00
cqy123456
317bbfbf81
enhance: milvus support minhash vector and mhjaccard metric (#42036)
issue:
https://github.com/issues/assigned?issue=milvus-io%7Cmilvus%7C41746

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2025-06-10 14:38:34 +08:00
aoiasd
2ae4d80120
enhance: support run analyzer by loaded collection field (#42113)
relate: https://github.com/milvus-io/milvus/issues/42094

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-05-29 10:54:30 +08:00
congqixia
4cab236bca
enhance: [AddField][Nullable] Fill absent nullable field server-side (#42095)
Related to #39718

The absent nullable field shall be filled at server-side in nullable
design. While the implementation here was buggy causing the feature was
not able to serve.

This PR make proxy fill the field data in correct format so that field
data with absent column(s) will be accepted.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-05-27 18:50:28 +08:00
Xianhui Lin
6a0e182e13
enhance: support TTL expiration with queries returning no results (#42086)
support TTL expiration with queries returning no results
issue:https://github.com/milvus-io/milvus/issues/41959

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-05-27 18:28:27 +08:00
SimFG
9f866dd7c3
enhance: add privilege group message types and corresponding tests (#41939)
- issue: #41938

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-05-21 11:12:28 +08:00
congqixia
d791cb0979
enhance: [AddField] Support explicit dynamic output fields (#41717)
Related to #39718

After support add field with dynamic fields enabled, the masked dynamic
field shall be able to return with `$meta["name"]`

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-05-09 16:46:54 +08:00
SimFG
dfd2548c1b
enhance: support more ddl apis to replicate according to cdc (#41678)
- issue: #41677

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-05-08 20:02:58 +08:00
SimFG
3bd6268d3c
enhance: Implement OperatePrivilegeV2 message handling and unmarshal support (#41355)
- issue: #41353

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-04-30 12:02:53 +08:00
aoiasd
f52c2909c4
feat: support multi analyzer for bm25 function (#41351)
relate: https://github.com/milvus-io/milvus/issues/41213

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-04-23 18:22:38 +08:00
SimFG
91d40fa558
fix: Update logging context and upgrade dependencies (#41318)
- issue: #41291

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-23 10:52:38 +08:00
Xianhui Lin
f9febe3bae
enhance: Merge RootCoord, DataCoord And QueryCoord into MixCoord (#41006)
Merge RootCoord, DataCoord And QueryCoord into MixCoord
Make Session into one
issue : https://github.com/milvus-io/milvus/issues/37764

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-04-11 16:36:30 +08:00
smellthemoon
cb1e86e17c
enhance: support add field (#39800)
after the pr merged, we can support to insert, upsert, build index,
query, search in the added field.
can only do the above operates in added field after add field request
complete, which is a sync operate.

compact will be supported in the next pr.
#39718

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-04-02 14:24:31 +08:00
aoiasd
1cc88d7755
enhance: add utf8 check for all varchar field (#40670)
https://github.com/milvus-io/milvus/issues/40684

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-03-28 19:38:17 +08:00
Buqian Zheng
7a056aff9d
enhance: avoid re-query if hybrid search requested only pk as output field (#40842)
proxy to always remove pk field from output field when forwarding
request to QN, and if user requested pk, fill it from IDs.

issue: https://github.com/milvus-io/milvus/issues/40833

---------

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2025-03-28 14:32:18 +08:00
Ted Xu
688505ab1c
enhance: cleanup lint check exclusions (#40829)
See: #40828

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2025-03-21 18:12:14 +08:00
SimFG
bf4fc6a8c6
feat: add DDLDB rate type and related quota configurations (#40651)
- issue: #40650

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-03-14 15:38:09 +08:00
junjiejiangjjj
359e7efd8e
feat: Add function running monitoring (#40358)
#35856 
#40004 
1. Optimize model verification logic
2. Add profiling code

Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com>
2025-03-10 22:28:05 +08:00
junjiejiangjjj
b2e630b1a1
feat: Support TEI serving and support int8 embedding (#40199)
#35856

Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com>
2025-03-05 10:18:00 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990)
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
Patrick Weizhi Xu
04fff74a56
feat: introduce Text data type (#39874)
issue: https://github.com/milvus-io/milvus/issues/39818

This PR mimics Varchar data type, allows insert, search, query, delete,
full-text search and others.
Functionalities related to filter expressions are disabled temporarily. 

Storage changes for Text data type will be in the following PRs.

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2025-02-19 11:04:51 +08:00
aoiasd
2b4caba76e
fix: check utf-8 format for varchar with analyzer open (#39299)
relate: https://github.com/milvus-io/milvus/issues/39285

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-02-06 17:11:51 +08:00
junjiejiangjjj
16cbdfb3b1
feat: Add Text Embedding Function (#36366)
https://github.com/milvus-io/milvus/issues/35856

Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com>
2025-01-24 14:23:06 +08:00
smellthemoon
47d280d974
fix: not enable rate limiter for restful v1 (#39553)
issue: #39556

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-01-24 14:05:13 +08:00
congqixia
05b6ea1351
enhance: Refine error msg for schema & index checking (#39533)
The error message was malformated or missing some meta info, say field
name. This PR recitfies some message format and add field name in error
message when type param check fails.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-24 09:57:06 +08:00
Gao
75d7978a18
enhance: pass partition key scalar info if enable for vector mem index (#39123)
issue: #34332

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2025-01-16 14:33:03 +08:00
sthuang
5c5948cb70
fix: rbac custom group privilege level check (#39164)
related: https://github.com/milvus-io/milvus/issues/39086

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-01-13 11:24:59 +08:00
Zhen Ye
bb8d1ab3bf
enhance: make new go package to manage proto (#39114)
issue: #39095

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-10 10:49:01 +08:00
Patrick Weizhi Xu
d3a5282eaa
enhance: add param for tuning max VARCHAR length and restore limit to 65535 (#38884)
issue: #38882 

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 8e740e004151dd2c11918aad2857c1c8d1bd98f5)
2025-01-02 14:42:53 +08:00
Xiaofan
cb6eca8e91
fix: drop partition can not be successful if load failed (#38793)
fix #38649
when partition load failed, the partition drop will also fail due to the
wrong error message

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-12-30 19:42:52 +08:00
Xianhui Lin
90de37e60a
enhance: field stringtype maxlength raise to 1M (#38592)
enhance: field stringtype maxlength raise to 1M
issue: https://github.com/milvus-io/milvus/issues/37436

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-20 14:32:45 +08:00
Bingyi Sun
894c203f27
enhance: allow hyphen in partition name (#38494)
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-18 15:04:45 +08:00
SimFG
2afe2eaf3e
feat: support to replicate collection when the services contains the system tt msg (#37559)
- issue: #37105

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-17 09:08:46 +08:00
tinswzy
27229f7907
enhance: refine exists log print with ctx (#38080)
issue: #35917 
Refines exists log print with ctx

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-14 22:36:44 +08:00
congqixia
304cdc7783
fix: Check target partition load state only when dropping partition (#38375)
Related to #38372

This PR make drop partition only check target partition load states only
in case of concurrent releasing other partition in same collection.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-11 21:06:42 +08:00
Gao
8977454311
enhance: support recall estimation (#38017)
issue: #37899 
Only `search` api will be supported

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-12-11 20:40:48 +08:00
tinswzy
5768dbbb5d
enhance: refine pular related mq interfaces (#38007)
issue: #35917 
Refines the pulsar-related mq APIs to allow the ctx to be passed down

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-04 20:50:39 +08:00
sthuang
a65d395ecd
fix: grant v2 collection name and privilege error typo (#38195)
issue: https://github.com/milvus-io/milvus/issues/38042

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-04 16:14:38 +08:00
smellthemoon
e359725530
enhance: support db request in Restful api (#38140)
#38077

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-03 20:04:41 +08:00
sthuang
a5e0a56a8e
fix: move grant/revoke v2 params check from rootcoord to proxy (#38130)
related issue: https://github.com/milvus-io/milvus/issues/37031

fixed issues #38042: The interface "grant_v2" does not support empty
collectionName while the error says it does

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-02 19:48:37 +08:00
sthuang
23dc313c44
fix: fix grant/revoke v2 meta and unclear error messages (#38110)
related issue: https://github.com/milvus-io/milvus/issues/37031

fixed issues:
#37974: better error messages for grant v2 interface
#37903: fix meta built-in privilege group object name
#37843: better error messages for custom privilege group interface 
#38002: fix built-in privilege group meta to pass proxy interceptor
check
#38008: fix revoke v2 to support revoking v1 granted privileges

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-02 11:36:39 +08:00
cai.zhang
2319018fcb
fix: Check if the dynamic fields contain any static fields (#38025)
issue: #38024

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-01 18:06:37 +08:00
sthuang
19572f5b06
enhance: RBAC new grant/revoke privilege (#37785)
issue: https://github.com/milvus-io/milvus/issues/37031
also fix issues: https://github.com/milvus-io/milvus/issues/37843,
https://github.com/milvus-io/milvus/issues/37842,
https://github.com/milvus-io/milvus/issues/37887

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-21 22:20:34 +08:00
Buqian Zheng
511edd29fd
enhance: disallow get raw vector data of a BM25 Function output field (#37800)
issue: https://github.com/milvus-io/milvus/issues/35853

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-11-20 14:22:30 +08:00