1194 Commits

Author SHA1 Message Date
Xianhui Lin
0574fc7b7b
enhance: support TTL expiration with queries returning no results (#41960)
support TTL expiration with queries returning no results
issue:https://github.com/milvus-io/milvus/issues/41959
pr:https://github.com/milvus-io/milvus/pull/41720

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-05-26 15:52:28 +08:00
Chun Han
81ed143132
enhance: refine expiring compaction(#41336) (#42052)
related: #41336
pr: https://github.com/milvus-io/milvus/pull/42056

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-05-25 16:44:27 +08:00
yihao.dai
83ca664150
fix: [2.5] Fix import slot assignment (#41982)
Assign the import task to the worker with the most available slots, even
if availableSlots < requiredSlots. This ensures tasks won’t be blocked
indefinitely.

issue: https://github.com/milvus-io/milvus/issues/41981

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-05-23 01:36:30 +08:00
SimFG
00336a8c4c
enhance: [2.5] add privilege group message types and corresponding tests (#41940)
- issue: #41938
- pr: #41939

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-05-21 11:12:29 +08:00
Chun Han
043e333290
enhance: support strict expiry compaction for milvus(#41855) (#41856)
related: #41855

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-05-19 09:50:24 +08:00
yihao.dai
7c8370ccd2
fix: [2.5] Fix ants.Pool goroutine leak (#41893)
1. Release the pool after it is no longer in use.
2. Upgrade ants.Pool to fix the goroutine leak issue (see
https://github.com/panjf2000/ants/pull/287).

issue: https://github.com/milvus-io/milvus/issues/41838

pr: https://github.com/milvus-io/milvus/pull/41892

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-05-16 19:12:22 +08:00
aoiasd
daf745ffa3
fix: [2.5] hybird search sub requset not set analyzer name (#41897)
relate: https://github.com/milvus-io/milvus/issues/41213
pr: https://github.com/milvus-io/milvus/pull/41896

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-05-16 17:58:22 +08:00
congqixia
e0c79ffee8
enhance: Bump milvus & proto version to v2.5.12 (#41888)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-05-16 13:06:23 +08:00
congqixia
30dda24982
enhance: [2.5] Bump x/net fixing CVE-2025-22872 (#41878)
Cherry-pick from master
pr: #41861
Related to #41291
Related to CVE-2025-22872

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-05-16 10:20:24 +08:00
cai.zhang
dc1e9e2f81
fix: [2.5] Don't create index for unsorted importing segment when enable stats (#41865)
issue: #41863 

master pr: #41864

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-05-16 10:18:35 +08:00
SimFG
94e2871348
enhance: [2.5] implement authorization checks for DescribeCollection and DescribeDatabase tasks (#41799)
- issue: #41694
- pr: #41798

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-05-15 17:48:23 +08:00
foxspy
e36df6991b
enhance: [2.5] Output index version information in the DescribeIndex interface (#41841)
issue: #41431 
pr: #41847

Signed-off-by: foxspy <xianliang.li@zilliz.com>
2025-05-15 14:34:23 +08:00
wei liu
aed074d83e
fix: unexpected password for root user (#41818)
issue: #41816
pr: #41817
pr #37983 introduced an issue, if doesn't specified
`defaultRootPassword` in milvus.yaml, then `"Milvus"` will be used as
default password for root user, instead of `Milvus`.

This PR fix the unexpected password for root, and add comment for case
which use large numeric password requires double quotes.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-05-14 19:40:22 +08:00
Julien Salleyron
9b8a8f7607
fix: Allow to compile on windows (#41617)
This PR fixes https://github.com/milvus-io/milvus/issues/41384 on 2.5.

Related to #41448.

When using milvus client and compile on windows, the compilation failed
with the undefined RSS error.

On windows, the way to get memory used is the same as on darwin.

Signed-off-by: Julien Salleyron <julien.salleyron@gmail.com>
2025-05-14 11:33:40 +08:00
yihao.dai
a7c818cadb
fix: [2.5] Fix no candidate segments error for small import (#41772)
When autoID is enabled, the preimport task estimates row distribution by
evenly dividing the total row count (numRows) across all vchannels:
`estimatedCount = numRows / vchannelNum`.
However, the actual import task hashes real auto-generated IDs to
determine
the target vchannel. This mismatch can lead to inaccurate row
distribution estimation
in such corner cases:
- Importing 1 row into 2 vchannels:
				• Preimport: 1 / 2 = 0 → both v0 and v1 are estimated to have 0 rows
				• Import: real autoID (e.g., 457975852966809057) hashes to v1
				  → actual result: v0 = 0, v1 = 1

To resolve such corner case, we now allocate at least one segment for
each vchannel
when autoID is enabled, ensuring all vchannels are prepared to receive
data even
if no rows are estimated for them.

issue: https://github.com/milvus-io/milvus/issues/41759

pr: https://github.com/milvus-io/milvus/pull/41771

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-05-14 10:36:22 +08:00
Xianhui Lin
548754a5e3
fix: fallback to mixcoord session when upgrade to mixCoord (#41773)
fix: fallback to mixcoord  session when upgrade to mixCoord
issue:https://github.com/milvus-io/milvus/issues/41737

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-05-13 23:12:58 +08:00
zhagnlu
5b8ea84d38
fix: add params to ignore config type exception (#41777)
pr: #41776

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-05-13 11:28:57 +08:00
groot
bf8e6e8ac0
enhance: Alter collection description (#41547)
issue: https://github.com/milvus-io/milvus/issues/41557
master pr: https://github.com/milvus-io/milvus/pull/41558

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2025-05-12 11:22:56 +08:00
shaoyue
973f6d80a1
enhance: update go version to fix cve (#41697)
master pr: #41696
/cc @congqixia @czs007

Signed-off-by: haorenfsa <haorenfsa@gmail.com>
2025-05-09 11:22:55 +08:00
SimFG
4c656e8a7a
enhance: support more ddl apis to replicate according to cdc (#41679)
- issue: #41677
- pr: #41678

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-05-08 20:02:58 +08:00
SimFG
57c472e41f
enhance: [2.5] add timeout for message reception in mqMsgStream (#41603)
- pr: #41602

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-05-06 10:16:52 +08:00
SimFG
b69bf42a04
enhance: [2.5] Implement OperatePrivilegeV2 message handling and unmarshal support (#41594)
- issue: #41353
- pr: #41355

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-04-30 11:52:53 +08:00
Xianhui Lin
17465ffe9e
feat: [2.5]add str update ts in describecollection (#41600)
issue:https://github.com/milvus-io/milvus/issues/39093
pr:https://github.com/milvus-io/milvus/pull/39096

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-04-29 17:50:53 +08:00
Chun Han
69a80b9ce3
enhance: resize high priority wqthreadpool dynamically(#40838) (#41549)
related: #40838

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-04-28 00:44:39 +08:00
congqixia
41d910451f
enhance: Bump milvus & proto version to v2.5.11 (#41541)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-27 19:56:39 +08:00
Xianhui Lin
21ca05e445
feat: refine drop parition through the new interface notifydroppartition in datacoord (#41543)
refine drop parition through the new interface notifydroppartition in
datacoord
 issue: https://github.com/milvus-io/milvus/issues/41542

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-04-27 17:42:40 +08:00
SimFG
6e18ededab
fix: [2.5] mockery too unavailable after upgrade golang version (#41522)
- issue: ##41291
- pr: #41481

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-04-25 14:40:40 +08:00
SimFG
18eb627533
fix: [2.5] Update logging context and upgrade dependencies (#41319)
- issue: #41291
- pr: #41318

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-24 23:50:40 +08:00
aoiasd
8b3353cdab
feat: [2.5] Support run analyzer and more tokenizer. (#41444)
relate: https://github.com/milvus-io/milvus/issues/39659
https://github.com/milvus-io/milvus/issues/39705
pr: https://github.com/milvus-io/milvus/pull/40416
https://github.com/milvus-io/milvus/pull/40458
https://github.com/milvus-io/milvus/pull/39723
https://github.com/milvus-io/milvus/pull/40813
https://github.com/milvus-io/milvus/pull/39854
https://github.com/milvus-io/milvus/pull/39660

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-04-24 10:48:39 +08:00
foxspy
d5977ec521
enhance: [2.5] add force rebuild index configuration (#41432)
issue: #41431 
pr: #41473

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2025-04-23 21:44:38 +08:00
aoiasd
544493e3e2
feat:[2.5] support multi analyzer for bm25 function (#41456)
relate: https://github.com/milvus-io/milvus/issues/41213
pr: https://github.com/milvus-io/milvus/pull/41351

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-04-23 20:52:39 +08:00
congqixia
f2a5542996
enhance: [2.5] Adapt hyphen in grpc metadata header (#41358) (#41372)
Cherry-pick from master
pr: #41358
Related to #41357

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-17 19:12:39 +08:00
Ted Xu
daa48f6806
fix: errorous deadlock report in unittests (#41350) (#41377)
See #41349 #41291
pr: #41350

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2025-04-17 18:00:37 +08:00
Chun Han
a89b611b2a
fix: solve incompitable problem for none-encoding index (#41346)
related: https://github.com/milvus-io/milvus/issues/40838
pr: https://github.com/milvus-io/milvus/pull/40839

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-04-16 21:04:52 +08:00
congqixia
55b37c3bb6
enhance: Bump milvus & proto version to v2.5.10 (#41328)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-15 19:50:32 +08:00
zhagnlu
2498eb0691
enhance: add optimize for like expr (#41222)
pr: #41066  cherry-pick from master

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-04-14 21:06:32 +08:00
congqixia
b073112e16
enhance: [2.5][Restful] Make default timeout configurable (#41211) (#41225)
Cherry-pick from master
pr: #41211

The restful API default timeout was hard-coded. This PR make this
timeout value configurable via paramtable.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-11 13:12:27 +08:00
congqixia
d75596456a
enhance: [2.5] Rectify client_request_id logic (#41089) (#41149)
Cherry-pick from master
pr: #41089 
The traceID is not initialized by client_request_id in context. If the
client sent valid traceID, milvus log will print two different traceID
which is wierd.

This PR add the logic to tray parsing incoming `client_request_id` into
traceID. If it works just use it the request traceID, otherwise set it
to a different field named `client_request_id`.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-09 10:26:27 +08:00
liliu-z
cb0f984155
enhance: Revert "separate for index completed (#40873)" (#41152)
This reverts commit 23e579e3240a30397f05f5b308be687f6f16b013. #40873

issue: #39519

Signed-off-by: Li Liu <li.liu@zilliz.com>
2025-04-08 17:36:30 +08:00
congqixia
d679195a5a
enhance: Bump milvus & proto version to v2.5.9 (#41144)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-07 20:37:35 +08:00
cai.zhang
0db5e0c4f6
enhance: [2.5]Deprecate disk params about indexing (#41078)
issue: #40863 

master pr: #41045

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-04-07 11:36:34 +08:00
Chun Han
23e579e324
separate for index completed (#40873)
related: https://github.com/milvus-io/milvus/issues/40781

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-04-05 10:20:24 +08:00
wei liu
37a533fe6d
fix: [2.5] Address manual balance and balance check issues (#41038)
issue: #37651
pr: #41037
- Fix context propagation for manual balance segment task creation from
PR #38080.
- Optimize stopping balance by preventing redundant checks per round,
addressing performance regression from PR #40297.
- Decrease default `checkBalanceInterval` from 3000ms to 300ms.
- Correct minor log messages in `BalanceChecker`.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-04-03 01:26:23 +08:00
congqixia
e0f274a1b8
enhance: Bump milvus & proto version to v2.5.8 (#41019)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-03-31 22:27:12 +08:00
yihao.dai
cba8feade3
enhance: [2.5] Add UTF-8 string validation for import (#40746)
issue: https://github.com/milvus-io/milvus/issues/40684

pr: https://github.com/milvus-io/milvus/pull/40694

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-03-31 21:08:21 +08:00
wei liu
d185a8f941
enhance: Balance the collection with the largest row count first (#40958)
issue: #37651
pr: #40297
this PR enable to balance the collection with largest row count first,
to avoid temporary migration of small table data to new nodes during
their onboarding, only to be moved out again after the large table
balance, which would cause unnecessary load.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-03-31 16:14:21 +08:00
aoiasd
d49f0b65d1
fix: rocksmq consumer register not concurrent safe (#39387) (#40885)
relate: https://github.com/milvus-io/milvus/issues/39336
pr: https://github.com/milvus-io/milvus/pull/39387

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-03-28 16:08:18 +08:00
yihao.dai
27ea5d14dc
fix: [2.5] Fix delete data loss due to duplicate binlogID (#40976)
With concurrenct L0 compaction
(https://github.com/milvus-io/milvus/pull/36816), delta logs might be
written to the same L1 segment, causing logID duplication when using the
incremental beginLogID. This PR removes the beginLogID mechanism and
instead passes a log ID range, where the number of IDs in the range
equals the number of compaction segment binlogs multiplied by an
expansion factor.

issue: https://github.com/milvus-io/milvus/issues/40207

pr: https://github.com/milvus-io/milvus/pull/40960

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-03-28 14:34:21 +08:00
wei liu
b64bb63e77
enhance: [2.5] Add trigger interval config for auto balance (#39154) (#39918)
issue: #39156
pr: #39154

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-03-27 16:40:23 +08:00
congqixia
2411c184a8
enhance: [2.5] Support detailed manual compaction criterion (#40892) (#40924)
Cherry-pick from master
pr: #40892
Related to #40866

This PR:
- update go-api/v2 and support partition id/channel/segment level manual
compaction
- refines the compaction trigger implementation
- unify the compaction signal usage

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-03-27 10:36:22 +08:00