33 Commits

Author SHA1 Message Date
Xianhui Lin
a1927e22a5
fix: add ShowLoadCollections and ShowLoadPartitions for compatibale mixcoord (#42514)
fix: add ShowLoadCollections and ShowLoadPartitions for compatibale
mixcoord
issue:https://github.com/milvus-io/milvus/issues/42492

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-06-05 15:46:33 +08:00
aoiasd
198ff1f150
enhance: [2.5] support run analyzer by loaded collection field (#42119)
relate: https://github.com/milvus-io/milvus/issues/42094
pr: https://github.com/milvus-io/milvus/pull/42113

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-05-29 10:26:30 +08:00
Xianhui Lin
0490344442
fix: support TTL expiration with queries returning no results (#42103)
support TTL expiration with queries returning no results
issue:https://github.com/milvus-io/milvus/issues/41959
pr:https://github.com/milvus-io/milvus/pull/42086

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-05-27 15:18:28 +08:00
Xianhui Lin
0574fc7b7b
enhance: support TTL expiration with queries returning no results (#41960)
support TTL expiration with queries returning no results
issue:https://github.com/milvus-io/milvus/issues/41959
pr:https://github.com/milvus-io/milvus/pull/41720

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-05-26 15:52:28 +08:00
aoiasd
daf745ffa3
fix: [2.5] hybird search sub requset not set analyzer name (#41897)
relate: https://github.com/milvus-io/milvus/issues/41213
pr: https://github.com/milvus-io/milvus/pull/41896

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-05-16 17:58:22 +08:00
foxspy
e36df6991b
enhance: [2.5] Output index version information in the DescribeIndex interface (#41841)
issue: #41431 
pr: #41847

Signed-off-by: foxspy <xianliang.li@zilliz.com>
2025-05-15 14:34:23 +08:00
Xianhui Lin
21ca05e445
feat: refine drop parition through the new interface notifydroppartition in datacoord (#41543)
refine drop parition through the new interface notifydroppartition in
datacoord
 issue: https://github.com/milvus-io/milvus/issues/41542

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-04-27 17:42:40 +08:00
aoiasd
544493e3e2
feat:[2.5] support multi analyzer for bm25 function (#41456)
relate: https://github.com/milvus-io/milvus/issues/41213
pr: https://github.com/milvus-io/milvus/pull/41351

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-04-23 20:52:39 +08:00
zhagnlu
2498eb0691
enhance: add optimize for like expr (#41222)
pr: #41066  cherry-pick from master

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-04-14 21:06:32 +08:00
liliu-z
cb0f984155
enhance: Revert "separate for index completed (#40873)" (#41152)
This reverts commit 23e579e3240a30397f05f5b308be687f6f16b013. #40873

issue: #39519

Signed-off-by: Li Liu <li.liu@zilliz.com>
2025-04-08 17:36:30 +08:00
cai.zhang
0db5e0c4f6
enhance: [2.5]Deprecate disk params about indexing (#41078)
issue: #40863 

master pr: #41045

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-04-07 11:36:34 +08:00
Chun Han
23e579e324
separate for index completed (#40873)
related: https://github.com/milvus-io/milvus/issues/40781

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-04-05 10:20:24 +08:00
yihao.dai
27ea5d14dc
fix: [2.5] Fix delete data loss due to duplicate binlogID (#40976)
With concurrenct L0 compaction
(https://github.com/milvus-io/milvus/pull/36816), delta logs might be
written to the same L1 segment, causing logID duplication when using the
incremental beginLogID. This PR removes the beginLogID mechanism and
instead passes a log ID range, where the number of IDs in the range
equals the number of compaction segment binlogs multiplied by an
expansion factor.

issue: https://github.com/milvus-io/milvus/issues/40207

pr: https://github.com/milvus-io/milvus/pull/40960

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-03-28 14:34:21 +08:00
XuanYang-cn
1a6761ac69
enhance: [cp25]Replace currRows with NumOfRows (#40074) (#40681)
See also: #40068
pr: #40074

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-03-20 11:24:13 +08:00
cai.zhang
0cfe835cac
enhance: [2.5] Support slot for index task and stats task (#40288)
issue: #39101 

master pr: #39084

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-03-19 16:50:11 +08:00
congqixia
e49294cd16
enhance: [2.5] Add buffer forwarder for stream delta loading (#40559) (#40699)
Cherry-pick from master
pr: #40559
See also #40558
Related to #35303 & #38066 as well

This PR:
- Add `BufferedForward` to limit memory usage forwarding stream delete
- Add `UseLoad` flag to determine `Delete` shall use `segment.Delete` or
`segment.LoadDelta`
- Fix delegator accidentally use always true candidate while load
streaming delta

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-03-19 10:28:14 +08:00
Xianhui Lin
705b3c90a5
fix: Failed to rolling upgrade from v2.5.6 to new 2.5 version when enable JsonKeyStats (#40661)
fix: Failed to rolling upgrade from v2.5.6 to new 2.5 version when
enable JsonKeyStats.The reason is that the file path of the jsonkeyindex
has changed.
issue: https://github.com/milvus-io/milvus/issues/40649https://github.com/milvus-io/milvus/issues/40669
https://github.com/milvus-io/milvus/issues/40707
master-pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-03-18 17:32:16 +08:00
Zhen Ye
167d058682
enhance: remove redundant resource key watch operation, just keep consistency of wal (#40503)
issue: #38399
pr: #40235

related PR: #39522

- Just implement exclusive broadcaster between broadcast message with
same resource key to keep same order in different wal.
- After simplify the broadcast model, original watch-based broadcast is
too complicated and redundant, remove it.
- Add metrics for broadcast.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-03-18 15:46:25 +08:00
SimFG
a3c60a03b5
feat: [2.5] add DDLDB rate type and related quota configurations (#40653)
- issue: #40650
- pr: #40651

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-03-14 15:36:17 +08:00
cai.zhang
bdc0e68aaf
feat: [2.5] Support get segment binlogs info with new interface GetSegmentsInfo (#40466)
issue: #40341 

master pr: #40464

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-03-14 00:14:08 +08:00
Xianhui Lin
f5e9dea2aa
fix: [2.5]fix the garbage cleanup logic of jsonkey stats && improve json key stats filer (#40039)
fix: fix the garbage collection cleanup logic of jsonkey stats &&
improve json key stats filer
issue: https://github.com/milvus-io/milvus/issues/36995
https://github.com/milvus-io/milvus/issues/40034
https://github.com/milvus-io/milvus/issues/40041
https://github.com/milvus-io/milvus/issues/40106
https://github.com/milvus-io/milvus/issues/40138
pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-03-13 20:18:10 +08:00
Xianhui Lin
a4eb2ce224
fix: [2.5]Revert qc statschecker for json key stats (#40125)
Revert qc statschecker for json key stats
issue:https://github.com/milvus-io/milvus/issues/36995
pr:https://github.com/milvus-io/milvus/pull/39876

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-02-24 13:31:55 +08:00
congqixia
709594f158
enhance: [2.5] Use v2 package name for pkg module (#40117)
Cherry-pick from master
pr: #39990
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-23 00:46:01 +08:00
wei liu
3c2d8c1419
enhance: [2.5] Add management api to check querycoord balance status (#37784) (#39909)
issue: #37783
pr: #37784

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-19 10:56:49 +08:00
Zhen Ye
56c1a8d462
fix: ignore growing option is lost at hibridsearch (#39900)
issue: #39892
pr: #39799

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-17 15:30:15 +08:00
Xianhui Lin
f0964f769d
enhance: [2.5]Add json key inverted index in stats for optimization (#39876)
Add json key inverted index in stats for optimization
issue: https://github.com/milvus-io/milvus/issues/36995
pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-16 20:12:15 +08:00
Zhen Ye
858dc10ef9
enhance: broadcast with event-based notification (#39550)
issue: #38399
pr: #39522

- broadcast message can carry multi resource key now.
- implement event-based notification for broadcast messages
- broadcast message use broadcast id as a unique identifier in message
- broadcasted message on vchannels keep the broadcasted vchannel now.
- broadcasted message and broadcast message have a common broadcast
header now.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-07 11:50:50 +08:00
Xianhui Lin
3b3959c625
enhance: [2.5]Add schema update time verification for insert and upsert to use cache (#39382)
enhance: Add schema update time verification for insert and upsert to
use cache
issue: https://github.com/milvus-io/milvus/issues/39093
Related to pr: https://github.com/milvus-io/milvus/pull/39096

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-01-21 10:19:11 +08:00
yihao.dai
6773fb10a8
enhance: [2.5] Read metadata concurrently to accelerate recovery (#38900)
Read metadata such as segments, binlogs, and partitions concurrently at
the collection level.

issue: https://github.com/milvus-io/milvus/issues/37630

pr: https://github.com/milvus-io/milvus/pull/38403

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-16 17:53:01 +08:00
smellthemoon
1602390734
enhance: support null expr(#38772) (#38931)
pr: #38772 
issue: #31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-01-16 10:55:07 +08:00
cai.zhang
4270174899
fix: [2.5] Add scalar index engine version for compatibility (#39236)
issue: #39203 

master pr: #39204

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-01-14 21:01:01 +08:00
Zhen Ye
adfc3f945e
enhance: record memory size (uncompressed) item for index (#38844)
issue: #38715 
pr: #38770

- Current milvus use a serialized index size(compressed) for estimate
resource for loading.
- Add a new field MemSize (before compressing) for index to estimate
resource.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-14 10:33:06 +08:00
Zhen Ye
95809ca767
enhance: make new go package to manage proto (#39128)
issue: #39095
pr: #39114

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-10 10:53:01 +08:00