admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
21,986 Commits 137 Branches 188 Tags
Commit Graph

1171 Commits

Author SHA1 Message Date
Chun Han
69a80b9ce3
enhance: resize high priority wqthreadpool dynamically(#40838) (#41549) 
related: #40838

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-04-28 00:44:39 +08:00
congqixia
41d910451f
enhance: Bump milvus & proto version to v2.5.11 (#41541) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-04-27 19:56:39 +08:00
Xianhui Lin
21ca05e445
feat: refine drop parition through the new interface notifydroppartition in datacoord (#41543) 
refine drop parition through the new interface notifydroppartition in
datacoord
 issue: https://github.com/milvus-io/milvus/issues/41542

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-04-27 17:42:40 +08:00
SimFG
6e18ededab
fix: [2.5] mockery too unavailable after upgrade golang version (#41522) 
- issue: ##41291
- pr: #41481

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-04-25 14:40:40 +08:00
SimFG
18eb627533
fix: [2.5] Update logging context and upgrade dependencies (#41319) 
- issue: #41291
- pr: #41318

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2025-04-24 23:50:40 +08:00
aoiasd
8b3353cdab
feat: [2.5] Support run analyzer and more tokenizer. (#41444) 
relate: https://github.com/milvus-io/milvus/issues/39659
https://github.com/milvus-io/milvus/issues/39705
pr: https://github.com/milvus-io/milvus/pull/40416
https://github.com/milvus-io/milvus/pull/40458
https://github.com/milvus-io/milvus/pull/39723
https://github.com/milvus-io/milvus/pull/40813
https://github.com/milvus-io/milvus/pull/39854
https://github.com/milvus-io/milvus/pull/39660

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-04-24 10:48:39 +08:00
foxspy
d5977ec521
enhance: [2.5] add force rebuild index configuration (#41432) 
issue: #41431 
pr: #41473

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
 2025-04-23 21:44:38 +08:00
aoiasd
544493e3e2
feat:[2.5] support multi analyzer for bm25 function (#41456) 
relate: https://github.com/milvus-io/milvus/issues/41213
pr: https://github.com/milvus-io/milvus/pull/41351

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-04-23 20:52:39 +08:00
congqixia
f2a5542996
enhance: [2.5] Adapt hyphen in grpc metadata header (#41358) (#41372) 
Cherry-pick from master
pr: #41358
Related to #41357

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-04-17 19:12:39 +08:00
Ted Xu
daa48f6806
fix: errorous deadlock report in unittests (#41350) (#41377) 
See #41349 #41291
pr: #41350

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-04-17 18:00:37 +08:00
Chun Han
a89b611b2a
fix: solve incompitable problem for none-encoding index (#41346) 
related: https://github.com/milvus-io/milvus/issues/40838
pr: https://github.com/milvus-io/milvus/pull/40839

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-04-16 21:04:52 +08:00
congqixia
55b37c3bb6
enhance: Bump milvus & proto version to v2.5.10 (#41328) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-04-15 19:50:32 +08:00
zhagnlu
2498eb0691
enhance: add optimize for like expr (#41222) 
pr: #41066  cherry-pick from master

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-04-14 21:06:32 +08:00
congqixia
b073112e16
enhance: [2.5][Restful] Make default timeout configurable (#41211) (#41225) 
Cherry-pick from master
pr: #41211

The restful API default timeout was hard-coded. This PR make this
timeout value configurable via paramtable.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-04-11 13:12:27 +08:00
congqixia
d75596456a
enhance: [2.5] Rectify client_request_id logic (#41089) (#41149) 
Cherry-pick from master
pr: #41089 
The traceID is not initialized by client_request_id in context. If the
client sent valid traceID, milvus log will print two different traceID
which is wierd.

This PR add the logic to tray parsing incoming `client_request_id` into
traceID. If it works just use it the request traceID, otherwise set it
to a different field named `client_request_id`.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-04-09 10:26:27 +08:00
liliu-z
cb0f984155
enhance: Revert "separate for index completed (#40873)" (#41152) 
This reverts commit 23e579e3240a30397f05f5b308be687f6f16b013. #40873

issue: #39519

Signed-off-by: Li Liu <li.liu@zilliz.com>
 2025-04-08 17:36:30 +08:00
congqixia
d679195a5a
enhance: Bump milvus & proto version to v2.5.9 (#41144) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-04-07 20:37:35 +08:00
cai.zhang
0db5e0c4f6
enhance: [2.5]Deprecate disk params about indexing (#41078) 
issue: #40863 

master pr: #41045

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-04-07 11:36:34 +08:00
Chun Han
23e579e324
separate for index completed (#40873) 
related: https://github.com/milvus-io/milvus/issues/40781

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-04-05 10:20:24 +08:00
wei liu
37a533fe6d
fix: [2.5] Address manual balance and balance check issues (#41038) 
issue: #37651
pr: #41037
- Fix context propagation for manual balance segment task creation from
PR #38080.
- Optimize stopping balance by preventing redundant checks per round,
addressing performance regression from PR #40297.
- Decrease default `checkBalanceInterval` from 3000ms to 300ms.
- Correct minor log messages in `BalanceChecker`.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-04-03 01:26:23 +08:00
congqixia
e0f274a1b8
enhance: Bump milvus & proto version to v2.5.8 (#41019) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-31 22:27:12 +08:00
yihao.dai
cba8feade3
enhance: [2.5] Add UTF-8 string validation for import (#40746) 
issue: https://github.com/milvus-io/milvus/issues/40684

pr: https://github.com/milvus-io/milvus/pull/40694

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-31 21:08:21 +08:00
wei liu
d185a8f941
enhance: Balance the collection with the largest row count first (#40958) 
issue: #37651
pr: #40297
this PR enable to balance the collection with largest row count first,
to avoid temporary migration of small table data to new nodes during
their onboarding, only to be moved out again after the large table
balance, which would cause unnecessary load.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-03-31 16:14:21 +08:00
aoiasd
d49f0b65d1
fix: rocksmq consumer register not concurrent safe (#39387) (#40885) 
relate: https://github.com/milvus-io/milvus/issues/39336
pr: https://github.com/milvus-io/milvus/pull/39387

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-03-28 16:08:18 +08:00
yihao.dai
27ea5d14dc
fix: [2.5] Fix delete data loss due to duplicate binlogID (#40976) 
With concurrenct L0 compaction
(https://github.com/milvus-io/milvus/pull/36816), delta logs might be
written to the same L1 segment, causing logID duplication when using the
incremental beginLogID. This PR removes the beginLogID mechanism and
instead passes a log ID range, where the number of IDs in the range
equals the number of compaction segment binlogs multiplied by an
expansion factor.

issue: https://github.com/milvus-io/milvus/issues/40207

pr: https://github.com/milvus-io/milvus/pull/40960

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-28 14:34:21 +08:00
wei liu
b64bb63e77
enhance: [2.5] Add trigger interval config for auto balance (#39154) (#39918) 
issue: #39156
pr: #39154

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-03-27 16:40:23 +08:00
congqixia
2411c184a8
enhance: [2.5] Support detailed manual compaction criterion (#40892) (#40924) 
Cherry-pick from master
pr: #40892
Related to #40866

This PR:
- update go-api/v2 and support partition id/channel/segment level manual
compaction
- refines the compaction trigger implementation
- unify the compaction signal usage

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-27 10:36:22 +08:00
cai.zhang
e8970e8bf4
fix: [2.5] Update milvus key words (#40870) 
issue: #40290 
master pr: #40869

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-26 19:44:32 +08:00
Zhen Ye
53438c751d
fix: skip consuming from streaming service message (#40879) 
issue: #40532
pr: #40877

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-25 15:22:23 +08:00
cai.zhang
ec0c25aaec
fix: [2.5] As the segment size increases the number of slots should also increase (#40862) 
issue: #40858 
master pr: #39084

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-25 14:42:24 +08:00
yihao.dai
9b7d66d9da
fix: [2.5] Fix channel not balance on datanodes (#40854) 
1. Prevent channels from being assigned to only one datanode during
datacoord startup.
2. Optimize the channel assignment policy by considering newly assigned
channels.

issue: https://github.com/milvus-io/milvus/issues/40421,
https://github.com/milvus-io/milvus/issues/37630

pr: https://github.com/milvus-io/milvus/pull/40422

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-25 10:06:24 +08:00
yihao.dai
b534c9d804
enhance: [2.5] Introduce batch subscription in msgdispatcher (#40596) 
Introduce a batch subscription mechanism in msgdispatcher: the
msgdispatcher now includes a vchannel watch task queue, where all
vchannels in the queue will subscribe to the MQ only once and pull
messages from the oldest vchannel checkpoint to the latest.

issue: https://github.com/milvus-io/milvus/issues/39862

pr: https://github.com/milvus-io/milvus/pull/39863

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-24 10:18:17 +08:00
cai.zhang
d703d8dac8
fix: [2.5] Set correct default configuration for task slots (#40821) 
issue: #39101

master pr: #39084

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-23 21:28:19 +08:00
SimFG
1263505808
feat: [2.5] support to deny dll according to database property (#40784) 
- issue: #40762
- pr: #40764

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-03-23 11:18:28 +08:00
cqy123456
7c716d61dc
fix:[2.5]wrong p.InterimIndexRefineQuantType default value and reduce ut run time (#40830) 
issue: https://github.com/milvus-io/milvus/issues/27678

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2025-03-21 19:10:13 +08:00
Buqian Zheng
cff0e82f57
enhance: [2.5] improve sparse query nnz metric (#40714) 
add query type and field id label; add metric for hybrid search

issue: https://github.com/milvus-io/milvus/issues/35853
pr: https://github.com/milvus-io/milvus/pull/40713

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2025-03-20 14:14:21 +08:00
XuanYang-cn
1a6761ac69
enhance: [cp25]Replace currRows with NumOfRows (#40074) (#40681) 
See also: #40068
pr: #40074

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-03-20 11:24:13 +08:00
congqixia
cd8d7efc96
enhance: [2.5] Add an extra parameter for slow log (#40759) 
Cherry-pick from master
pr: #40758
Related to #40756

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-19 18:10:13 +08:00
cai.zhang
0cfe835cac
enhance: [2.5] Support slot for index task and stats task (#40288) 
issue: #39101 

master pr: #39084

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-19 16:50:11 +08:00
congqixia
e49294cd16
enhance: [2.5] Add buffer forwarder for stream delta loading (#40559) (#40699) 
Cherry-pick from master
pr: #40559
See also #40558
Related to #35303 & #38066 as well

This PR:
- Add `BufferedForward` to limit memory usage forwarding stream delete
- Add `UseLoad` flag to determine `Delete` shall use `segment.Delete` or
`segment.LoadDelta`
- Fix delegator accidentally use always true candidate while load
streaming delta

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-19 10:28:14 +08:00
yihao.dai
aca9dd9105
fix: [2.5] Fix messages loss during seek (#40736) 
issue: https://github.com/milvus-io/milvus/issues/40733

pr: https://github.com/milvus-io/milvus/pull/40735

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-18 21:56:33 +08:00
congqixia
6d25fc56aa
enhance: Bump milvus & proto version to v2.5.7 (#40728) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-18 19:58:15 +08:00
congqixia
8de5174e4d
fix: [2.5] Remove lag targets after main dispatcher (#40717) 
Related to #40716

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-18 19:10:34 +08:00
Xianhui Lin
705b3c90a5
fix: Failed to rolling upgrade from v2.5.6 to new 2.5 version when enable JsonKeyStats (#40661) 
fix: Failed to rolling upgrade from v2.5.6 to new 2.5 version when
enable JsonKeyStats.The reason is that the file path of the jsonkeyindex
has changed.
issue: https://github.com/milvus-io/milvus/issues/40649
，https://github.com/milvus-io/milvus/issues/40669
https://github.com/milvus-io/milvus/issues/40707
master-pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-03-18 17:32:16 +08:00
Zhen Ye
167d058682
enhance: remove redundant resource key watch operation, just keep consistency of wal (#40503) 
issue: #38399
pr: #40235

related PR: #39522

- Just implement exclusive broadcaster between broadcast message with
same resource key to keep same order in different wal.
- After simplify the broadcast model, original watch-based broadcast is
too complicated and redundant, remove it.
- Add metrics for broadcast.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-18 15:46:25 +08:00
cqy123456
b8abd91962
enhance[2.5]: add more config for interimIndex to support more refine mode (#40429) 
issue: https://github.com/milvus-io/milvus/issues/27678

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2025-03-17 22:04:12 +08:00
XuanYang-cn
f455923ac9
enhance: Use correct counter metrics for overall wa calculation (#40394) (#40679) 
pr: #40394

- Use CounterVec to calculate sum of increase during a time period.
- Use entries number instead of binlog size

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-03-17 15:06:19 +08:00
Gao
09f7a60fe1
enhance: [2.5] make segment prune config refreshable (#40632) 
pr: #40539

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2025-03-14 16:12:09 +08:00
SimFG
a3c60a03b5
feat: [2.5] add DDLDB rate type and related quota configurations (#40653) 
- issue: #40650
- pr: #40651

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-03-14 15:36:17 +08:00
zhagnlu
6b9e141ada
enhance: reorder sub expr for conjunct expr (#40186) 
pr:#39872

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-03-14 15:16:08 +08:00