admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00
Code Issues Packages Projects Releases Wiki Activity
22,097 Commits 146 Branches 196 Tags
Commit Graph

823 Commits

Author SHA1 Message Date
aoiasd
5110130b2e
enhance: add segment bm25 stats local cache (#41775) (#42646) 
relate: https://github.com/milvus-io/milvus/issues/41424
pr: https://github.com/milvus-io/milvus/pull/41775

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-06-13 16:50:37 +08:00
Bingyi Sun
a32b55ed71
enhance: support auto index type for json index (#42161) 
issue: #42070 
pr: #42071

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-06-09 21:20:34 +08:00
yihao.dai
28aa364bf7
enhance: [2.5] Adjust default import buffer size (#42542) 
Increase insert buffer size from 16MB to 64MB, while keeping delete
buffer size at 16MB.

issue: https://github.com/milvus-io/milvus/issues/42518

pr: https://github.com/milvus-io/milvus/pull/42541

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-06-05 18:46:33 +08:00
yihao.dai
72a8777c9d
enhance: [2.5] Accelerate dispatcher building (#42544) 
Reduce check interval to accelerate dispatcher building.

issue: https://github.com/milvus-io/milvus/issues/42067

pr: https://github.com/milvus-io/milvus/pull/42500

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-06-05 18:16:33 +08:00
yihao.dai
fdfb78b9e5
fix: [2.5] Fix duplicate autoID between import and insert (#42520) 
Remove the unlimited logID mechanism and switch to redundantly
allocating a large number of IDs.

issue: https://github.com/milvus-io/milvus/issues/42518

pr: https://github.com/milvus-io/milvus/pull/42519

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-06-05 00:54:33 +08:00
liliu-z
1cab5dc2b2
enhance: Make cagra gpu image default (#42193) 
pr: #41906
issue: #41907

Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
Signed-off-by: Li Liu <li.liu@zilliz.com>
Co-authored-by: presburger <yusheng.ma@zilliz.com>
 2025-05-30 03:12:30 +08:00
wei liu
4a05180f88
enhance: [2.5] support balancing multiple collections in single trigger (#41875) (#42134) 
issue: #41874
pr: #41875
- Optimize balance_checker to support balancing multiple collections
simultaneously
- Add new parameters for segment and channel balancing batch sizes
- Add enableBalanceOnMultipleCollections parameter
- Update tests for balance checker

This change improves resource utilization by allowing the system to
balance multiple collections in a single trigger with configurable batch
sizes.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-05-28 23:18:30 +08:00
congqixia
6c17cdffd8
enhance: [2.5] Take nq into slow query consideration (#42109) (#42125) 
Cherry-pick from master
pr: #42109
Related to #40756

Large nq will naturally increase query time, which causing lots of slow
log when user NQ numbers are very large.

This PR make slow search counts span per nq (using avg val) to decide
whether one request is slow or not.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-05-28 19:02:30 +08:00
Chun Han
81ed143132
enhance: refine expiring compaction(#41336) (#42052) 
related: #41336
pr: https://github.com/milvus-io/milvus/pull/42056

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-05-25 16:44:27 +08:00
yihao.dai
83ca664150
fix: [2.5] Fix import slot assignment (#41982) 
Assign the import task to the worker with the most available slots, even
if availableSlots < requiredSlots. This ensures tasks won’t be blocked
indefinitely.

issue: https://github.com/milvus-io/milvus/issues/41981

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-05-23 01:36:30 +08:00
Chun Han
043e333290
enhance: support strict expiry compaction for milvus(#41855) (#41856) 
related: #41855

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-05-19 09:50:24 +08:00
yihao.dai
7c8370ccd2
fix: [2.5] Fix ants.Pool goroutine leak (#41893) 
1. Release the pool after it is no longer in use.
2. Upgrade ants.Pool to fix the goroutine leak issue (see
https://github.com/panjf2000/ants/pull/287).

issue: https://github.com/milvus-io/milvus/issues/41838

pr: https://github.com/milvus-io/milvus/pull/41892

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-05-16 19:12:22 +08:00
cai.zhang
dc1e9e2f81
fix: [2.5] Don't create index for unsorted importing segment when enable stats (#41865) 
issue: #41863 

master pr: #41864

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-05-16 10:18:35 +08:00
wei liu
aed074d83e
fix: unexpected password for root user (#41818) 
issue: #41816
pr: #41817
pr #37983 introduced an issue, if doesn't specified
`defaultRootPassword` in milvus.yaml, then `"Milvus"` will be used as
default password for root user, instead of `Milvus`.

This PR fix the unexpected password for root, and add comment for case
which use large numeric password requires double quotes.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-05-14 19:40:22 +08:00
Julien Salleyron
9b8a8f7607
fix: Allow to compile on windows (#41617) 
This PR fixes https://github.com/milvus-io/milvus/issues/41384 on 2.5.

Related to #41448.

When using milvus client and compile on windows, the compilation failed
with the undefined RSS error.

On windows, the way to get memory used is the same as on darwin.

Signed-off-by: Julien Salleyron <julien.salleyron@gmail.com>
 2025-05-14 11:33:40 +08:00
Xianhui Lin
548754a5e3
fix: fallback to mixcoord session when upgrade to mixCoord (#41773) 
fix: fallback to mixcoord  session when upgrade to mixCoord
issue:https://github.com/milvus-io/milvus/issues/41737

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-05-13 23:12:58 +08:00
zhagnlu
5b8ea84d38
fix: add params to ignore config type exception (#41777) 
pr: #41776

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-05-13 11:28:57 +08:00
Chun Han
69a80b9ce3
enhance: resize high priority wqthreadpool dynamically(#40838) (#41549) 
related: #40838

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-04-28 00:44:39 +08:00
SimFG
18eb627533
fix: [2.5] Update logging context and upgrade dependencies (#41319) 
- issue: #41291
- pr: #41318

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2025-04-24 23:50:40 +08:00
foxspy
d5977ec521
enhance: [2.5] add force rebuild index configuration (#41432) 
issue: #41431 
pr: #41473

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
 2025-04-23 21:44:38 +08:00
aoiasd
544493e3e2
feat:[2.5] support multi analyzer for bm25 function (#41456) 
relate: https://github.com/milvus-io/milvus/issues/41213
pr: https://github.com/milvus-io/milvus/pull/41351

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-04-23 20:52:39 +08:00
congqixia
f2a5542996
enhance: [2.5] Adapt hyphen in grpc metadata header (#41358) (#41372) 
Cherry-pick from master
pr: #41358
Related to #41357

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-04-17 19:12:39 +08:00
Ted Xu
daa48f6806
fix: errorous deadlock report in unittests (#41350) (#41377) 
See #41349 #41291
pr: #41350

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-04-17 18:00:37 +08:00
congqixia
b073112e16
enhance: [2.5][Restful] Make default timeout configurable (#41211) (#41225) 
Cherry-pick from master
pr: #41211

The restful API default timeout was hard-coded. This PR make this
timeout value configurable via paramtable.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-04-11 13:12:27 +08:00
congqixia
d75596456a
enhance: [2.5] Rectify client_request_id logic (#41089) (#41149) 
Cherry-pick from master
pr: #41089 
The traceID is not initialized by client_request_id in context. If the
client sent valid traceID, milvus log will print two different traceID
which is wierd.

This PR add the logic to tray parsing incoming `client_request_id` into
traceID. If it works just use it the request traceID, otherwise set it
to a different field named `client_request_id`.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-04-09 10:26:27 +08:00
cai.zhang
0db5e0c4f6
enhance: [2.5]Deprecate disk params about indexing (#41078) 
issue: #40863 

master pr: #41045

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-04-07 11:36:34 +08:00
wei liu
37a533fe6d
fix: [2.5] Address manual balance and balance check issues (#41038) 
issue: #37651
pr: #41037
- Fix context propagation for manual balance segment task creation from
PR #38080.
- Optimize stopping balance by preventing redundant checks per round,
addressing performance regression from PR #40297.
- Decrease default `checkBalanceInterval` from 3000ms to 300ms.
- Correct minor log messages in `BalanceChecker`.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-04-03 01:26:23 +08:00
yihao.dai
cba8feade3
enhance: [2.5] Add UTF-8 string validation for import (#40746) 
issue: https://github.com/milvus-io/milvus/issues/40684

pr: https://github.com/milvus-io/milvus/pull/40694

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-31 21:08:21 +08:00
wei liu
d185a8f941
enhance: Balance the collection with the largest row count first (#40958) 
issue: #37651
pr: #40297
this PR enable to balance the collection with largest row count first,
to avoid temporary migration of small table data to new nodes during
their onboarding, only to be moved out again after the large table
balance, which would cause unnecessary load.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-03-31 16:14:21 +08:00
yihao.dai
27ea5d14dc
fix: [2.5] Fix delete data loss due to duplicate binlogID (#40976) 
With concurrenct L0 compaction
(https://github.com/milvus-io/milvus/pull/36816), delta logs might be
written to the same L1 segment, causing logID duplication when using the
incremental beginLogID. This PR removes the beginLogID mechanism and
instead passes a log ID range, where the number of IDs in the range
equals the number of compaction segment binlogs multiplied by an
expansion factor.

issue: https://github.com/milvus-io/milvus/issues/40207

pr: https://github.com/milvus-io/milvus/pull/40960

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-28 14:34:21 +08:00
wei liu
b64bb63e77
enhance: [2.5] Add trigger interval config for auto balance (#39154) (#39918) 
issue: #39156
pr: #39154

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-03-27 16:40:23 +08:00
cai.zhang
ec0c25aaec
fix: [2.5] As the segment size increases the number of slots should also increase (#40862) 
issue: #40858 
master pr: #39084

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-25 14:42:24 +08:00
yihao.dai
b534c9d804
enhance: [2.5] Introduce batch subscription in msgdispatcher (#40596) 
Introduce a batch subscription mechanism in msgdispatcher: the
msgdispatcher now includes a vchannel watch task queue, where all
vchannels in the queue will subscribe to the MQ only once and pull
messages from the oldest vchannel checkpoint to the latest.

issue: https://github.com/milvus-io/milvus/issues/39862

pr: https://github.com/milvus-io/milvus/pull/39863

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-24 10:18:17 +08:00
cai.zhang
d703d8dac8
fix: [2.5] Set correct default configuration for task slots (#40821) 
issue: #39101

master pr: #39084

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-23 21:28:19 +08:00
cqy123456
7c716d61dc
fix:[2.5]wrong p.InterimIndexRefineQuantType default value and reduce ut run time (#40830) 
issue: https://github.com/milvus-io/milvus/issues/27678

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2025-03-21 19:10:13 +08:00
Buqian Zheng
cff0e82f57
enhance: [2.5] improve sparse query nnz metric (#40714) 
add query type and field id label; add metric for hybrid search

issue: https://github.com/milvus-io/milvus/issues/35853
pr: https://github.com/milvus-io/milvus/pull/40713

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2025-03-20 14:14:21 +08:00
congqixia
cd8d7efc96
enhance: [2.5] Add an extra parameter for slow log (#40759) 
Cherry-pick from master
pr: #40758
Related to #40756

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-19 18:10:13 +08:00
cai.zhang
0cfe835cac
enhance: [2.5] Support slot for index task and stats task (#40288) 
issue: #39101 

master pr: #39084

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-19 16:50:11 +08:00
congqixia
e49294cd16
enhance: [2.5] Add buffer forwarder for stream delta loading (#40559) (#40699) 
Cherry-pick from master
pr: #40559
See also #40558
Related to #35303 & #38066 as well

This PR:
- Add `BufferedForward` to limit memory usage forwarding stream delete
- Add `UseLoad` flag to determine `Delete` shall use `segment.Delete` or
`segment.LoadDelta`
- Fix delegator accidentally use always true candidate while load
streaming delta

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-19 10:28:14 +08:00
Xianhui Lin
705b3c90a5
fix: Failed to rolling upgrade from v2.5.6 to new 2.5 version when enable JsonKeyStats (#40661) 
fix: Failed to rolling upgrade from v2.5.6 to new 2.5 version when
enable JsonKeyStats.The reason is that the file path of the jsonkeyindex
has changed.
issue: https://github.com/milvus-io/milvus/issues/40649
，https://github.com/milvus-io/milvus/issues/40669
https://github.com/milvus-io/milvus/issues/40707
master-pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-03-18 17:32:16 +08:00
cqy123456
b8abd91962
enhance[2.5]: add more config for interimIndex to support more refine mode (#40429) 
issue: https://github.com/milvus-io/milvus/issues/27678

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2025-03-17 22:04:12 +08:00
Gao
09f7a60fe1
enhance: [2.5] make segment prune config refreshable (#40632) 
pr: #40539

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2025-03-14 16:12:09 +08:00
SimFG
a3c60a03b5
feat: [2.5] add DDLDB rate type and related quota configurations (#40653) 
- issue: #40650
- pr: #40651

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-03-14 15:36:17 +08:00
zhagnlu
6b9e141ada
enhance: reorder sub expr for conjunct expr (#40186) 
pr:#39872

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-03-14 15:16:08 +08:00
congqixia
d8a2c1a907
enhance: [2.5] Add channel seal policy based on blocking l0 (#40505) (#40535) 
Cherry-pick from 2.5
pr: #40505
Related to #40502

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-14 10:02:12 +08:00
Xianhui Lin
f5e9dea2aa
fix: [2.5]fix the garbage cleanup logic of jsonkey stats && improve json key stats filer (#40039) 
fix: fix the garbage collection cleanup logic of jsonkey stats &&
improve json key stats filer
issue: https://github.com/milvus-io/milvus/issues/36995
https://github.com/milvus-io/milvus/issues/40034
https://github.com/milvus-io/milvus/issues/40041
https://github.com/milvus-io/milvus/issues/40106
https://github.com/milvus-io/milvus/issues/40138
pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-03-13 20:18:10 +08:00
yihao.dai
7cbd40c7a1
fix: [2.5] Fix parsing import endTs (#40333) 
Parsing import beginTs, endTs as a hybrid timestamp.

issue: https://github.com/milvus-io/milvus/issues/40326

pr: https://github.com/milvus-io/milvus/pull/40332

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-05 11:40:13 +08:00
cai.zhang
9cbab98382
enhance: [2.5] Export request timeout interval in config (#40118) 2025-02-23 13:47:12 +08:00
congqixia
709594f158
enhance: [2.5] Use v2 package name for pkg module (#40117) 
Cherry-pick from master
pr: #39990
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-23 00:46:01 +08:00
Xianhui Lin
c1de61ff7c
fix: [2.5]Replace the position of EnabledJSONKeyStats (#40108) 
Replace the position of EnabledJSONKeyStats
issue: https://github.com/milvus-io/milvus/issues/36995
pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-02-22 14:35:54 +08:00