admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00
Code Issues Packages Projects Releases Wiki Activity
21,956 Commits 146 Branches 196 Tags
Commit Graph

1161 Commits

Author SHA1 Message Date
Chun Han
a89b611b2a
fix: solve incompitable problem for none-encoding index (#41346) 
related: https://github.com/milvus-io/milvus/issues/40838
pr: https://github.com/milvus-io/milvus/pull/40839

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-04-16 21:04:52 +08:00
congqixia
55b37c3bb6
enhance: Bump milvus & proto version to v2.5.10 (#41328) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-04-15 19:50:32 +08:00
zhagnlu
2498eb0691
enhance: add optimize for like expr (#41222) 
pr: #41066  cherry-pick from master

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-04-14 21:06:32 +08:00
congqixia
b073112e16
enhance: [2.5][Restful] Make default timeout configurable (#41211) (#41225) 
Cherry-pick from master
pr: #41211

The restful API default timeout was hard-coded. This PR make this
timeout value configurable via paramtable.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-04-11 13:12:27 +08:00
congqixia
d75596456a
enhance: [2.5] Rectify client_request_id logic (#41089) (#41149) 
Cherry-pick from master
pr: #41089 
The traceID is not initialized by client_request_id in context. If the
client sent valid traceID, milvus log will print two different traceID
which is wierd.

This PR add the logic to tray parsing incoming `client_request_id` into
traceID. If it works just use it the request traceID, otherwise set it
to a different field named `client_request_id`.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-04-09 10:26:27 +08:00
liliu-z
cb0f984155
enhance: Revert "separate for index completed (#40873)" (#41152) 
This reverts commit 23e579e3240a30397f05f5b308be687f6f16b013. #40873

issue: #39519

Signed-off-by: Li Liu <li.liu@zilliz.com>
 2025-04-08 17:36:30 +08:00
congqixia
d679195a5a
enhance: Bump milvus & proto version to v2.5.9 (#41144) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-04-07 20:37:35 +08:00
cai.zhang
0db5e0c4f6
enhance: [2.5]Deprecate disk params about indexing (#41078) 
issue: #40863 

master pr: #41045

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-04-07 11:36:34 +08:00
Chun Han
23e579e324
separate for index completed (#40873) 
related: https://github.com/milvus-io/milvus/issues/40781

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-04-05 10:20:24 +08:00
wei liu
37a533fe6d
fix: [2.5] Address manual balance and balance check issues (#41038) 
issue: #37651
pr: #41037
- Fix context propagation for manual balance segment task creation from
PR #38080.
- Optimize stopping balance by preventing redundant checks per round,
addressing performance regression from PR #40297.
- Decrease default `checkBalanceInterval` from 3000ms to 300ms.
- Correct minor log messages in `BalanceChecker`.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-04-03 01:26:23 +08:00
congqixia
e0f274a1b8
enhance: Bump milvus & proto version to v2.5.8 (#41019) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-31 22:27:12 +08:00
yihao.dai
cba8feade3
enhance: [2.5] Add UTF-8 string validation for import (#40746) 
issue: https://github.com/milvus-io/milvus/issues/40684

pr: https://github.com/milvus-io/milvus/pull/40694

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-31 21:08:21 +08:00
wei liu
d185a8f941
enhance: Balance the collection with the largest row count first (#40958) 
issue: #37651
pr: #40297
this PR enable to balance the collection with largest row count first,
to avoid temporary migration of small table data to new nodes during
their onboarding, only to be moved out again after the large table
balance, which would cause unnecessary load.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-03-31 16:14:21 +08:00
aoiasd
d49f0b65d1
fix: rocksmq consumer register not concurrent safe (#39387) (#40885) 
relate: https://github.com/milvus-io/milvus/issues/39336
pr: https://github.com/milvus-io/milvus/pull/39387

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-03-28 16:08:18 +08:00
yihao.dai
27ea5d14dc
fix: [2.5] Fix delete data loss due to duplicate binlogID (#40976) 
With concurrenct L0 compaction
(https://github.com/milvus-io/milvus/pull/36816), delta logs might be
written to the same L1 segment, causing logID duplication when using the
incremental beginLogID. This PR removes the beginLogID mechanism and
instead passes a log ID range, where the number of IDs in the range
equals the number of compaction segment binlogs multiplied by an
expansion factor.

issue: https://github.com/milvus-io/milvus/issues/40207

pr: https://github.com/milvus-io/milvus/pull/40960

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-28 14:34:21 +08:00
wei liu
b64bb63e77
enhance: [2.5] Add trigger interval config for auto balance (#39154) (#39918) 
issue: #39156
pr: #39154

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-03-27 16:40:23 +08:00
congqixia
2411c184a8
enhance: [2.5] Support detailed manual compaction criterion (#40892) (#40924) 
Cherry-pick from master
pr: #40892
Related to #40866

This PR:
- update go-api/v2 and support partition id/channel/segment level manual
compaction
- refines the compaction trigger implementation
- unify the compaction signal usage

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-27 10:36:22 +08:00
cai.zhang
e8970e8bf4
fix: [2.5] Update milvus key words (#40870) 
issue: #40290 
master pr: #40869

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-26 19:44:32 +08:00
Zhen Ye
53438c751d
fix: skip consuming from streaming service message (#40879) 
issue: #40532
pr: #40877

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-25 15:22:23 +08:00
cai.zhang
ec0c25aaec
fix: [2.5] As the segment size increases the number of slots should also increase (#40862) 
issue: #40858 
master pr: #39084

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-25 14:42:24 +08:00
yihao.dai
9b7d66d9da
fix: [2.5] Fix channel not balance on datanodes (#40854) 
1. Prevent channels from being assigned to only one datanode during
datacoord startup.
2. Optimize the channel assignment policy by considering newly assigned
channels.

issue: https://github.com/milvus-io/milvus/issues/40421,
https://github.com/milvus-io/milvus/issues/37630

pr: https://github.com/milvus-io/milvus/pull/40422

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-25 10:06:24 +08:00
yihao.dai
b534c9d804
enhance: [2.5] Introduce batch subscription in msgdispatcher (#40596) 
Introduce a batch subscription mechanism in msgdispatcher: the
msgdispatcher now includes a vchannel watch task queue, where all
vchannels in the queue will subscribe to the MQ only once and pull
messages from the oldest vchannel checkpoint to the latest.

issue: https://github.com/milvus-io/milvus/issues/39862

pr: https://github.com/milvus-io/milvus/pull/39863

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-24 10:18:17 +08:00
cai.zhang
d703d8dac8
fix: [2.5] Set correct default configuration for task slots (#40821) 
issue: #39101

master pr: #39084

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-23 21:28:19 +08:00
SimFG
1263505808
feat: [2.5] support to deny dll according to database property (#40784) 
- issue: #40762
- pr: #40764

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-03-23 11:18:28 +08:00
cqy123456
7c716d61dc
fix:[2.5]wrong p.InterimIndexRefineQuantType default value and reduce ut run time (#40830) 
issue: https://github.com/milvus-io/milvus/issues/27678

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2025-03-21 19:10:13 +08:00
Buqian Zheng
cff0e82f57
enhance: [2.5] improve sparse query nnz metric (#40714) 
add query type and field id label; add metric for hybrid search

issue: https://github.com/milvus-io/milvus/issues/35853
pr: https://github.com/milvus-io/milvus/pull/40713

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2025-03-20 14:14:21 +08:00
XuanYang-cn
1a6761ac69
enhance: [cp25]Replace currRows with NumOfRows (#40074) (#40681) 
See also: #40068
pr: #40074

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-03-20 11:24:13 +08:00
congqixia
cd8d7efc96
enhance: [2.5] Add an extra parameter for slow log (#40759) 
Cherry-pick from master
pr: #40758
Related to #40756

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-19 18:10:13 +08:00
cai.zhang
0cfe835cac
enhance: [2.5] Support slot for index task and stats task (#40288) 
issue: #39101 

master pr: #39084

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-19 16:50:11 +08:00
congqixia
e49294cd16
enhance: [2.5] Add buffer forwarder for stream delta loading (#40559) (#40699) 
Cherry-pick from master
pr: #40559
See also #40558
Related to #35303 & #38066 as well

This PR:
- Add `BufferedForward` to limit memory usage forwarding stream delete
- Add `UseLoad` flag to determine `Delete` shall use `segment.Delete` or
`segment.LoadDelta`
- Fix delegator accidentally use always true candidate while load
streaming delta

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-19 10:28:14 +08:00
yihao.dai
aca9dd9105
fix: [2.5] Fix messages loss during seek (#40736) 
issue: https://github.com/milvus-io/milvus/issues/40733

pr: https://github.com/milvus-io/milvus/pull/40735

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-18 21:56:33 +08:00
congqixia
6d25fc56aa
enhance: Bump milvus & proto version to v2.5.7 (#40728) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-18 19:58:15 +08:00
congqixia
8de5174e4d
fix: [2.5] Remove lag targets after main dispatcher (#40717) 
Related to #40716

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-18 19:10:34 +08:00
Xianhui Lin
705b3c90a5
fix: Failed to rolling upgrade from v2.5.6 to new 2.5 version when enable JsonKeyStats (#40661) 
fix: Failed to rolling upgrade from v2.5.6 to new 2.5 version when
enable JsonKeyStats.The reason is that the file path of the jsonkeyindex
has changed.
issue: https://github.com/milvus-io/milvus/issues/40649
，https://github.com/milvus-io/milvus/issues/40669
https://github.com/milvus-io/milvus/issues/40707
master-pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-03-18 17:32:16 +08:00
Zhen Ye
167d058682
enhance: remove redundant resource key watch operation, just keep consistency of wal (#40503) 
issue: #38399
pr: #40235

related PR: #39522

- Just implement exclusive broadcaster between broadcast message with
same resource key to keep same order in different wal.
- After simplify the broadcast model, original watch-based broadcast is
too complicated and redundant, remove it.
- Add metrics for broadcast.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-18 15:46:25 +08:00
cqy123456
b8abd91962
enhance[2.5]: add more config for interimIndex to support more refine mode (#40429) 
issue: https://github.com/milvus-io/milvus/issues/27678

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2025-03-17 22:04:12 +08:00
XuanYang-cn
f455923ac9
enhance: Use correct counter metrics for overall wa calculation (#40394) (#40679) 
pr: #40394

- Use CounterVec to calculate sum of increase during a time period.
- Use entries number instead of binlog size

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-03-17 15:06:19 +08:00
Gao
09f7a60fe1
enhance: [2.5] make segment prune config refreshable (#40632) 
pr: #40539

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2025-03-14 16:12:09 +08:00
SimFG
a3c60a03b5
feat: [2.5] add DDLDB rate type and related quota configurations (#40653) 
- issue: #40650
- pr: #40651

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-03-14 15:36:17 +08:00
zhagnlu
6b9e141ada
enhance: reorder sub expr for conjunct expr (#40186) 
pr:#39872

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-03-14 15:16:08 +08:00
congqixia
d8a2c1a907
enhance: [2.5] Add channel seal policy based on blocking l0 (#40505) (#40535) 
Cherry-pick from 2.5
pr: #40505
Related to #40502

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-14 10:02:12 +08:00
cai.zhang
bdc0e68aaf
feat: [2.5] Support get segment binlogs info with new interface GetSegmentsInfo (#40466) 
issue: #40341 

master pr: #40464

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-14 00:14:08 +08:00
Xianhui Lin
f5e9dea2aa
fix: [2.5]fix the garbage cleanup logic of jsonkey stats && improve json key stats filer (#40039) 
fix: fix the garbage collection cleanup logic of jsonkey stats &&
improve json key stats filer
issue: https://github.com/milvus-io/milvus/issues/36995
https://github.com/milvus-io/milvus/issues/40034
https://github.com/milvus-io/milvus/issues/40041
https://github.com/milvus-io/milvus/issues/40106
https://github.com/milvus-io/milvus/issues/40138
pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-03-13 20:18:10 +08:00
yihao.dai
cd8f1fe0e4
enhance: [2.5] Remove unnecessary collection and partition label from the metrics (#40593) 
/kind improvement

pr: https://github.com/milvus-io/milvus/pull/39536

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-13 12:24:08 +08:00
Bingyi Sun
683b26ffb7
feat: cherry pick json path index (#40313) 
issue: #35528 
pr: #36750 
this pr includes json path index pr and some related prs:
1. update tantivy version #39253 
2. json path index #36750 
3. fall back to brute force #40076 
4. term filter #40140 
5. bug fix #40336

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-10 22:14:05 +08:00
aoiasd
ebf3416245
fix: main dispatcher not terminate cause messsage stream leak. (#40061) (#40351) 
Main dispatcher will leak when we remove solo dispatcher in the end.
relate: https://github.com/milvus-io/milvus/issues/40046
pr: https://github.com/milvus-io/milvus/pull/40061

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-03-05 20:14:02 +08:00
yihao.dai
7cbd40c7a1
fix: [2.5] Fix parsing import endTs (#40333) 
Parsing import beginTs, endTs as a hybrid timestamp.

issue: https://github.com/milvus-io/milvus/issues/40326

pr: https://github.com/milvus-io/milvus/pull/40332

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-05 11:40:13 +08:00
congqixia
4272703de4
enhance: Bump milvus & proto version to v2.5.6 (#40293) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-03 16:52:00 +08:00
SimFG
a945345110
fix: [2.5] use the different msg type for the OperatePrivilegeV2 api (#40193) 
- issue: #40178
- pr: #40192

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-03-03 10:20:01 +08:00
cai.zhang
4a75fdac03
enhance: Remove task meta when task is no need to do (#40146) 
issue: #39911 

master pr: #39084

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-02-25 10:35:57 +08:00