admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00
Code Issues Packages Projects Releases Wiki Activity
22,154 Commits 146 Branches 196 Tags
Commit Graph

22154 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
wei liu
b08d9efe69
fix: Prevent delegator unserviceable due to shard leader change (#42689) (#43309) 
issue: #42098 #42404
pr: #42689
Fix critical issue where concurrent balance segment and balance channel
operations cause delegator view inconsistency. When shard leader
switches between load and release phases of segment balance, it results
in loading segments on old delegator but releasing on new delegator,
making the new delegator unserviceable.

The root cause is that balance segment modifies delegator views, and if
these modifications happen on different delegators due to leader change,
it corrupts the delegator state and affects query availability.

Changes include:
- Add shardLeaderID field to SegmentTask to track delegator for load
- Record shard leader ID during segment loading in move operations
- Skip release if shard leader changed from the one used for loading
- Add comprehensive unit tests for leader change scenarios

This ensures balance segment operations are atomic on single delegator,
preventing view corruption and maintaining delegator serviceability.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-07-15 17:46:51 +08:00
zhikunyao
180214d406
test: update pytest base image 2.5 (#43328) 
pr: #43327

Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>
 2025-07-15 15:34:51 +08:00
Spade A
21d6866a56
fix: fix text match index / json key stats index leak when segment released (#43308) 
pr: https://github.com/milvus-io/milvus/pull/42655
issue: https://github.com/milvus-io/milvus/issues/42626

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-07-15 15:22:18 +08:00
Bingyi Sun
fb25da1419
enhance: Save meta with txn limit (#43263) (#43287) 
pr: #43263

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-07-14 16:42:50 +08:00
wei liu
4952b8c416
enhance: apply load config changes after QueryCoord restart (#43108) (#43236) 
issue: #43107
pr: #43108
- Add checkLoadConfigChanges() to apply load config during startup
- Call config check in startQueryCoord() after restart
- Skip auto-updates for collections with user-specified replica numbers
- Add is_user_specified_replica_mode field to preserve user settings
- Add comprehensive unit tests with mockey

Ensures existing collections use latest cluster-level config after
restart.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-07-14 10:22:50 +08:00
wei liu
33e9b873de
enhance: add logs for count(*) (#43001) (#43249) 
pr: #43001

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-07-11 19:14:48 +08:00
cai.zhang
fa382ed50a
fix: [2.5] Fix regeneratePartitionStats failed after restore clusteringCompactionTask (#43206) 
issue: #43186 
master pr: #43205

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-07-10 15:30:48 +08:00
yihao.dai
e0729bf1ae
enhance: [2.5] Print segments info after import done (#43201) 
issue: https://github.com/milvus-io/milvus/issues/42488

pr: https://github.com/milvus-io/milvus/pull/43200

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-07-10 14:58:49 +08:00
cai.zhang
b900e4a63a
fix: [2.5] Handle error for invalid function params and prevent panic (#43190) 
issue: #43188 
master pr: #43189

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-07-09 11:40:47 +08:00
Xianhui Lin
bdfc740a73
feat: [2.5]reorder RootCoord shutdown to the last in stopCoordinators (#43024) 
reorder RootCoord shutdown to the last in stopCoordinators
issue:https://github.com/milvus-io/milvus/issues/43018

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-07-07 17:08:49 +08:00
congqixia
e13b55fc0a
fix: [2.5] Remove trimming space logic when validating collection name (#43064) (#43138) 
Cherry-pick from master
pr: #43064
Related to #43031

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-07-07 09:54:46 +08:00
Chun Han
4fe8011a70
enhance: refine variable-length-type memory usage(#38736) (#43093) 
related: #38736
pr: https://github.com/milvus-io/milvus/pull/39578

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-07-04 18:26:45 +08:00
congqixia
2531ebda27
fix: [2.5] Check field mmap property before apply collection level one (#43091) 
Cherry-pick from master
pr: #43090
Related to #43089

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-07-03 14:32:45 +08:00
XuanYang-cn
3666fced6c
enhance: Upgrade pymilvus to 2.5.12rc8 (#43084) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-07-03 11:30:45 +08:00
yanliang567
c84e0119db
test: [cp 2.5]Add verfification tests for partition key filter and ttl issue (#43077) 
related issue: #42918 
pr: #43052

also improve performance in check query results

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2025-07-03 10:22:46 +08:00
nico
5f7ef6d9dd
test: update test cases[2.5] (#42998) 
pr: #42991

Signed-off-by: nico <cheng.yuan@zilliz.com>
 2025-07-03 09:18:51 +08:00
yihao.dai
53cd81b2aa
fix: [2.5] Fix index blocked by failed sort stats (#43061) 
When l0 compaction is executing, do not mark the stats task as failed;
keep it in the init state to allow retry.

issue: https://github.com/milvus-io/milvus/issues/43039

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-07-02 22:32:45 +08:00
zhikunyao
d6ee1d1c3c
enhance: update minio to RELEASE.2024-05-28T17-19-04Z (#43071) 
pr: #43063

Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>
 2025-07-02 16:18:45 +08:00
wei liu
73210303a9
fix: Fix exclude nodes clearing logic position in load balancer retry (#43002) 
issue: #42994
pr: #42577 #40438

cp partial logic from pr #42577 and #40438

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-06-30 16:10:44 +08:00
zhikunyao
23f4c2adf5
enhance: fix cmake version update to 3.31.8 (#43017) 
Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>
 2025-06-30 14:24:43 +08:00
zhikunyao
fc97b06239
enhance: update cmake version in .env (#43011) 
Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>
 2025-06-30 10:24:54 +08:00
sre-ci-robot
af8479b084
enhance: update cmake version to 3.31.8 (#43004) 
Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>
Co-authored-by: Zhikun Yao <zhikun.yao@zilliz.com>
 2025-06-28 15:56:44 +08:00
sre-ci-robot
7208225f5d
[automated] Bump milvus version to v2.5.14 (#43006) 
Bump milvus version to v2.5.14
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2025-06-28 15:54:47 +08:00
zhikunyao
4e98647cf5
test: [skip e2e]update helm to fix kafka image update (#42997) 
pr: #42993

Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>
 2025-06-27 17:17:43 +08:00
congqixia
062fc368a5
enhance: Bump milvus & proto version to v2.5.14 (#42967) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
pkg/v2.5.14 v2.5.14 		2025-06-26 18:18:43 +08:00
Chun Han
bfa9688da3
enhance: supporting separate chunk cache pool(#42803) (#42901) 
related: #42803

1. add a new thread pools using folly::CPUThreadPoolExecutor, named by
FThreadPools
2. reading vectors from chunkcache will use the separated
CHUNKCACHE_POOL to avoid being influenced by load collection
3. Note. For safety on cloud side on 2.5.x, only read-chunk-cache
operations is using this newly created thread pools other caller points
for threadpool will be mutated in the near future
4. master-branch doesn't need this pr as caching layer unified the chunk
cache behaviour

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-06-26 15:52:43 +08:00
yihao.dai
5aaaef3f7e
fix: [2.5] Consider fields number when preallocating ids for import (#42935) 
In corner cases where there are many fields but only a small number of
rows to import, the default preallocated IDs may be insufficient. To
address this, consider the number of fields when preallocating IDs.

issue: https://github.com/milvus-io/milvus/issues/42518

pr: https://github.com/milvus-io/milvus/pull/42810

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-06-26 09:56:41 +08:00
foxspy
47f265864b
enhance: [2.5] update knowhere version for milvus 2.5.14 (#42939) 
issue: #42937 
pr: #42938

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
 2025-06-25 21:06:43 +08:00
congqixia
b1bbc56b54
fix: [2.5] Use task timestamp to calculate TTL timestamp (#42944) 
Cherry-pick from master
pr: #42920
Related to #42918

Previously the `CollectionTtlTimestamp` could be overflowed when the
guarantee_ts==1, which means using `Eventually` consistency level.

This patch use task timestamp, allocated by scheduler, to generate ttl
timestamp ignore the potential very small timestamp being used.

Also add overflow check for ttl timestamp calculated.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-06-25 20:56:42 +08:00
zhagnlu
fe05970eba
fix is_not_in for trie index (#42886) 
pr: #42716

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-06-25 17:30:43 +08:00
cai.zhang
0a62d6d509
enhance: Add Size interface to FileReader to eliminate the StatObject call during Read (#42911) 
issue: #42907 

master pr: #42908

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-06-25 15:26:42 +08:00
aoiasd
7feeeabca5
enhance: [2.5] bm25 stats local cache use local storage path (#42924) 
relate: https://github.com/milvus-io/milvus/pull/42923

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-06-25 13:44:46 +08:00
congqixia
3efb0366f8
fix: [2.5] OR binary expr is prunable only when both children are prunable (#42912) (#42915) 
Cherry-pick from master
pr: #42912 
Related to #42903

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-06-24 17:30:42 +08:00
Spade A
017eb9ffe2
fix: fix some bugs discovered by chaos tests --- cherry pick (#42909) 
master: https://github.com/milvus-io/milvus/pull/42906
issue: https://github.com/milvus-io/milvus/issues/42870

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-06-24 17:10:42 +08:00
congqixia
1e55112de7
enhance: [2.5] Fill dbName for OperatePrivilegeV2Request in interceptor (#42898) (#42904) 
Cherry-pick from master
pr: #42898

Related to #40340

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-06-24 10:08:41 +08:00
zhagnlu
c88f1be985
fix:fix wrong use return error for parse unsupported arith (#42729) (#42890) 
pr: #42729

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-06-23 20:36:46 +08:00
zhagnlu
00549c0729
fix:reject div or mod by zero for binaryarith expr (#42691) (#42887) 
pr: #42691

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-06-23 20:34:50 +08:00
nico
c39ea71638
test: update nightly script (#42868) 
pr: #41782

Signed-off-by: nico <cheng.yuan@zilliz.com>
 2025-06-23 18:40:41 +08:00
zhuwenxing
591d7cc51a
test: [cp2.5]add negative import case (#42883) 
pr: https://github.com/milvus-io/milvus/pull/42100

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2025-06-23 18:38:41 +08:00
cai.zhang
513ff7d331
enhance: [2.5] Skip modify field meta when rename collection or rename dbName (#42876) 
issue: #42873 

master pr: #42875

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-06-23 18:24:43 +08:00
Bingyi Sun
1a1d46d695
fix: Remove cached null expr result (#42783) 
pr: https://github.com/milvus-io/milvus/pull/42818
issue: #42698
cached result may be changed in caller so there is no need to cache it

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-06-23 16:02:42 +08:00
aoiasd
f3e8f61f60
enhance: [GoSDK]support run analyzer by collection field (#42642) (#42812) 
relate: https://github.com/milvus-io/milvus/issues/42094
pr: https://github.com/milvus-io/milvus/pull/42642

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-06-23 15:42:41 +08:00
cai.zhang
78b66a29b6
fix: [2.5] Reduce task slot for standalone to 1/4 of normal datanode (#42809) 
issue: #42129 

master pr: #42808

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-06-20 16:38:46 +08:00
Spade A
79ffc17bfe
enhance: optimize tantivy cargo config cherry-pick (#42881) 
master: https://github.com/milvus-io/milvus/pull/42880
issue: https://github.com/milvus-io/milvus/issues/42879

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-06-20 16:18:11 +08:00
Zhen Ye
30b2a66f59
fix: rocksmq cannot graceful stop (#42843) 
issue: #40532
pr: #42841

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-06-19 22:18:40 +08:00
Spade A
b837fb2d1e
fix: update tantivy for fixing threads explosion in file watcher (#42828) 
master pr: #41450

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-06-18 15:40:40 +08:00
cai.zhang
e30fc0fbaf
enhance: [2.5] Make Web UI toggleable via config (#42815) 
issue: #42813

master pr: #42814

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-06-18 12:20:39 +08:00
zhuwenxing
95ac0415c4
test:[skip e2e][cp2.5]remove tls connection (#42800) 
pr: https://github.com/milvus-io/milvus/pull/42799

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2025-06-18 10:34:43 +08:00
Chun Han
c403f01e6d
enhance: avoid using threadpool when column is ready in chunk cache(#42803) (#42804) 
related: #42803

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-06-18 02:44:40 +08:00
Spade A
77bcd1dac0
fix: update tantivy for solving threads exposion for tantivy (#42713) 
update tantivy: https://github.com/zilliztech/tantivy/pull/16

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-06-17 16:01:01 +08:00