admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00
Code Issues Packages Projects Releases Wiki Activity
21,895 Commits 146 Branches 196 Tags
Commit Graph

21895 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
yihao.dai
cba8feade3
enhance: [2.5] Add UTF-8 string validation for import (#40746) 
issue: https://github.com/milvus-io/milvus/issues/40684

pr: https://github.com/milvus-io/milvus/pull/40694

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-31 21:08:21 +08:00
yihao.dai
753f8c8410
enhance: [2.5] Reduce the allocation of ID (#41005) 
This PR is a supplement to PR
https://github.com/milvus-io/milvus/pull/40960.

issue: https://github.com/milvus-io/milvus/issues/40207

pr: https://github.com/milvus-io/milvus/pull/40960

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-31 17:58:21 +08:00
wei liu
d185a8f941
enhance: Balance the collection with the largest row count first (#40958) 
issue: #37651
pr: #40297
this PR enable to balance the collection with largest row count first,
to avoid temporary migration of small table data to new nodes during
their onboarding, only to be moved out again after the large table
balance, which would cause unnecessary load.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-03-31 16:14:21 +08:00
cqy123456
48eabe3d2e
fix:[2.5]growing mmap data type crashed by nullable input (#40980) 
issue: https://github.com/milvus-io/milvus/issues/40981
master pr: https://github.com/milvus-io/milvus/pull/40994

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2025-03-31 14:42:20 +08:00
groot
712d1644d8
enhance: bulkinsert supports parsing sparse vector form parquet struct (#40874) 
issue: https://github.com/milvus-io/milvus/issues/40777
pr: https://github.com/milvus-io/milvus/pull/40927

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2025-03-31 14:20:31 +08:00
aoiasd
0ba389e434
enhance: [2.5] add utf8 check for all varchar field #40670 (#40993) 
https://github.com/milvus-io/milvus/issues/40684
pr: https://github.com/milvus-io/milvus/pull/40670

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-03-31 14:08:20 +08:00
yihao.dai
1caeac937b
fix: [2.5] Fix delete data loss due to duplicate binlogID (#40985) 
This PR is a supplement to PR
[#40960](https://github.com/milvus-io/milvus/pull/40960).

issue: https://github.com/milvus-io/milvus/issues/40207

pr: https://github.com/milvus-io/milvus/pull/40960

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-31 10:38:31 +08:00
cai.zhang
7e00ab6671
fix: [2.5] Add the field index lock for getSegmentsIndexStates (#40969) 
issue: #40966 

master pr: #40968

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-30 22:50:20 +08:00
Buqian Zheng
64c8e009be
fix: [2.5] update gitignore file (#40911) 
pr: https://github.com/milvus-io/milvus/pull/40910
issue: https://github.com/milvus-io/milvus/issues/40833
using this issue only to remove mergify bot label

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2025-03-28 18:40:19 +08:00
aoiasd
d49f0b65d1
fix: rocksmq consumer register not concurrent safe (#39387) (#40885) 
relate: https://github.com/milvus-io/milvus/issues/39336
pr: https://github.com/milvus-io/milvus/pull/39387

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-03-28 16:08:18 +08:00
cai.zhang
7db59fb1fb
fix: [2.5] Get all children deltalogs for segment to load (#40957) 
issue: #40207
master pr: #40956

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-28 16:04:19 +08:00
Buqian Zheng
8504a7b98d
enhance: [2.5] avoid re-query if hybrid search requested only pk as output field (#40906) 
pr: https://github.com/milvus-io/milvus/pull/40842
issue: https://github.com/milvus-io/milvus/issues/40833

---------

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2025-03-28 14:50:26 +08:00
yihao.dai
27ea5d14dc
fix: [2.5] Fix delete data loss due to duplicate binlogID (#40976) 
With concurrenct L0 compaction
(https://github.com/milvus-io/milvus/pull/36816), delta logs might be
written to the same L1 segment, causing logID duplication when using the
incremental beginLogID. This PR removes the beginLogID mechanism and
instead passes a log ID range, where the number of IDs in the range
equals the number of compaction segment binlogs multiplied by an
expansion factor.

issue: https://github.com/milvus-io/milvus/issues/40207

pr: https://github.com/milvus-io/milvus/pull/40960

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-28 14:34:21 +08:00
Chun Han
d420f100c0
enhance: refine array view to optimize memory usage(#38736) (#40206) 
related: #38736
pr: https://github.com/milvus-io/milvus/pull/38808

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-03-28 11:08:18 +08:00
congqixia
51efe9a60c
fix: [2.5][GoSDK] Set nullable according to fieldSchema for RowBased insert (#40928) (#40962) 
Cherry-pick from master
pr: #40928

Related to #40737

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-28 10:30:20 +08:00
Bingyi Sun
f0346a149a
fix: Fix using json index when iterative_filter is specified (#40946) 
issue: #40934 
pr: #40945

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-27 23:02:20 +08:00
wei liu
b64bb63e77
enhance: [2.5] Add trigger interval config for auto balance (#39154) (#39918) 
issue: #39156
pr: #39154

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-03-27 16:40:23 +08:00
zhagnlu
deed5b5df4
enhance:change multi or expr to in expr (#40751) 
pr: #40757

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-03-27 11:08:20 +08:00
congqixia
2411c184a8
enhance: [2.5] Support detailed manual compaction criterion (#40892) (#40924) 
Cherry-pick from master
pr: #40892
Related to #40866

This PR:
- update go-api/v2 and support partition id/channel/segment level manual
compaction
- refines the compaction trigger implementation
- unify the compaction signal usage

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-27 10:36:22 +08:00
cai.zhang
534b628278
fix: [2.5] exists should have a higher priority (#40865) 
issue: #40837 

master pr: #40864

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-26 20:02:34 +08:00
cai.zhang
e8970e8bf4
fix: [2.5] Update milvus key words (#40870) 
issue: #40290 
master pr: #40869

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-26 19:44:32 +08:00
aoiasd
182b3c17d0
enhance: remain raw token for audit log (#40637) (#40867) 
pr: https://github.com/milvus-io/milvus/pull/40637

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-03-26 14:54:18 +08:00
Gao
9c41636a95
fix: [2.5] correct WithGroupSize while reducing (#40920) 
issue: #40887 
pr: #40888

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2025-03-26 12:02:20 +08:00
zhuwenxing
8ec8a7a9c4
test: using pytest_tagging to replace pytest_tags (#40902) 
pr: https://github.com/milvus-io/milvus/pull/40878

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2025-03-26 10:46:31 +08:00
zhuwenxing
b0cc8bc9e0
test: [cherry-pick][2.5] add more restful v2 api testcases (#40915) 
pr: https://github.com/milvus-io/milvus/pull/39558

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2025-03-26 10:44:25 +08:00
zhuwenxing
99670db5e9
test: update text empty percent in go client fts test (#40010) 
pr: https://github.com/milvus-io/milvus/pull/40009

/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2025-03-25 17:28:23 +08:00
Zhen Ye
53438c751d
fix: skip consuming from streaming service message (#40879) 
issue: #40532
pr: #40877

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-25 15:22:23 +08:00
cai.zhang
ec0c25aaec
fix: [2.5] As the segment size increases the number of slots should also increase (#40862) 
issue: #40858 
master pr: #39084

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-25 14:42:24 +08:00
congqixia
97fc68bf08
fix: [2.5] Set task queue time before enqueue (#40815) (#40853) 
Cherry-pick from master
pr: #40815 
Related to #40814

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-25 14:02:22 +08:00
yihao.dai
e02a3daff5
enhance: [2.5] Optimize datacoord meta mutex (#40753) 
Use a separate collection mutex.

issue: https://github.com/milvus-io/milvus/issues/40551

pr: https://github.com/milvus-io/milvus/pull/40552

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-25 13:50:21 +08:00
wt
34131d93ae
test: test mix scenes support batch insertion (#40871) 
issues: https://github.com/milvus-io/milvus/issues/40868

Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2025-03-25 10:52:29 +08:00
yihao.dai
9b7d66d9da
fix: [2.5] Fix channel not balance on datanodes (#40854) 
1. Prevent channels from being assigned to only one datanode during
datacoord startup.
2. Optimize the channel assignment policy by considering newly assigned
channels.

issue: https://github.com/milvus-io/milvus/issues/40421,
https://github.com/milvus-io/milvus/issues/37630

pr: https://github.com/milvus-io/milvus/pull/40422

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-25 10:06:24 +08:00
yihao.dai
b534c9d804
enhance: [2.5] Introduce batch subscription in msgdispatcher (#40596) 
Introduce a batch subscription mechanism in msgdispatcher: the
msgdispatcher now includes a vchannel watch task queue, where all
vchannels in the queue will subscribe to the MQ only once and pull
messages from the oldest vchannel checkpoint to the latest.

issue: https://github.com/milvus-io/milvus/issues/39862

pr: https://github.com/milvus-io/milvus/pull/39863

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-24 10:18:17 +08:00
cai.zhang
d703d8dac8
fix: [2.5] Set correct default configuration for task slots (#40821) 
issue: #39101

master pr: #39084

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-23 21:28:19 +08:00
SimFG
1263505808
feat: [2.5] support to deny dll according to database property (#40784) 
- issue: #40762
- pr: #40764

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-03-23 11:18:28 +08:00
ThreadDao
31f9e3c789
[skip-e2e] test: enable jsonKeyStats in pulsar nightly test (#40826) 
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
v2.5.7 		2025-03-21 19:28:15 +08:00
congqixia
0f0cb4c189
enhance: [2.5] Close component in topological order when unsub channel (#40796) (#40819) 
Cherry-pick from master
pr: #40796

Related to #40795

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-21 19:12:15 +08:00
cqy123456
7c716d61dc
fix:[2.5]wrong p.InterimIndexRefineQuantType default value and reduce ut run time (#40830) 
issue: https://github.com/milvus-io/milvus/issues/27678

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2025-03-21 19:10:13 +08:00
Bingyi Sun
d09483809c
fix: Fix tasks will panic if one of them throw an exception. (#40760) 
issue: #40690 
pr: https://github.com/milvus-io/milvus/pull/40691

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-21 17:50:18 +08:00
XuanYang-cn
70ad96ffb3
enhance: Upgrade PyMilvus version to 2.5.6rc5 (#40748) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-03-21 17:48:28 +08:00
jaime
e538d2c106
fix:etcd advertise-client-urls in docker-compose (#40805) 
issue: #40696
pr: #40804

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2025-03-21 15:50:13 +08:00
ThreadDao
635f6c5300
[GoSDK][2.5] test: Cherry-pick gosdk test commits from master branch (#40778) 
pr: #39694 #40352 #40775 #40787

---------

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2025-03-21 14:30:14 +08:00
Bingyi Sun
0a1272fc6f
fix: Revert "enhance: update tantivy version (#39253)" (#40769) 
issue: #35528 
pr: #40313 
This reverts commit d3cec7f277b5f73a8acf5100eb8d744fbc476348.

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-21 11:24:17 +08:00
congqixia
50241e4cf9
enhance: [2.5][GoSDK] Support JSON Path index (#40754) (#40791) 
Cherry-pick from master
pr: #40754 
Related to #35528

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-21 10:28:13 +08:00
zhagnlu
14259a3f8a
fix:fix exception when do arith expr when using index (#40799) 
pr: #40783

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-03-20 19:34:49 +08:00
Xianhui Lin
8bdff401a3
fix: fix indexchecker schema released (#40809) 
pr:https://github.com/milvus-io/milvus/pull/38039
issue:https://github.com/milvus-io/milvus/issues/36995

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-03-20 18:05:22 +08:00
XuanYang-cn
5ca2af8124
fix: [2.5]L0 brings its own start pos when syncing (#40664) 
See also: #40388, #40207
pr: #40663

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-03-20 14:20:17 +08:00
Buqian Zheng
cff0e82f57
enhance: [2.5] improve sparse query nnz metric (#40714) 
add query type and field id label; add metric for hybrid search

issue: https://github.com/milvus-io/milvus/issues/35853
pr: https://github.com/milvus-io/milvus/pull/40713

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2025-03-20 14:14:21 +08:00
XuanYang-cn
281260e48a
fix: Massive memory cost when compacting (#40763) 
downloads batch binlogs instead of all segment's binlogs

See also: #40761

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-03-20 11:28:11 +08:00
XuanYang-cn
1a6761ac69
enhance: [cp25]Replace currRows with NumOfRows (#40074) (#40681) 
See also: #40068
pr: #40074

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-03-20 11:24:13 +08:00