admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00
Code Issues Packages Projects Releases Wiki Activity
21,837 Commits 146 Branches 196 Tags
Commit Graph

1131 Commits

Author SHA1 Message Date
yihao.dai
aca9dd9105
fix: [2.5] Fix messages loss during seek (#40736) 
issue: https://github.com/milvus-io/milvus/issues/40733

pr: https://github.com/milvus-io/milvus/pull/40735

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-18 21:56:33 +08:00
congqixia
6d25fc56aa
enhance: Bump milvus & proto version to v2.5.7 (#40728) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-18 19:58:15 +08:00
congqixia
8de5174e4d
fix: [2.5] Remove lag targets after main dispatcher (#40717) 
Related to #40716

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-18 19:10:34 +08:00
Xianhui Lin
705b3c90a5
fix: Failed to rolling upgrade from v2.5.6 to new 2.5 version when enable JsonKeyStats (#40661) 
fix: Failed to rolling upgrade from v2.5.6 to new 2.5 version when
enable JsonKeyStats.The reason is that the file path of the jsonkeyindex
has changed.
issue: https://github.com/milvus-io/milvus/issues/40649
，https://github.com/milvus-io/milvus/issues/40669
https://github.com/milvus-io/milvus/issues/40707
master-pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-03-18 17:32:16 +08:00
Zhen Ye
167d058682
enhance: remove redundant resource key watch operation, just keep consistency of wal (#40503) 
issue: #38399
pr: #40235

related PR: #39522

- Just implement exclusive broadcaster between broadcast message with
same resource key to keep same order in different wal.
- After simplify the broadcast model, original watch-based broadcast is
too complicated and redundant, remove it.
- Add metrics for broadcast.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-18 15:46:25 +08:00
cqy123456
b8abd91962
enhance[2.5]: add more config for interimIndex to support more refine mode (#40429) 
issue: https://github.com/milvus-io/milvus/issues/27678

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2025-03-17 22:04:12 +08:00
XuanYang-cn
f455923ac9
enhance: Use correct counter metrics for overall wa calculation (#40394) (#40679) 
pr: #40394

- Use CounterVec to calculate sum of increase during a time period.
- Use entries number instead of binlog size

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-03-17 15:06:19 +08:00
Gao
09f7a60fe1
enhance: [2.5] make segment prune config refreshable (#40632) 
pr: #40539

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2025-03-14 16:12:09 +08:00
SimFG
a3c60a03b5
feat: [2.5] add DDLDB rate type and related quota configurations (#40653) 
- issue: #40650
- pr: #40651

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-03-14 15:36:17 +08:00
zhagnlu
6b9e141ada
enhance: reorder sub expr for conjunct expr (#40186) 
pr:#39872

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-03-14 15:16:08 +08:00
congqixia
d8a2c1a907
enhance: [2.5] Add channel seal policy based on blocking l0 (#40505) (#40535) 
Cherry-pick from 2.5
pr: #40505
Related to #40502

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-14 10:02:12 +08:00
cai.zhang
bdc0e68aaf
feat: [2.5] Support get segment binlogs info with new interface GetSegmentsInfo (#40466) 
issue: #40341 

master pr: #40464

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-14 00:14:08 +08:00
Xianhui Lin
f5e9dea2aa
fix: [2.5]fix the garbage cleanup logic of jsonkey stats && improve json key stats filer (#40039) 
fix: fix the garbage collection cleanup logic of jsonkey stats &&
improve json key stats filer
issue: https://github.com/milvus-io/milvus/issues/36995
https://github.com/milvus-io/milvus/issues/40034
https://github.com/milvus-io/milvus/issues/40041
https://github.com/milvus-io/milvus/issues/40106
https://github.com/milvus-io/milvus/issues/40138
pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-03-13 20:18:10 +08:00
yihao.dai
cd8f1fe0e4
enhance: [2.5] Remove unnecessary collection and partition label from the metrics (#40593) 
/kind improvement

pr: https://github.com/milvus-io/milvus/pull/39536

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-13 12:24:08 +08:00
Bingyi Sun
683b26ffb7
feat: cherry pick json path index (#40313) 
issue: #35528 
pr: #36750 
this pr includes json path index pr and some related prs:
1. update tantivy version #39253 
2. json path index #36750 
3. fall back to brute force #40076 
4. term filter #40140 
5. bug fix #40336

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-10 22:14:05 +08:00
aoiasd
ebf3416245
fix: main dispatcher not terminate cause messsage stream leak. (#40061) (#40351) 
Main dispatcher will leak when we remove solo dispatcher in the end.
relate: https://github.com/milvus-io/milvus/issues/40046
pr: https://github.com/milvus-io/milvus/pull/40061

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-03-05 20:14:02 +08:00
yihao.dai
7cbd40c7a1
fix: [2.5] Fix parsing import endTs (#40333) 
Parsing import beginTs, endTs as a hybrid timestamp.

issue: https://github.com/milvus-io/milvus/issues/40326

pr: https://github.com/milvus-io/milvus/pull/40332

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-05 11:40:13 +08:00
congqixia
4272703de4
enhance: Bump milvus & proto version to v2.5.6 (#40293) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-03 16:52:00 +08:00
SimFG
a945345110
fix: [2.5] use the different msg type for the OperatePrivilegeV2 api (#40193) 
- issue: #40178
- pr: #40192

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-03-03 10:20:01 +08:00
cai.zhang
4a75fdac03
enhance: Remove task meta when task is no need to do (#40146) 
issue: #39911 

master pr: #39084

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-02-25 10:35:57 +08:00
Xianhui Lin
a4eb2ce224
fix: [2.5]Revert qc statschecker for json key stats (#40125) 
Revert qc statschecker for json key stats
issue:https://github.com/milvus-io/milvus/issues/36995
pr:https://github.com/milvus-io/milvus/pull/39876

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-02-24 13:31:55 +08:00
cai.zhang
9cbab98382
enhance: [2.5] Export request timeout interval in config (#40118) 2025-02-23 13:47:12 +08:00
congqixia
709594f158
enhance: [2.5] Use v2 package name for pkg module (#40117) 
Cherry-pick from master
pr: #39990
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-23 00:46:01 +08:00
Xianhui Lin
c1de61ff7c
fix: [2.5]Replace the position of EnabledJSONKeyStats (#40108) 
Replace the position of EnabledJSONKeyStats
issue: https://github.com/milvus-io/milvus/issues/36995
pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-02-22 14:35:54 +08:00
congqixia
cb0dc6293c
enhance: [2.5] Sync proxy.maxTaskNum default value to 1024 (#40073) 
Cherry-pick from master
pr: #40072
Sync milvus.yaml value to default value for maxTaskNum

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-22 11:31:53 +08:00
cai.zhang
0caad5fe37
enhance: [2.5] Add metrics for proxy queue (#40071) 
master pr: #40070

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-02-21 21:39:54 +08:00
yihao.dai
b8a758b6c4
enhance: [2.5] Add get vector latency metric and refine request limit error message (#40085) 
issue: https://github.com/milvus-io/milvus/issues/40078

pr: https://github.com/milvus-io/milvus/pull/40083

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-02-21 20:19:55 +08:00
congqixia
cae8f75e47
enhance: Bump milvus version to 2.5.5 (#40017) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-19 18:48:54 +08:00
wei liu
3c2d8c1419
enhance: [2.5] Add management api to check querycoord balance status (#37784) (#39909) 
issue: #37783
pr: #37784

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-02-19 10:56:49 +08:00
Xianhui Lin
04175d8549
enhance: [2.5]compress jsonstatslog path only save filename (#39979) 
enhance: compress jsonstatslog path only save filename
issue: https://github.com/milvus-io/milvus/issues/36995
pr: https://github.com/milvus-io/milvus/pull/38039

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-02-19 10:20:52 +08:00
yihao.dai
7a92121c92
fix: [2.5] Fix failed to seek to earliest position (#39966) 
If it is the earliest message ID, skip the seek to prevent failure.

issue: https://github.com/milvus-io/milvus/issues/39964

pr: https://github.com/milvus-io/milvus/pull/39965

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-02-18 20:36:53 +08:00
zhagnlu
de8abe7970
fix:reset default auto index for varchar (#39908) 
pr: #39819

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-02-17 20:55:26 +08:00
cai.zhang
52434ccc78
enhance: [2.5] Limit the speed of the generating stats task (#39645) 
master pr: #39644

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-02-17 16:06:17 +08:00
XuanYang-cn
8067113133
enhance: [cp25]Enable to observe write amplification (#39661) (#39743) 
pr: #39661

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-02-17 16:00:17 +08:00
Zhen Ye
56c1a8d462
fix: ignore growing option is lost at hibridsearch (#39900) 
issue: #39892
pr: #39799

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-02-17 15:30:15 +08:00
XuanYang-cn
ee25af4c9b
enhance: Add configs for compaction schedule (#39010) (#39511) 
pr: #39010

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-02-17 11:44:15 +08:00
Xianhui Lin
f0964f769d
enhance: [2.5]Add json key inverted index in stats for optimization (#39876) 
Add json key inverted index in stats for optimization
issue: https://github.com/milvus-io/milvus/issues/36995
pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-02-16 20:12:15 +08:00
zhagnlu
16ea4fcb90
Revert "fix:reset default auto index type for scalar (#37086)" (#39820) 
This reverts commit eeb67a38450732eb8ab3a4eb4aaac6340142662.
pr: #39819

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-02-13 17:36:14 +08:00
wei liu
a50249e0bb
enhance: [2.5] dump pprof info if component stop progress timeout (#39760) 
issue: #39735
pr: #39726

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-02-13 15:30:47 +08:00
cqy123456
d121ac3a7a
enhance: [2.5]intermin index support different index type and more data type(fp16/bf16) (#39180) 
issue: https://github.com/milvus-io/milvus/issues/27678
related: https://github.com/milvus-io/milvus/pull/39753
some raw data status will change:
Intermin index has raw data: 
<html xmlns:v="urn:schemas-microsoft-com:vml"
xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:x="urn:schemas-microsoft-com:office:excel"
xmlns="http://www.w3.org/TR/REC-html40">

<head>

<meta name=ProgId content=Excel.Sheet>
<meta name=Generator content="Microsoft Excel 15">
<link id=Main-File rel=Main-File

href="file:////Users/cqy/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip.htm">
<link rel=File-List

href="file:////Users/cqy/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip_filelist.xml">
<!--table
	{mso-displayed-decimal-separator:"\.";
	mso-displayed-thousand-separator:"\,";}
@page
	{margin:.75in .7in .75in .7in;
	mso-header-margin:.3in;
	mso-footer-margin:.3in;}
.font5
	{color:windowtext;
	font-size:9.0pt;
	font-weight:400;
	font-style:normal;
	text-decoration:none;
	font-family:等线;
	mso-generic-font-family:auto;
	mso-font-charset:134;}
tr
	{mso-height-source:auto;
	mso-ruby-visibility:none;}
col
	{mso-width-source:auto;
	mso-ruby-visibility:none;}
br
	{mso-data-placement:same-cell;}
td
	{padding-top:1px;
	padding-right:1px;
	padding-left:1px;
	mso-ignore:padding;
	color:black;
	font-size:12.0pt;
	font-weight:400;
	font-style:normal;
	text-decoration:none;
	font-family:等线;
	mso-generic-font-family:auto;
	mso-font-charset:134;
	mso-number-format:General;
	text-align:general;
	vertical-align:middle;
	border:none;
	mso-background-source:auto;
	mso-pattern:auto;
	mso-protection:locked visible;
	white-space:nowrap;
	mso-rotate:0;}
ruby
	{ruby-align:left;}
rt
	{color:windowtext;
	font-size:9.0pt;
	font-weight:400;
	font-style:normal;
	text-decoration:none;
	font-family:等线;
	mso-generic-font-family:auto;
	mso-font-charset:134;
	mso-char-type:none;
	display:none;}
-->
</head>

<body link="#0563C1" vlink="#954F72">


sparse vector | growing segment | sealed segment
-- | -- | --
BM25 | no | no
IP | yes | no
  |   |  
dense vector | growing segment | sealed segment
ivf flat cc | yes | yes
scann_dvr | no | no



</body>

</html>

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2025-02-13 09:56:47 +08:00
cai.zhang
667c84740c
enhance: [2.5] Add metrics for parse expression (#39716) 
master pr: #39654

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-02-12 10:30:46 +08:00
yihao.dai
c2986d4ab9
fix: [2.5] Fix concurrent map (#39776) 
issue: https://github.com/milvus-io/milvus/issues/39778

pr: https://github.com/milvus-io/milvus/pull/39775

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-02-11 22:12:46 +08:00
SimFG
f542c866b3
fix: [2.5] root check the role list if rootShouldBindRole is true (#39714) 
- issue: #39712
- pr: #39713

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-02-11 10:10:51 +08:00
yihao.dai
4a8e6fc59c
fix: [2.5] Fix flowgraph leak (#39686) 
Consider lag targets when determining whether to remove the dispatcher
manager.

issue: https://github.com/milvus-io/milvus/issues/39642

pr: https://github.com/milvus-io/milvus/pull/39656

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-02-08 15:40:45 +08:00
Zhen Ye
858dc10ef9
enhance: broadcast with event-based notification (#39550) 
issue: #38399
pr: #39522

- broadcast message can carry multi resource key now.
- implement event-based notification for broadcast messages
- broadcast message use broadcast id as a unique identifier in message
- broadcasted message on vchannels keep the broadcasted vchannel now.
- broadcasted message and broadcast message have a common broadcast
header now.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-02-07 11:50:50 +08:00
jaime
ddc5b299ad
enhance: expose more metrics data (#39466) 
issue: #36621 #39417
pr: #39456
1. Adjust the server-side cache size.
2. Add source information for configurations.
3. Add node ID for compaction and indexing tasks.
4. Resolve localhost access issues to fix health check failures for
etcd.

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2025-02-07 11:48:45 +08:00
congqixia
c32ec451b0
fix: [2.5] Use param item formatter to avoid SetConfig to overlay (#39597) (#39636) 
Cherry-pick from master
pr: #39597 
Related to #39596

When updating the build param configuration, the `Formatter` could be
used to do so and completed avoid touching the `overlay` config items

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-07 10:02:44 +08:00
SimFG
30411d6d3a
fix: [2.5] deny to set the mmap param for the alter index api (#39520) 
- issue: #39517
- pr: #39518

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-01-22 23:55:06 +08:00
Xianhui Lin
20c6de156e
feat: update milvusproto version to 2.5.5 (#39478) 
feat: update milvusproto version to 2.5
issue: https://github.com/milvus-io/milvus/issues/39093

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-01-21 15:11:05 +08:00
Xianhui Lin
3b3959c625
enhance: [2.5]Add schema update time verification for insert and upsert to use cache (#39382) 
enhance: Add schema update time verification for insert and upsert to
use cache
issue: https://github.com/milvus-io/milvus/issues/39093
Related to pr: https://github.com/milvus-io/milvus/pull/39096

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-01-21 10:19:11 +08:00