admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
21,734 Commits 137 Branches 188 Tags
Commit Graph

10121 Commits

Author SHA1 Message Date
congqixia
709594f158
enhance: [2.5] Use v2 package name for pkg module (#40117) 
Cherry-pick from master
pr: #39990
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-23 00:46:01 +08:00
congqixia
ecdbc12678
enhance: [2.5] Remove hardcoded partition num in restful handler (#40113) 
Cherry-pick from master
pr: #40112
The partition num shall be determined by core logic if user did not
specifiy the partition num in request.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-22 16:58:17 +08:00
Xianhui Lin
c1de61ff7c
fix: [2.5]Replace the position of EnabledJSONKeyStats (#40108) 
Replace the position of EnabledJSONKeyStats
issue: https://github.com/milvus-io/milvus/issues/36995
pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-02-22 14:35:54 +08:00
Chun Han
2677f0b834
fix: cancel sub contexts casade when http request timeout(#40030) (#40060) 
related: #40030
pr: https://github.com/milvus-io/milvus/pull/40059

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-02-22 11:39:53 +08:00
zhagnlu
d66d3f0d47
enhance:decrease dump snapshot limit from 10w to 1w (#40102) 
pr: #40101

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-02-22 11:29:53 +08:00
cai.zhang
be30f5301d
enhance: Refine index task scheduler policy (#40104) 
master pr: #39084 

issue: #39101

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-02-22 11:25:54 +08:00
congqixia
898606ae4c
enhance: [2.5] Avoid stringtoslicebytes copy for BatchPKExists (#40097) 
Cherry-pick from master
pr: #40096 
Using unsafe.Slice to convert string to []byte by directly using
underlying data could avoid lots of copy and cpu time

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-21 21:41:21 +08:00
cai.zhang
0caad5fe37
enhance: [2.5] Add metrics for proxy queue (#40071) 
master pr: #40070

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-02-21 21:39:54 +08:00
yihao.dai
b8a758b6c4
enhance: [2.5] Add get vector latency metric and refine request limit error message (#40085) 
issue: https://github.com/milvus-io/milvus/issues/40078

pr: https://github.com/milvus-io/milvus/pull/40083

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-02-21 20:19:55 +08:00
congqixia
113d17646b
fix: [2.5] Add sub task pool for multi-stage tasks (#40081) 
Cherry-pick from master
pr: #40079
Related to #40078

Add a subTaskPool to execute sub task in case of logic deadlock
described in issue.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-21 19:17:55 +08:00
Chun Han
39b1f4aaea
enhance: support return configable properties when describing index(#39951) (#40043) 
related: #39951
pr: https://github.com/milvus-io/milvus/pull/40042

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-02-21 18:51:55 +08:00
wei liu
82fb0bf9c1
fix: [2.5] task delta cache leak on reduce task (#40056) 
issue: #40052
pr: #40055

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-02-21 16:49:54 +08:00
wei liu
e42c944e04
fix: [2.5] querycoord panic in cornor case (#40058) 
issue: #40050 
pr: #40057

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-02-21 11:19:58 +08:00
congqixia
b7c631f0b7
enhance: [2.5] Remove frequent json stats distribution log (#40027) 
Info log is too frequent only printing the json stats & index info

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-20 13:33:57 +08:00
zhenshan.cao
f142b5bff8
fix: Fix wrong trigger of json key index (#40026) 
pr: https://github.com/milvus-io/milvus/pull/38039

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2025-02-20 03:59:11 +08:00
SimFG
37874ed6d7
fix: [2.5] enhance isBalanced function to correctly count quote pairs (#40002) 
- issue: #39999
- pr: #40001

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-02-19 20:19:00 +08:00
zhagnlu
ce6d8ba836
enhance: optimize expr performace for some points (#39695) (#39938) 
1. skip get expr arguments which deserialize proto for every batch
execute.
2. replace unordered_set with sort array that has better performace for
small set.
pr: #39695

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-02-19 19:52:53 +08:00
XuanYang-cn
bd5c56a77b
fix: [cp25]Negative -1 executing compaction tasks (#39955) 
See also: #39675
pr: #39954

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-02-19 18:42:53 +08:00
congqixia
96922c0a87
fix: [2.5] Use start pos ts instead for sealSegmentByLifetime policy (#39994) 
Cherry-pick from master
pr: #39982
Related to #39981

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-19 17:48:55 +08:00
XuanYang-cn
cc6b2ff150
fix: [cp25]A segment may never transfer from sealed to flushing (#39996) 
See also: #39717
pr: #39993

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-02-19 16:50:59 +08:00
Bingyi Sun
6cd00880e2
fix: Skip create primary key index when loading pk index (#39922) 
issue: https://github.com/milvus-io/milvus/issues/39907
pr: https://github.com/milvus-io/milvus/pull/39921

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-02-19 15:54:52 +08:00
cai.zhang
f904b73662
fix: [2.5]Skip text index creation when segment is zero after sorting (#39969) 
issue: #39961 

master pr: #39962

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-02-19 10:58:51 +08:00
wei liu
3c2d8c1419
enhance: [2.5] Add management api to check querycoord balance status (#37784) (#39909) 
issue: #37783
pr: #37784

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-02-19 10:56:49 +08:00
Xianhui Lin
04175d8549
enhance: [2.5]compress jsonstatslog path only save filename (#39979) 
enhance: compress jsonstatslog path only save filename
issue: https://github.com/milvus-io/milvus/issues/36995
pr: https://github.com/milvus-io/milvus/pull/38039

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-02-19 10:20:52 +08:00
Xianhui Lin
63576738b6
fix: [2.5]exists expr on nested dictionaries fail in JSONkeyIndex (#39972) 
fix: exists expr on nested dictionaries fail in JSONkeyIndex
issue: https://github.com/milvus-io/milvus/issues/39963
pr: https://github.com/milvus-io/milvus/pull/38039

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-02-19 10:18:51 +08:00
wei liu
bf54f47c34
enhance: [2.5] use rated logger for high frequency log in dist handler (#39452) (#39928) 
pr: #39452

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-02-18 14:32:52 +08:00
wei liu
7ea0af572a
enhance: [2.5] Optimize the result format of GetQueryNodeDistribution (#39664) (#39926) 
pr: #39664
Use string array for SealedSegmentIDs to prevent precision loss in JSON
parsers. Large integers (int64) may be incorrectly rounded when parsed
as double.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-02-18 09:59:07 +08:00
congqixia
2eaa1a4869
fix: [2.5] Use text_log prefix for TextMatchIndex null offset file (#39936) 
Cherry-pick from master
pr: #39935
Related to #39933

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-18 00:43:07 +08:00
zhagnlu
de8abe7970
fix:reset default auto index for varchar (#39908) 
pr: #39819

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-02-17 20:55:26 +08:00
cai.zhang
52434ccc78
enhance: [2.5] Limit the speed of the generating stats task (#39645) 
master pr: #39644

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-02-17 16:06:17 +08:00
XuanYang-cn
8067113133
enhance: [cp25]Enable to observe write amplification (#39661) (#39743) 
pr: #39661

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-02-17 16:00:17 +08:00
Zhen Ye
56c1a8d462
fix: ignore growing option is lost at hibridsearch (#39900) 
issue: #39892
pr: #39799

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-02-17 15:30:15 +08:00
XuanYang-cn
ee25af4c9b
enhance: Add configs for compaction schedule (#39010) (#39511) 
pr: #39010

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-02-17 11:44:15 +08:00
Xianhui Lin
8c86e4e6ac
fix: [2.5]AlterCollection unable to modify ConsistencyLevel (#39902) 
fix: AlterCollection unable to modify ConsistencyLevel
issue: https://github.com/milvus-io/milvus/issues/39707
pr: https://github.com/milvus-io/milvus/pull/39708

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-02-17 10:00:15 +08:00
zhenshan.cao
9918e1008d
fix: Fix import failed due to 0 row num (#39887) (#39904) 
issue: https://github.com/milvus-io/milvus/issues/39885

pr: https://github.com/milvus-io/milvus/pull/39886

---------

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: yihao.dai <yihao.dai@zilliz.com>
 2025-02-17 01:36:15 +08:00
Xianhui Lin
f0964f769d
enhance: [2.5]Add json key inverted index in stats for optimization (#39876) 
Add json key inverted index in stats for optimization
issue: https://github.com/milvus-io/milvus/issues/36995
pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-02-16 20:12:15 +08:00
presburger
e372b1733c
enhance: [2.5]update raft to cuvs (#39221) 
pr: #39873
issue: #39883

Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
 2025-02-14 16:52:14 +08:00
cai.zhang
401794f405
fix: [2.5] Fix module for long type (#39802) 
issue: #39640 

master pr: #39722

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-02-14 15:18:21 +08:00
congqixia
5da9262f58
fix: [2.5] Add and use lifetime context for compaction trigger (#39857) (#39880) 
Cherry-pick from master
pr: #39857 
Related to #39856

This PR add lifetime bound context for compaction trigger and use it
instead of context.Background in case of rootcoord down and some grpc
call retry forever

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-14 14:18:14 +08:00
cai.zhang
418f971d2d
fix: [2.5] ReEnqueue L0 compaction task when preCheck failed (#39871) 
issue: #39868 

master pr: #39870

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-02-14 13:38:15 +08:00
smellthemoon
ba0edb8383
enhance: [2.5]return topks when search in restful v2 (#39839) 
pr: #39812

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2025-02-14 11:30:14 +08:00
congqixia
9407a3c9b1
fix: [2.5] Check collection released before target checks (#39843) 
Cherry-pick from master
pr: #39841 
Related to #39840

The target could be updated async in previous code. This PR make remove
collection from target observer block until all tasks related in
dispatchers are removed preventing the metrics being updated after
collection released.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-13 20:00:15 +08:00
Zhen Ye
f85bf59921
fix: rootcoord graceful stop failed and limit resource of ci (#39793) 
issue: #38399
pr: #39781

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-02-13 19:34:13 +08:00
zhagnlu
16ea4fcb90
Revert "fix:reset default auto index type for scalar (#37086)" (#39820) 
This reverts commit eeb67a38450732eb8ab3a4eb4aaac6340142662.
pr: #39819

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-02-13 17:36:14 +08:00
wei liu
a50249e0bb
enhance: [2.5] dump pprof info if component stop progress timeout (#39760) 
issue: #39735
pr: #39726

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-02-13 15:30:47 +08:00
congqixia
df28d2200d
fix: [2.5] Remove load field & schema column size check #39834 (#39835) 
Cherry-pick from master
pr: #39833
Related to #39788

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-13 12:42:53 +08:00
SimFG
cb1bf6d122
fix:[2.5] remove the mmap.enable param in the type param when creating index (#39806) 
Because when GetIndexParams is used, index params and type params are
concatenated, so when loading index, the mmap.enable parameter in type
params is also referenced.

- issue: #39801 
- pr: #39803

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-02-13 10:08:53 +08:00
cqy123456
d121ac3a7a
enhance: [2.5]intermin index support different index type and more data type(fp16/bf16) (#39180) 
issue: https://github.com/milvus-io/milvus/issues/27678
related: https://github.com/milvus-io/milvus/pull/39753
some raw data status will change:
Intermin index has raw data: 
<html xmlns:v="urn:schemas-microsoft-com:vml"
xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:x="urn:schemas-microsoft-com:office:excel"
xmlns="http://www.w3.org/TR/REC-html40">

<head>

<meta name=ProgId content=Excel.Sheet>
<meta name=Generator content="Microsoft Excel 15">
<link id=Main-File rel=Main-File

href="file:////Users/cqy/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip.htm">
<link rel=File-List

href="file:////Users/cqy/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip_filelist.xml">
<!--table
	{mso-displayed-decimal-separator:"\.";
	mso-displayed-thousand-separator:"\,";}
@page
	{margin:.75in .7in .75in .7in;
	mso-header-margin:.3in;
	mso-footer-margin:.3in;}
.font5
	{color:windowtext;
	font-size:9.0pt;
	font-weight:400;
	font-style:normal;
	text-decoration:none;
	font-family:等线;
	mso-generic-font-family:auto;
	mso-font-charset:134;}
tr
	{mso-height-source:auto;
	mso-ruby-visibility:none;}
col
	{mso-width-source:auto;
	mso-ruby-visibility:none;}
br
	{mso-data-placement:same-cell;}
td
	{padding-top:1px;
	padding-right:1px;
	padding-left:1px;
	mso-ignore:padding;
	color:black;
	font-size:12.0pt;
	font-weight:400;
	font-style:normal;
	text-decoration:none;
	font-family:等线;
	mso-generic-font-family:auto;
	mso-font-charset:134;
	mso-number-format:General;
	text-align:general;
	vertical-align:middle;
	border:none;
	mso-background-source:auto;
	mso-pattern:auto;
	mso-protection:locked visible;
	white-space:nowrap;
	mso-rotate:0;}
ruby
	{ruby-align:left;}
rt
	{color:windowtext;
	font-size:9.0pt;
	font-weight:400;
	font-style:normal;
	text-decoration:none;
	font-family:等线;
	mso-generic-font-family:auto;
	mso-font-charset:134;
	mso-char-type:none;
	display:none;}
-->
</head>

<body link="#0563C1" vlink="#954F72">


sparse vector | growing segment | sealed segment
-- | -- | --
BM25 | no | no
IP | yes | no
  |   |  
dense vector | growing segment | sealed segment
ivf flat cc | yes | yes
scann_dvr | no | no



</body>

</html>

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2025-02-13 09:56:47 +08:00
Buqian Zheng
0f4d3ef8b6
enhance[2.5]: move GetVector_EmptySparseVector ut to growing test (#39286) 
issue: https://github.com/milvus-io/milvus/issues/35853
pr: https://github.com/milvus-io/milvus/pull/39234

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2025-02-12 16:00:49 +08:00
smellthemoon
dd516a5716
fix: [2.5]not pass the indexname when drop properties (#39679) 
pr: #39678 
issue: #38967

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2025-02-12 15:52:48 +08:00