admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2025-12-06 17:18:35 +08:00
Code Issues Packages Projects Releases Wiki Activity
22,019 Commits 133 Branches 183 Tags
Commit Graph

22019 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
sre-ci-robot
5c8d13632e
[automated] Bump milvus version to v2.5.7 (#40766) 
Bump milvus version to v2.5.7
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2025-03-19 18:00:24 +08:00
Bingyi Sun
5a6b4e56d5
fix: Fix tasks will panic if one of them throw an exception. (#40691) 
issue: https://github.com/milvus-io/milvus/issues/40690

the variable rcm will be dangling if a future throws an exception and
return.

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-19 16:52:09 +08:00
aoiasd
92bdf7a0c1
enhance: support run anayser return detaild token (#40458) 
relate: https://github.com/milvus-io/milvus/issues/39705

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-03-19 15:48:15 +08:00
zhagnlu
6c55db44f1
enhance: reorder sub expr for conjunct expr (#39872) 
two point:
 (1) reoder conjucts expr's subexpr, postpone heavy operations
sequence: int(column) -> index(column) -> string(column) -> light
conjuct
...... -> json(column) -> heavy conjuct -> two_column_compare
(2) support pre filter for expr execute, skip scan raw data that had
been skipped
     because of preceding expr result.

#39869

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-03-19 14:50:14 +08:00
Zhen Ye
8db708f67d
enhance: enable memory prof based on jemalloc (#40731) 
issue: #40730

also see: https://github.com/milvus-io/cgosymbolizer/pull/2

After these PR, at linux:

- the milvus will always enable jemalloc by default.
- jemalloc will always compiled with --enable-prof options.
- all image will always enable the jemalloc prof by default.
- a pprof http service for jemalloc at `/debug/jemalloc/` will be
registered into restful.
- `jeprof` can remote profile the memory of milvus.

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-19 14:46:18 +08:00
yanliang567
cf223bae7b
test: Split test_search and refactor on test class to share collections (#40677) 
issue: #40698

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2025-03-19 14:20:15 +08:00
Zhen Ye
b119ac5d30
enhance: add wal access mode options (#40617) 
issue: #40532

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-19 14:02:11 +08:00
Buqian Zheng
03b63bf982
fix: use NewInsertDataWithFunctionOutputField when importing binlog file (#40741) 
issue: https://github.com/milvus-io/milvus/issues/40740

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2025-03-19 10:50:14 +08:00
aoiasd
e5c12421f7
enhance: remain raw token for audit log (#40637) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-03-18 18:02:20 +08:00
Buqian Zheng
c12abf4e2a
enhance: improve sparse query nnz metric (#40713) 
add query type and field id label; add metric for hybrid search

issue: https://github.com/milvus-io/milvus/issues/35853

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2025-03-18 17:20:16 +08:00
congqixia
4ca1a7a76d
fix: Protect getSegmentIndexes with rlock (#40719) 
Related to #40718

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-18 16:40:14 +08:00
XuanYang-cn
015a8f7631
fix: L0 brings its own start pos when syncing (#40663) 
See also: #40388, #40207

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-03-18 15:56:16 +08:00
zhikunyao
4f93694404
enhance: mergify to check correct ut code (#40723) 
Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>
 2025-03-18 15:46:25 +08:00
zhuwenxing
69fc9aa271
test: add empty sparse in import test (#40682) 
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2025-03-18 15:16:12 +08:00
Ted Xu
be86d31ea3
feat: compaction to support add field (#40415) 
See: #39718

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-03-18 11:32:12 +08:00
congqixia
f4f81b7d0c
enhance: [skip e2e][mergify rule] Resolve go tests rule conflict (#40697) 
This PR:
- Add macro for only go test file changed
- Resolve conflict for only go test file changed and remove ci-passed
rule

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-17 19:28:11 +08:00
aoiasd
f943a8c78b
enhance: accesslog support fetch hybrid search expr and field (#40167) 
relate: https://github.com/milvus-io/milvus/issues/40166

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-03-17 18:08:10 +08:00
Ted Xu
d903b943c6
enhance: improve documentation on large page size support (#40695) 
See: #30843

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-03-17 17:14:13 +08:00
congqixia
94a859c028
enhance: Add buffer forwarder for stream delta loading (#40559) 
See also #40558
Related to #35303 & #38066 as well

This PR:
- Add `BufferedForward` to limit memory usage forwarding stream delete
- Add `UseLoad` flag to determine `Delete` shall use `segment.Delete` or
`segment.LoadDelta`
- Fix delegator accidentally use always true candidate while load
streaming delta

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-17 15:24:10 +08:00
congqixia
b3edb76516
enhance: [skip e2e][GoSDK] Add client example for common usage (#40643) 
Related to #31293

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-17 14:16:10 +08:00
zhagnlu
7ebe3d7038
enhance: refine chunk access logic and add some comment on data (#40618) 
#40367

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-03-16 22:20:08 +08:00
cai.zhang
123b6588b6
feat: Support get segment binlogs info with new interface GetSegmentsInfo (#40464) 
issue: #40341

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-16 21:04:07 +08:00
SimFG
5a87eb40ee
enhance: add GetService method to retrieve MilvusServiceClient (#40658) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-03-14 18:44:10 +08:00
groot
9fbfcda48e
fix: Fix a crash issue of bulkinsert (#40331) 
issue: https://github.com/milvus-io/milvus/issues/40291
pr: https://github.com/milvus-io/milvus/pull/40304

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2025-03-14 18:14:07 +08:00
Bingyi Sun
6249335859
fix: Catch invalid json pointer error (#40625) 
issue: #35528

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-14 16:56:08 +08:00
Bingyi Sun
d3adab15ac
fix: Build double index for all json numeric field (#40619) 
issue: #35528

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-14 16:52:11 +08:00
Bingyi Sun
8f66a0432d
fix: json path should be empty string if it's root (#40611) 
issue: #35528

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-14 16:50:16 +08:00
Bingyi Sun
a7cff3873b
fix: Verify json_cast_type (#40604) 
issue: https://github.com/milvus-io/milvus/issues/40420

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-14 16:48:26 +08:00
zhagnlu
8f85acc944
Revert "fix:reset default auto index type for scalar (#37086)" (#39819) 
This reverts commit eeb67a38450732eb8ab3a4eb4aaac63401426627.
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-03-14 16:16:11 +08:00
Bingyi Sun
8fbacf3583
fix: Null expr does not work for json field (#40456) 
issue: https://github.com/milvus-io/milvus/issues/40455

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-14 16:06:08 +08:00
Spade A
001fc992df
enhance: get doc ids by batch (#40608) 
issue: #40607

tantivy change: https://github.com/zilliztech/tantivy/pull/3

Benchmarks:
Test Envrioment: CPU 9900K
The data is insert by:
```
for i in 0..N {
    for j in 0..UNIQUE {
        let key = format!("hello{}", j);
        index_writer.add_string(&key, i * UNIQUE + j).unwrap();
    }
}
```
So the unique influences the locality of the matched docs.
The latency is the avg latency over 1000 repeate quries.
The result shows 22.5%-34.8% latency reduction.

![image](https://github.com/user-attachments/assets/dd8af75a-ddc3-445d-92df-50d354dd5645)

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-03-14 15:48:09 +08:00
cai.zhang
6dbe5d475e
enhance: Refine task meta with key lock (#40613) 
issue: #39101

2.5 pr: #40146 #40353

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-14 15:44:22 +08:00
SimFG
bf4fc6a8c6
feat: add DDLDB rate type and related quota configurations (#40651) 
- issue: #40650

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-03-14 15:38:09 +08:00
Zhen Ye
6982f007e2
enhance: add walimpls access mode options (#40591) 
issue: #40532

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-14 10:58:11 +08:00
Spade A
f36d1562bd
enhance: add metrics for random sample (#40634) 
issue: #39541

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-03-13 21:42:11 +08:00
yihao.dai
bab30a41bf
enhance: Improve import error msgs (#40567) 
issue: https://github.com/milvus-io/milvus/issues/40208

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-13 21:02:07 +08:00
Xiaofan
7210fc9780
feature: add a prefix on environment config (#40623) 
fix #40622

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2025-03-13 16:44:07 +08:00
Zhen Ye
f6fb4bc442
fix: backoff will retry infinitely after reaching max elapse (#40589) 
issue: #40588

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-13 16:24:06 +08:00
congqixia
e93d53ca28
doc: Add README for go milvusclient (#40630) 
Related to #31293

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-13 16:02:07 +08:00
ThreadDao
aaf8ddcec6
test: add cases for go client rbac APIs (#40352) 
issue: #33419
- add cases for go client rbac APIs
- add dbName params for grant/revoke v1 & v2 api
- add dbName in GrantItem

---------

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2025-03-13 14:28:08 +08:00
yihao.dai
b2a8694686
enhance: Merge IndexNode and DataNode (#40272) 
Merge DataNode and IndexNode into DataNode.

issue: https://github.com/milvus-io/milvus/issues/39115

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-13 14:26:11 +08:00
Ted Xu
df4285c9ef
enhance: API integration with storage v2 in clustering-compactions (#40133) 
See #39173

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-03-13 14:12:06 +08:00
wei liu
972e47043a
enhance: Remove CPU profile to prevent blocking stop progress (#40460) 
issue: #39735
related to #39726

- Removed CPU profile dump from util.go's pprof collection
- Avoid potential blocking in StopCPUProfile() during shutdown
- Maintain goroutine/heap/block/mutex profiles for diagnostics
- Ensure safe shutdown timeout handling without profile stalls

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-03-13 10:00:09 +08:00
Zhen Ye
5735c3ef19
fix: too many memory usage of streaming node (#40606) 
issue: #40592

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-13 07:10:07 +08:00
Xiaofan
fb48b3c7ac
fix: empty sparse row in importer (#40585) 
fix #40584

parquet bulk writer can not finish 0 dim sparse vector.

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2025-03-13 01:29:41 +08:00
Spade A
9f3bd55755
fix: avoid panic when field not exists in schema in query node (#40541) 
ref #40473

This PR is a workaround to avoid the panic described in the issue.

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-03-12 22:44:08 +08:00
jaime
c8a96377bb
enhance: move object storage client creation to pkg package (#40440) 
issue: #40439

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2025-03-12 20:38:07 +08:00
yihao.dai
27c7cbbc72
fix: Fix QueryNodeNumEntities metric (#40602) 
fix QueryNodeNumEntities metric introduced by pr
https://github.com/milvus-io/milvus/pull/39536

issue: https://github.com/milvus-io/milvus/issues/38162

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-12 19:08:05 +08:00
wei liu
0420dc1eb1
fix: use correct delete checkpoint to prevent premature data cleanup (#40366) 
issue: #40292
related to #39552

- Fix incorrect delete checkpoint usage in SyncDistribution
- Change checkpoint parameter from action.GetCheckpoint() to
action.GetDeleteCP() in SyncTargetVersion call
- This resolves the issue where delete buffer data was being cleaned
prematurely due to wrong checkpoint reference

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-03-12 15:00:08 +08:00
sthuang
c0e03b6ca4
fix: rbac star privilege return empty when listing policy (#40553) 
related: https://github.com/milvus-io/milvus/issues/40547

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2025-03-12 14:16:05 +08:00