admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
22,009 Commits 137 Branches 188 Tags
Commit Graph

22009 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
congqixia
4ca1a7a76d
fix: Protect getSegmentIndexes with rlock (#40719) 
Related to #40718

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-18 16:40:14 +08:00
XuanYang-cn
015a8f7631
fix: L0 brings its own start pos when syncing (#40663) 
See also: #40388, #40207

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-03-18 15:56:16 +08:00
zhikunyao
4f93694404
enhance: mergify to check correct ut code (#40723) 
Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>
 2025-03-18 15:46:25 +08:00
zhuwenxing
69fc9aa271
test: add empty sparse in import test (#40682) 
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2025-03-18 15:16:12 +08:00
Ted Xu
be86d31ea3
feat: compaction to support add field (#40415) 
See: #39718

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-03-18 11:32:12 +08:00
congqixia
f4f81b7d0c
enhance: [skip e2e][mergify rule] Resolve go tests rule conflict (#40697) 
This PR:
- Add macro for only go test file changed
- Resolve conflict for only go test file changed and remove ci-passed
rule

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-17 19:28:11 +08:00
aoiasd
f943a8c78b
enhance: accesslog support fetch hybrid search expr and field (#40167) 
relate: https://github.com/milvus-io/milvus/issues/40166

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-03-17 18:08:10 +08:00
Ted Xu
d903b943c6
enhance: improve documentation on large page size support (#40695) 
See: #30843

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-03-17 17:14:13 +08:00
congqixia
94a859c028
enhance: Add buffer forwarder for stream delta loading (#40559) 
See also #40558
Related to #35303 & #38066 as well

This PR:
- Add `BufferedForward` to limit memory usage forwarding stream delete
- Add `UseLoad` flag to determine `Delete` shall use `segment.Delete` or
`segment.LoadDelta`
- Fix delegator accidentally use always true candidate while load
streaming delta

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-17 15:24:10 +08:00
congqixia
b3edb76516
enhance: [skip e2e][GoSDK] Add client example for common usage (#40643) 
Related to #31293

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-17 14:16:10 +08:00
zhagnlu
7ebe3d7038
enhance: refine chunk access logic and add some comment on data (#40618) 
#40367

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-03-16 22:20:08 +08:00
cai.zhang
123b6588b6
feat: Support get segment binlogs info with new interface GetSegmentsInfo (#40464) 
issue: #40341

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-16 21:04:07 +08:00
SimFG
5a87eb40ee
enhance: add GetService method to retrieve MilvusServiceClient (#40658) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-03-14 18:44:10 +08:00
groot
9fbfcda48e
fix: Fix a crash issue of bulkinsert (#40331) 
issue: https://github.com/milvus-io/milvus/issues/40291
pr: https://github.com/milvus-io/milvus/pull/40304

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2025-03-14 18:14:07 +08:00
Bingyi Sun
6249335859
fix: Catch invalid json pointer error (#40625) 
issue: #35528

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-14 16:56:08 +08:00
Bingyi Sun
d3adab15ac
fix: Build double index for all json numeric field (#40619) 
issue: #35528

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-14 16:52:11 +08:00
Bingyi Sun
8f66a0432d
fix: json path should be empty string if it's root (#40611) 
issue: #35528

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-14 16:50:16 +08:00
Bingyi Sun
a7cff3873b
fix: Verify json_cast_type (#40604) 
issue: https://github.com/milvus-io/milvus/issues/40420

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-14 16:48:26 +08:00
zhagnlu
8f85acc944
Revert "fix:reset default auto index type for scalar (#37086)" (#39819) 
This reverts commit eeb67a38450732eb8ab3a4eb4aaac63401426627.
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-03-14 16:16:11 +08:00
Bingyi Sun
8fbacf3583
fix: Null expr does not work for json field (#40456) 
issue: https://github.com/milvus-io/milvus/issues/40455

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-14 16:06:08 +08:00
Spade A
001fc992df
enhance: get doc ids by batch (#40608) 
issue: #40607

tantivy change: https://github.com/zilliztech/tantivy/pull/3

Benchmarks:
Test Envrioment: CPU 9900K
The data is insert by:
```
for i in 0..N {
    for j in 0..UNIQUE {
        let key = format!("hello{}", j);
        index_writer.add_string(&key, i * UNIQUE + j).unwrap();
    }
}
```
So the unique influences the locality of the matched docs.
The latency is the avg latency over 1000 repeate quries.
The result shows 22.5%-34.8% latency reduction.

![image](https://github.com/user-attachments/assets/dd8af75a-ddc3-445d-92df-50d354dd5645)

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-03-14 15:48:09 +08:00
cai.zhang
6dbe5d475e
enhance: Refine task meta with key lock (#40613) 
issue: #39101

2.5 pr: #40146 #40353

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-14 15:44:22 +08:00
SimFG
bf4fc6a8c6
feat: add DDLDB rate type and related quota configurations (#40651) 
- issue: #40650

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2025-03-14 15:38:09 +08:00
Zhen Ye
6982f007e2
enhance: add walimpls access mode options (#40591) 
issue: #40532

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-14 10:58:11 +08:00
Spade A
f36d1562bd
enhance: add metrics for random sample (#40634) 
issue: #39541

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-03-13 21:42:11 +08:00
yihao.dai
bab30a41bf
enhance: Improve import error msgs (#40567) 
issue: https://github.com/milvus-io/milvus/issues/40208

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-13 21:02:07 +08:00
Xiaofan
7210fc9780
feature: add a prefix on environment config (#40623) 
fix #40622

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2025-03-13 16:44:07 +08:00
Zhen Ye
f6fb4bc442
fix: backoff will retry infinitely after reaching max elapse (#40589) 
issue: #40588

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-13 16:24:06 +08:00
congqixia
e93d53ca28
doc: Add README for go milvusclient (#40630) 
Related to #31293

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-13 16:02:07 +08:00
ThreadDao
aaf8ddcec6
test: add cases for go client rbac APIs (#40352) 
issue: #33419
- add cases for go client rbac APIs
- add dbName params for grant/revoke v1 & v2 api
- add dbName in GrantItem

---------

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2025-03-13 14:28:08 +08:00
yihao.dai
b2a8694686
enhance: Merge IndexNode and DataNode (#40272) 
Merge DataNode and IndexNode into DataNode.

issue: https://github.com/milvus-io/milvus/issues/39115

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-13 14:26:11 +08:00
Ted Xu
df4285c9ef
enhance: API integration with storage v2 in clustering-compactions (#40133) 
See #39173

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-03-13 14:12:06 +08:00
wei liu
972e47043a
enhance: Remove CPU profile to prevent blocking stop progress (#40460) 
issue: #39735
related to #39726

- Removed CPU profile dump from util.go's pprof collection
- Avoid potential blocking in StopCPUProfile() during shutdown
- Maintain goroutine/heap/block/mutex profiles for diagnostics
- Ensure safe shutdown timeout handling without profile stalls

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-03-13 10:00:09 +08:00
Zhen Ye
5735c3ef19
fix: too many memory usage of streaming node (#40606) 
issue: #40592

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-13 07:10:07 +08:00
Xiaofan
fb48b3c7ac
fix: empty sparse row in importer (#40585) 
fix #40584

parquet bulk writer can not finish 0 dim sparse vector.

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2025-03-13 01:29:41 +08:00
Spade A
9f3bd55755
fix: avoid panic when field not exists in schema in query node (#40541) 
ref #40473

This PR is a workaround to avoid the panic described in the issue.

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-03-12 22:44:08 +08:00
jaime
c8a96377bb
enhance: move object storage client creation to pkg package (#40440) 
issue: #40439

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2025-03-12 20:38:07 +08:00
yihao.dai
27c7cbbc72
fix: Fix QueryNodeNumEntities metric (#40602) 
fix QueryNodeNumEntities metric introduced by pr
https://github.com/milvus-io/milvus/pull/39536

issue: https://github.com/milvus-io/milvus/issues/38162

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-12 19:08:05 +08:00
wei liu
0420dc1eb1
fix: use correct delete checkpoint to prevent premature data cleanup (#40366) 
issue: #40292
related to #39552

- Fix incorrect delete checkpoint usage in SyncDistribution
- Change checkpoint parameter from action.GetCheckpoint() to
action.GetDeleteCP() in SyncTargetVersion call
- This resolves the issue where delete buffer data was being cleaned
prematurely due to wrong checkpoint reference

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-03-12 15:00:08 +08:00
sthuang
c0e03b6ca4
fix: rbac star privilege return empty when listing policy (#40553) 
related: https://github.com/milvus-io/milvus/issues/40547

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2025-03-12 14:16:05 +08:00
Zhen Ye
96a010da7a
fix: msgstream adaptor may not gc quickly after comsumed (#40555) 
issue: #40540

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-12 14:00:04 +08:00
Spade A
95e2680a36
fix: ref collection for search/query (#40549) 
ref https://github.com/milvus-io/milvus/issues/40473

Collection is got without ref which means the collection could be
releases and the struct could be freed during the search which leads
schema inconsistency.

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-03-12 11:30:07 +08:00
Bingyi Sun
0698d04f7d
enhance: Upgrade simdjson version (#40538) 
issue: https://github.com/milvus-io/milvus/issues/40519
simdjson returns better error code in newer version.

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-11 15:04:05 +08:00
cai.zhang
e5f50076ec
enhance: Only check element type with not null array (#40446) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-03-11 14:58:07 +08:00
yihao.dai
a33c9372ce
fix: Fix channel not balance on datanodes (#40422) 
1. Prevent channels from being assigned to only one datanode during
datacoord startup.
2. Optimize the channel assignment policy by considering newly assigned
channels.
3. Make msgdispatcher manager lock-free.

issue: https://github.com/milvus-io/milvus/issues/40421,
https://github.com/milvus-io/milvus/issues/37630

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-11 14:56:16 +08:00
Bingyi Sun
a729bb84ba
enhance: add json path escape and replace $meta with dynamic field name (#40407) 
issue: #35528

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-03-11 14:00:05 +08:00
Zhen Ye
d9fe8f0dcf
fix: [skip e2e] wab unittest may failure (#40470) 
issue: #38399

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-03-11 11:34:06 +08:00
congqixia
3899b0f0d4
fix: Add duplicated type/index params check creating collection (#40462) 
Related to #40461

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-03-11 10:14:11 +08:00
aoiasd
3afc3a83ff
fix: rocksmq consumer register not concurrent safe (#39387) 
relate: https://github.com/milvus-io/milvus/issues/39336

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-03-11 00:56:04 +08:00
junjiejiangjjj
359e7efd8e
feat: Add function running monitoring (#40358) 
#35856 
#40004 
1. Optimize model verification logic
2. Add profiling code

Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com>
 2025-03-10 22:28:05 +08:00