admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-04 01:42:15 +08:00
Code Issues Packages Projects Releases Wiki Activity
22,710 Commits 138 Branches 188 Tags
Commit Graph

22710 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Bingyi Sun
ccfaa7bee8
fix: Fix the bug when offsets is nullptr in bulk api (#43127) 
issue: https://github.com/milvus-io/milvus/issues/42978

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-07-15 17:54:50 +08:00
zhikunyao
858115bfb8
test: update pytest base image (#43327) 
Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>
 2025-07-15 15:30:49 +08:00
sthuang
4f17640598
enhance: [StorageV2] clean up legacy flag (#43290) 
related: #39173

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2025-07-15 10:18:49 +08:00
Zhen Ye
ffc8c0730c
fix: wrong metric for sn timetick (#43312) 
issue: #43266

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-07-14 20:40:50 +08:00
Spade A
db91d85dbc
feat: more types of matches for ngram (#43081) 
Ref https://github.com/milvus-io/milvus/issues/42053

This PR enable ngram to support more kinds of matches such as prefix and
postfix match.

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-07-14 20:34:50 +08:00
tinswzy
0aeac94f8a
fix: no such file error was reported when reading an empty segment in local mode (#43284) 
#43185

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
 2025-07-14 19:30:49 +08:00
Spade A
e14a52721e
enhance: use stl sort with high cardinality for data_type int (#43305) 
fix: https://github.com/milvus-io/milvus/issues/43304

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-07-14 18:40:50 +08:00
congqixia
ae48f0e484
fix: [StorageV2] Handle missing column creating index (#43292) 
Related to #43250

Use FieldIDList to check missing field. If column is missing, return
empty resultset

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-07-14 17:06:50 +08:00
nico
19b2fc7132
test: update test cases (#43252) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2025-07-14 11:20:50 +08:00
wei liu
039564199c
fix: Prevent duplicate segment results in count queries (#43173) 
issue: #41570
Fix issue where growing and sealed segments could be searched
simultaneously, causing inflated count(*) results. This was caused by
logic introduced in PR #42009 that made sealed segments readable before
target version advancement.

Changes include:
- Fix conditional filtering logic in PinReadableSegments to prevent
sealed segments from becoming readable prematurely
- Use target version filter for full results (ratio=1.0) to ensure
sealed segments only become readable after target advancement
- Use query view segment list filter for partial results (ratio<1.0) to
maintain backward compatibility
- Simplify target version setting in AddDistributions to prevent
premature segment readability
- Add logging for redundant growing segments during sync
- Add comprehensive unit tests covering the duplicate segment scenario

This fix ensures count(*) queries return accurate results by preventing
the same segment from being counted in both growing and sealed states.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-07-14 11:10:49 +08:00
foxspy
8171a2a0b5
enhance: update knowhere version (#43246) 
issue: #42937

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
 2025-07-14 11:06:49 +08:00
Ted Xu
07894b37b6
enhance: returning collection metadata from cache (#42823) 
See #43187

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-07-14 10:54:50 +08:00
Bingyi Sun
21a96bc903
enhance: Save meta with txn limit (#43263) 
Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-07-14 10:32:49 +08:00
tinswzy
7da62698e0
enhance: improve WP parallel sync mechanism and fencing logic (#42892) 
related: #42595 
improve WP parallel sync mechanism and fencing logic; remove redundant
metrics and labels

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
 2025-07-13 23:04:49 +08:00
yihao.dai
1984be646c
fix: Fix storagev2 binlog import (#43221) 
issue: https://github.com/milvus-io/milvus/issues/43218

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-07-13 22:52:49 +08:00
Alexander Guzhva
a848c4a8c5
fix: fix incorrect bitset for the division comparison when the right is < 0 (#43179) 
issue: https://github.com/milvus-io/milvus/issues/42900
@sunby Unfortunately, it is not that easy to fix as it was thought in
#43177

Upd: also handles `Inf` and `NaN` values, and the division by zero case
for `fp32` and `fp64`

Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
 2025-07-11 19:04:49 +08:00
Zhen Ye
15a6631147
enhance: add quota limit based on sn consuming lag (#43105) 
issue: #42995

- The consuming lag at streaming node will be reported to coordinator.
- The consuming lag will trigger the write limit and deny by quota
center.
- Set the ttProtection by default.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-07-11 14:10:49 +08:00
cai.zhang
c54a04c71c
fix: L2 segments remain as L2 even after sort compaction (#43237) 
issue: #43186

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-07-11 11:30:48 +08:00
congqixia
002a325f0f
fix: [GoSDK] Slice valid data parsing nullable data (#43243) 
Related to #43003

When nq > 1, returning nullable data in search result will lead to
parsing error. This patch add slicing valid data logic to make nullable
parsing validation logic could work.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-07-11 10:32:48 +08:00
Zhen Ye
f598ca2b4e
fix: block at msgpack adaptor and wrong metrics (#43235) 
issue: #43018

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-07-11 10:14:49 +08:00
congqixia
5a9efb3f81
enhance: [StorageV2] Refine storage rw option usage & validation (#43175) 
Related to #39173

This PR:
- Make all datanode task passes storage config via storage config option
- Remove legacy comments, rootPath & bucketName parameters
- Fix clustering compaction option behavior
- Add validation logic for `rwOptions`
- Use correct storageType from storageConfig
- Add storage config in sync task

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-07-11 01:14:48 +08:00
congqixia
6bbed3b019
fix: [AddField] Add shared_lock for insert prevent race (#43229) 
Related to #43113

When schema change happens, insert shall not happen, otherwise:
- Data race may happen causing insertion failure
- Inconsistent data schema

This PR add shared_lock prevent this data race.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-07-10 21:26:48 +08:00
PjJinchen
a90694165b
feat: Supports tracing services that require header-based authentication. (#43211) 
issue: https://github.com/milvus-io/milvus/issues/43082

support tracing services that require header-based authentication.
for example: aliyun SLS, volcengine LogService etc...

[aliyun
SLS](https://help.aliyun.com/zh/sls/import-trace-data-from-golang-applications-to-log-service-by-using-opentelemetry-sdk-for-golang?spm=a2c4g.11186623.help-menu-search-28958.d_1#section-ktk-xxz-8om)

Add a headers config in trace config

```
trace:
  exporter: otlp
  sampleFraction: 1
  otlp:
    endpoint:  milvus-cn-beijing-pre.cn-beijing.log.aliyuncs.com:10010
    method:  # otlp export method, acceptable values: ["grpc", "http"],  using "grpc" by default
    secure: true
    headers:  # base64
  initTimeoutSeconds: 10
```

it is encoded as base64, raw data is json
```
{
    "x-sls-otel-project": "milvus-cn-beijing-pre",
    "x-sls-otel-instance-id": "milvus-cn-beijing-pre",
    "x-sls-otel-ak-id": "xxx",
    "x-sls-otel-ak-secret": "xxx"
}
```

[volcengine
tls](https://www.volcengine.com/docs/6470/812322#grpc-%E5%8D%8F%E8%AE%AE%E5%88%9D%E5%A7%8B%E5%8C%96%E7%A4%BA%E4%BE%8B)

Add a headers config in trace config

```
trace:
  exporter: otlp
  sampleFraction: 1
  otlp:
    endpoint:  xxx
    method:  # otlp export method, acceptable values: ["grpc", "http"],  using "grpc" by default
    secure: true
    headers:  # base64
  initTimeoutSeconds: 10
```

it is encoded as base64, raw data is json
```
{
    "x-tls-otel-region": "cn-beijing",
    "x-tls-otel-tracetopic": "milvus-cn-beijing-pre",
    "x-tls-otel-ak": "xxx",
    "x-tls-otel-sk": "xxx"
}
```

Signed-off-by: PjJinchen <6268414+pj1987111@users.noreply.github.com>
 2025-07-10 17:32:48 +08:00
zhuwenxing
85c8049296
test: add test cases for json modulo operation (#43191) 
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2025-07-10 17:08:48 +08:00
wei liu
b2597c6329
enhance: apply load config changes after QueryCoord restart (#43108) 
issue: #43107 
- Add checkLoadConfigChanges() to apply load config during startup
- Call config check in startQueryCoord() after restart
- Skip auto-updates for collections with user-specified replica numbers
- Add is_user_specified_replica_mode field to preserve user settings
- Add comprehensive unit tests with mockey

Ensures existing collections use latest cluster-level config after
restart.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-07-10 14:28:48 +08:00
cai.zhang
3ffd44f302
fix: Fix remaining issues with Datanode pooling and StorageV2 (#43147) 
issue: #43146

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-07-10 14:26:48 +08:00
yihao.dai
ee9a95189a
enhance: Print segments info after import done (#43200) 
issue: https://github.com/milvus-io/milvus/issues/42488

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-07-10 12:38:47 +08:00
Chun Han
07745439b5
fix: empty search groupby result causing crash(#43137) (#43214) 
related: #43137

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-07-10 12:04:48 +08:00
cai.zhang
47144429bf
fix: Fix regeneratePartitionStats failed after restore clusteringCompactionTask (#43205) 
issue: #43186

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-07-10 10:40:47 +08:00
Zhen Ye
490c5d5088
fix: lost message version after compatible message modification (#43217) 
issue: #43018

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-07-10 10:36:48 +08:00
Bingyi Sun
13f6e2130b
fix: Fix hybrid search return back empty set if one result is emtpy (#43209) 
issue: https://github.com/milvus-io/milvus/issues/43160

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-07-10 10:34:55 +08:00
congqixia
f027eea545
enhance: [AddField] Add log for segcore segment schema change (#43215) 
Related to #39178

This PR add logs for segment schema change operations.

Also fixes the nit comments from PR #42490

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-07-10 10:22:47 +08:00
congqixia
7f8c5c9bb8
fix: [GoSDK] Handle insufficient topks in search result (#43126) 
Related to #43118

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-07-10 01:30:48 +08:00
aoiasd
97b1c3ed96
enhance: add warn log if some segment's bm25 stats lacks (#43111) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-07-09 23:22:47 +08:00
cai.zhang
95e767611a
fix: Fix merge sort loss data when last row in a record is deleted (#43216) 
issue: #43207

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-07-09 22:18:48 +08:00
qixuan
1cd593b673
test:add go sdk case for add field (#43033) 
issue: #42126

Signed-off-by: qixuan <673771573@qq.com>
 2025-07-09 18:20:47 +08:00
zhagnlu
21d1fb2aa3
fix: fix move cursor bug for chunk segment with index (#43095) 
#42974

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-07-09 17:38:47 +08:00
cai.zhang
41d1c8d6b3
fix: Handle error for invalid function params and prevent panic (#43189) 
issue: #43188

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-07-09 12:46:46 +08:00
tinswzy
c4634d861e
fix: v2.6 WebUI metrics response schema change bug (#42957) 
#42919  
fix metrics response schema incompatibility with WebUI v2.6

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
 2025-07-08 22:56:47 +08:00
cai.zhang
6989e18599
enhance: Move sort stats task to sort compaction (#42562) 
issue: #42560

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-07-08 20:22:47 +08:00
zhuwenxing
f1e75085e8
test: add restful api case for rerank function (#42987) 
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2025-07-08 19:36:46 +08:00
aoiasd
54cc0b60f2
fix: dropped segment in excluded segment use wrong excluded ts (#43115) 
cause some excluded growing data insert again
relate: https://github.com/milvus-io/milvus/issues/43114

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-07-08 18:04:46 +08:00
Spade A
d41eec6f10
fix: void copy when getting json chunk (#43183) 
fix: https://github.com/milvus-io/milvus/issues/43182

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-07-08 15:28:46 +08:00
cai.zhang
8720feeb79
fix: Fix enqueuing when current batch is fully deleted (#43174) 
issue: #43045

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-07-08 12:20:46 +08:00
Zhen Ye
ed9aa1d4db
fix: limit GC concurrency as CPU number (#43165) 
issue: #42833

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-07-08 10:46:46 +08:00
Ted Xu
6153272d4b
enhance: disabling max entry limit by default (#43166) 
See: #43055

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-07-08 10:10:46 +08:00
yihao.dai
9cbd194c6b
fix: Prevent import from generating small binlogs (#43132) 
- Introduce dynamic buffer sizing to avoid generating small binlogs
during import
- Refactor import slot calculation based on CPU and memory constraints
- Implement dynamic pool sizing for sync manager and import tasks
according to CPU core count

issue: https://github.com/milvus-io/milvus/issues/43131

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-07-07 21:32:47 +08:00
sthuang
a0ae5bccc9
fix: [StorageV2] load growing segment get dim datatype check (#43168) 
related: https://github.com/milvus-io/milvus/issues/43072

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2025-07-07 15:46:47 +08:00
congqixia
ab818dcbca
fix: [StorageV2] Pass storage config for compaction rw (#43167) 
Related to #43148

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-07-07 15:32:46 +08:00
sthuang
276c52490d
fix: [StorageV2] missing arrow fs when building index (#43162) 
fix: https://github.com/milvus-io/milvus/issues/43150,
https://github.com/milvus-io/milvus/issues/43149

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2025-07-07 15:26:46 +08:00