zhagnlu
10a63b3f2e
enhance: add formatter for serveral types to remove compile warning ( #41094 )
...
#41091
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-04-07 11:54:24 +08:00
cai.zhang
a5be7cbce9
fix: Add the field index lock for getSegmentsIndexStates ( #40968 )
...
issue: #40966
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-04-07 11:38:24 +08:00
cai.zhang
05e25431d9
enhance: Deprecate disk params about indexing ( #41045 )
...
issue: #40863
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-04-07 11:36:34 +08:00
zhagnlu
0a378dc308
fix:fix format error for json ( #41026 )
...
#40963
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-04-07 10:22:22 +08:00
sthuang
a85e36bad2
fix: create collection task check failed after restart ( #40982 )
...
The fields and partitions information are stored and fetched with
different prefixes in the metadata. In the CreateCollectionTask, the
RootCoord checks the existing collection information against the
metadata. This check fails if the order of the fields or partitions info
differs, leading to an error after restarting Milvus. To resolve this,
we should use a map in the check logic to ensure consistency.
related: https://github.com/milvus-io/milvus/issues/40955
---------
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-04-05 06:58:22 +08:00
Zhen Ye
f18aa85083
enhance: vchannel fair balance policy for streaming ( #40959 )
...
issue: #40638
- Add `ChannelID` for streaming replica in future.
- Remove the pchannel count fair balance policy for streaming.
- Add Score based vchannel fair balance policy for streaming.
- Add pchannel stats manager to collect the stats of pchannel for
balancer.
- Add configuration and metrics for new balance policy
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-04 10:12:22 +08:00
Bingyi Sun
fcb03b5bd1
feat: add json null/exists expression ( #41004 )
...
issue: #35528
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-04-03 17:48:21 +08:00
zhuwenxing
276a8d36f8
test: add negative case when field name is reserved keywords ( #41082 )
...
/kind improvement
related: https://github.com/milvus-io/milvus/issues/40290
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-04-03 15:54:28 +08:00
zhuwenxing
4306ed4329
test: add import corner testcase ( #41079 )
...
/kind improvement
issue: https://github.com/milvus-io/milvus/issues/40291
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-04-03 15:52:29 +08:00
wei liu
bf8547578f
fix: Address manual balance and balance check issues ( #41037 )
...
issue: #37651
- Fix context propagation for manual balance segment task creation from
PR #38080 .
- Optimize stopping balance by preventing redundant checks per round,
addressing performance regression from PR #40297 .
- Decrease default `checkBalanceInterval` from 3000ms to 300ms.
- Correct minor log messages in `BalanceChecker`.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-04-03 15:48:27 +08:00
binbin
afb4621012
test: Add test cases for part of json path index ( #41016 )
...
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2025-04-03 13:02:27 +08:00
Zhen Ye
9f27d9af61
fix: segv if the LoadArrowReaderFromRemote run at the exception path ( #41069 )
...
issue: #41067
Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-03 02:54:21 +08:00
zhuwenxing
eb4884b5e7
test: add coo format for sparse vector import and some negative case ( #41040 )
...
/kind improvement
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-04-02 19:54:26 +08:00
Spade A
f552ec67dd
fix: support building tantivy index with low version(5) ( #40822 )
...
fix: https://github.com/milvus-io/milvus/issues/40823
To solve the problem in the issue, we have to support building tantivy
index with low version
for those query nodes with low tantivy version.
This PR does two things:
1. refactor codes for IndexWriterWrapper to make it concise
2. enable IndexWriterWrapper to build tantivy index by different tantivy
crate
---------
Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-04-02 18:46:20 +08:00
Chun Han
afa519b4c7
fix: array is null failed( #40686 ) ( #41027 )
...
related: #40686
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-04-02 18:20:22 +08:00
cai.zhang
902f6506ca
fix: Get all children deltalogs for segment to load ( #40956 )
...
issue: #40207
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-04-02 16:32:22 +08:00
Buqian Zheng
e1216829f7
enhance: weighted reranker to allow skip score normalization ( #40903 )
...
issue: https://github.com/milvus-io/milvus/issues/40836
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2025-04-02 16:24:23 +08:00
smellthemoon
cb1e86e17c
enhance: support add field ( #39800 )
...
after the pr merged, we can support to insert, upsert, build index,
query, search in the added field.
can only do the above operates in added field after add field request
complete, which is a sync operate.
compact will be supported in the next pr.
#39718
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-04-02 14:24:31 +08:00
congqixia
37cf9a0dc1
enhance: Use %v for missing id log ( #41036 )
...
`incomplete query result, missing id %!s(int64=348), len(searchIDs) =
10, len(queryIDs) = 9` error message format with error when missing id
is int64
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-02 11:54:23 +08:00
Spade A
216be1494b
fix: add log for object storage operation fail ( #40666 )
...
fix : #40665
---------
Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-04-02 01:26:21 +08:00
Zhen Ye
847b8c8fdc
fix: node version checker should use -dev before releasing ( #41039 )
...
issue: #40532
Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-01 20:04:20 +08:00
yihao.dai
b4cb8a4b13
enhance: Add UTF-8 string validation for import ( #40694 )
...
issue: https://github.com/milvus-io/milvus/issues/40684
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-04-01 19:04:21 +08:00
sre-ci-robot
599be2b88f
[automated] Bump milvus version to v2.5.8 ( #41029 )
...
Bump milvus version to v2.5.8
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-01 17:24:22 +08:00
sre-ci-robot
0e5ba195d5
[automated] Bump milvus version to v2.5.8 ( #41023 )
...
Bump milvus version to v2.5.8
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-01 15:08:28 +08:00
Zhen Ye
b03e60558a
enhance: add proxy and datanode checker when wal balance startup ( #40877 )
...
issue: #40532
- balance should enable only when there's no proxy and datanode which
version is lower than 2.6.0
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-01 11:24:22 +08:00
Zhen Ye
cef1d16454
fix: timetick interceptor panics when closing write ahead buffer ( #40970 )
...
issue: #40967
Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-01 10:44:22 +08:00
yihao.dai
5b78ef0a49
fix: Fix delete data loss due to duplicate binlogID ( #40960 )
...
With concurrenct L0 compaction
(https://github.com/milvus-io/milvus/pull/36816 ), delta logs might be
written to the same L1 segment, causing logID duplication when using the
incremental beginLogID. This PR removes the beginLogID mechanism and
instead passes a log ID range, where the number of IDs in the range
equals the number of compaction segment binlogs multiplied by an
expansion factor.
issue: https://github.com/milvus-io/milvus/issues/40207
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-04-01 10:36:22 +08:00
Bingyi Sun
ba5834adc9
fix: Fix passing wrong json_cast_type in e2e ( #41015 )
...
issue: #35528
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-04-01 10:28:21 +08:00
groot
5146b41aa4
fix: fix a 404 bug of WebUI when http.enablePprof is false ( #40951 )
...
issue: https://github.com/milvus-io/milvus/issues/40952
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2025-04-01 10:22:22 +08:00
cqy123456
6dc0f42830
fix:growing mmap data type crashed by nullable input ( #40994 )
...
issue: https://github.com/milvus-io/milvus/issues/40981
2.5 pr: https://github.com/milvus-io/milvus/pull/40980
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2025-03-31 20:32:19 +08:00
jaime
651afe5058
enhance: Aligning Etcd and MinIO versions in Docker Compose ( #40857 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2025-03-31 18:08:21 +08:00
Bingyi Sun
27ff3a42e7
enhance: Record simdjson error ( #41003 )
...
issue: #35528
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-03-31 17:56:19 +08:00
wei liu
c02892e9fb
enhance: Balance the collection with the largest row count first ( #40297 )
...
issue: #37651
this PR enable to balance the collection with largest row count first,
to avoid temporary migration of small table data to new nodes during
their onboarding, only to be moved out again after the large table
balance, which would cause unnecessary load.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-03-31 16:00:19 +08:00
Bingyi Sun
15ec7bae4d
fix: Fix using json index when iterative_filter is specified ( #40945 )
...
issue: #40934
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-03-31 15:26:19 +08:00
groot
aae3a3598e
enhance: bulkinsert supports parsing sparse vector form parquet struct ( #40927 )
...
issue: https://github.com/milvus-io/milvus/issues/40777
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2025-03-31 14:20:30 +08:00
yihao.dai
d8d1dcf076
fix: Fix dispatcher deregister and seek ( #40860 )
...
1. Fix deregister dispatcher concurrency. (Keep sure the same logic as
2.5 branch)
2. Fix seek if includeCurrentMsg. (This is only needed by CDC, so
there's no need to pick to 2.5 branch)
issue: issue: https://github.com/milvus-io/milvus/issues/39862
pr: https://github.com/milvus-io/milvus/pull/39863
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-03-31 11:38:19 +08:00
Xiaofan
3ecacc4493
Update README.md
2025-03-29 08:50:47 +08:00
Eli Boyarski
43cd9fa0f4
doc: README.md: has->have ( #40152 )
...
doc: "The vectors ... has" -> "The vectors ... have"
Signed-off-by: Eli Boyarski <eli@boyar.ski>
2025-03-29 02:12:21 +08:00
aoiasd
1cc88d7755
enhance: add utf8 check for all varchar field ( #40670 )
...
https://github.com/milvus-io/milvus/issues/40684
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-03-28 19:38:17 +08:00
Zhen Ye
528ce1a0fb
fix: getopts cannot find the non-args ulcgbZh ( #40984 )
...
issue: #39659
also see: #40813
Signed-off-by: chyezh <chyezh@outlook.com>
2025-03-28 15:47:29 +08:00
Buqian Zheng
7a056aff9d
enhance: avoid re-query if hybrid search requested only pk as output field ( #40842 )
...
proxy to always remove pk field from output field when forwarding
request to QN, and if user requested pk, fill it from IDs.
issue: https://github.com/milvus-io/milvus/issues/40833
---------
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2025-03-28 14:32:18 +08:00
Bingyi Sun
9676365af9
fix: Fix json index not equal filter ( #40647 )
...
issue: #35528
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-03-27 23:06:23 +08:00
congqixia
2e9e16d732
fix: [GoSDK] Set nullable according to fieldSchema for RowBased insert ( #40928 )
...
Related to #40737
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-03-27 17:50:20 +08:00
aoiasd
384d39ef5a
enhance: not build lindera features by default and support make milvus with tantivy features ( #40813 )
...
relate: https://github.com/milvus-io/milvus/issues/39659
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-03-27 14:08:22 +08:00
wei liu
06310a5994
fix: Fix L0 segment retention and improve delete buffer logging ( #40884 )
...
issue：#40207
related to https://github.com/milvus-io/milvus/pull/39552
- Correct comparison operator in UnRegister from > to >= to prevent
premature release of L0 segments with matching timestamps
- Add detailed logging for segment retention decisions during
unregistration
- Enhance error logging for buffer cleanup operations
- Add trace logs for segment registration/release lifecycle
- Include timestamp comparisons in debug logs for future troubleshooting
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-03-27 11:24:21 +08:00
zhagnlu
87e7d6d79f
fix:fix exception when do arith expr with using index ( #40794 )
...
#40783
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-03-27 11:10:21 +08:00
dependabot[bot]
b1e092673b
enhance: bump github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 ( #40923 )
...
Bumps [github.com/golang-jwt/jwt/v5](https://github.com/golang-jwt/jwt )
from 5.2.1 to 5.2.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/golang-jwt/jwt/releases ">github.com/golang-jwt/jwt/v5's
releases</a>.</em></p>
<blockquote>
<h2>v5.2.2</h2>
<h2>What's Changed</h2>
<ul>
<li>Fixed <a
href="https://github.com/golang-jwt/jwt/security/advisories/GHSA-mh63-6h87-95cp ">https://github.com/golang-jwt/jwt/security/advisories/GHSA-mh63-6h87-95cp </a>
by <a
href="https://github.com/mfridman "><code>@mfridman</code></a></li>
<li>Fixed some typos by <a
href="https://github.com/Ashikpaul "><code>@Ashikpaul</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/382 ">golang-jwt/jwt#382</a></li>
<li>build: add go1.22 to ci workflows by <a
href="https://github.com/mfridman "><code>@mfridman</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/383 ">golang-jwt/jwt#383</a></li>
<li>Bump golangci/golangci-lint-action from 4 to 5 by <a
href="https://github.com/dependabot "><code>@dependabot</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/387 ">golang-jwt/jwt#387</a></li>
<li>Bump golangci/golangci-lint-action from 5 to 6 by <a
href="https://github.com/dependabot "><code>@dependabot</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/389 ">golang-jwt/jwt#389</a></li>
<li>chore: bump ci tests to include go1.23 by <a
href="https://github.com/mfridman "><code>@mfridman</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/405 ">golang-jwt/jwt#405</a></li>
<li>Fix jwt -show by <a
href="https://github.com/AlexanderYastrebov "><code>@AlexanderYastrebov</code></a>
in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/406 ">golang-jwt/jwt#406</a></li>
<li>docs: typo by <a
href="https://github.com/kvii "><code>@kvii</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/407 ">golang-jwt/jwt#407</a></li>
<li>Update SECURITY.md by <a
href="https://github.com/oxisto "><code>@oxisto</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/416 ">golang-jwt/jwt#416</a></li>
<li>Update <code>jwt.Parse</code> example to use
<code>jwt.WithValidMethods</code> by <a
href="https://github.com/mattt "><code>@mattt</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/425 ">golang-jwt/jwt#425</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/Ashikpaul "><code>@Ashikpaul</code></a>
made their first contribution in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/382 ">golang-jwt/jwt#382</a></li>
<li><a href="https://github.com/kvii "><code>@kvii</code></a> made their
first contribution in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/407 ">golang-jwt/jwt#407</a></li>
<li><a href="https://github.com/mattt "><code>@mattt</code></a> made
their first contribution in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/425 ">golang-jwt/jwt#425</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/golang-jwt/jwt/compare/v5.2.1...v5.2.2 ">https://github.com/golang-jwt/jwt/compare/v5.2.1...v5.2.2 </a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="0951d18428c035977d9ehttps://redirect.github.com/golang-jwt/jwt/issues/425 ">#425</a>)</li>
<li><a
href="bc8bdca5cchttps://redirect.github.com/golang-jwt/jwt/issues/416 ">#416</a>)</li>
<li><a
href="5ec246c074https://redirect.github.com/golang-jwt/jwt/issues/407 ">#407</a>)</li>
<li><a
href="0123f1ad66https://redirect.github.com/golang-jwt/jwt/issues/406 ">#406</a>)</li>
<li><a
href="f961c72abdhttps://redirect.github.com/golang-jwt/jwt/issues/405 ">#405</a>)</li>
<li><a
href="62e504c281https://redirect.github.com/golang-jwt/jwt/issues/389 ">#389</a>)</li>
<li><a
href="1a56dcf532https://redirect.github.com/golang-jwt/jwt/issues/387 ">#387</a>)</li>
<li><a
href="c8043eab61https://redirect.github.com/golang-jwt/jwt/issues/383 ">#383</a>)</li>
<li><a
href="7c3f6dc563https://redirect.github.com/golang-jwt/jwt/issues/382 ">#382</a>)</li>
<li>See full diff in <a
href="https://github.com/golang-jwt/jwt/compare/v5.2.1...v5.2.2 ">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/milvus-io/milvus/network/alerts ).
</details>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2025-03-27 00:56:21 +08:00
Ted Xu
128efaa3e3
enhance: simplify size calculation in file writers ( #40808 )
...
See: #40342
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2025-03-26 20:04:22 +08:00
cai.zhang
e2e1493580
fix: exists should have a higher priority ( #40864 )
...
issue: #40837
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-03-26 20:02:34 +08:00
cai.zhang
4220e3355d
fix: Update milvus key words ( #40869 )
...
issue: #40290
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-03-26 19:44:32 +08:00
