Zhen Ye
f62a2e281e
enhance: add MustAsMutable... function, cooldown notifier, hardware listener ( #41637 )
...
issue: #41544
Signed-off-by: chyezh <chyezh@outlook.com>
2025-05-06 01:16:52 +08:00
Zhen Ye
de8f0af20d
enhance: use dispatcher at delegator when enable streaming ( #41266 )
...
issue: #38399
- add an adaptor type to adapt the streaming service client and
msgstream client to reuse the msgdispatcher.
Signed-off-by: chyezh <chyezh@outlook.com>
2025-05-06 01:12:53 +08:00
SimFG
623c728dac
enhance: add timeout for message reception in mqMsgStream ( #41602 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-05-06 00:50:52 +08:00
SimFG
3bd6268d3c
enhance: Implement OperatePrivilegeV2 message handling and unmarshal support ( #41355 )
...
- issue: #41353
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-04-30 12:02:53 +08:00
sthuang
6c377b6e86
feat: Storage v2 index and stats raw data ( #41534 )
...
related: #39173
---------
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-04-30 08:48:54 +08:00
XuanYang-cn
6b08eff4fe
enhance: Expose configs Manager ( #41586 )
...
See also: #41585
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-04-29 17:12:57 +08:00
foxspy
1d99f8bd67
enhance: add force rebuild index configuration ( #41473 )
...
issue: #41431
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2025-04-29 16:20:56 +08:00
XuanYang-cn
b6f3fd0de1
enhance: Update to the latest CipherPlugin API ( #41599 )
...
See also: #41585
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-04-29 14:50:50 +08:00
tinswzy
b36ed03141
enhance: Add Truncate Interface to WALImpls for Log Retention Control ( #41517 )
...
#41465 Add Truncate Interface to WALImpls for Proactive Log Retention
Management
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-04-29 14:36:49 +08:00
Zhen Ye
9cb5271027
enhance: remove support of embeded nats mq ( #41565 )
...
issue: #41564
Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-28 23:06:49 +08:00
Zhen Ye
dfbb02a5f7
enhance: make streaming message as a log field for easier coding ( #41545 )
...
issue: #41544
- implement message can be logged as a field by zap.
- fix too many slow log for woodpecker.
Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-28 14:38:42 +08:00
Buqian Zheng
3de904c7ea
feat: add cachinglayer to sealed segment ( #41436 )
...
issue: https://github.com/milvus-io/milvus/issues/41435
---------
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2025-04-28 10:52:40 +08:00
cai.zhang
640f526301
fix: Update current scalar index version to compatible tantivy different versions ( #41141 )
...
issue: #40823
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-04-27 20:44:39 +08:00
Zhen Ye
6a15790799
enhance: add interface for message and fix write ahead buffer ( #41470 )
...
issue: #41439
- add IsPersisted and VChannel interface for message
- add WithNotPersisted() for message builder
- fix the persisted time tick lost at write ahead buffer
Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-27 10:24:38 +08:00
congqixia
b5443ddbd0
enhance: [AddField] Reopen loaded segments after AddField ( #41529 )
...
Related to #39718
This PR:
- Add reopen logic for growing & sealed segments
- Lazy reopen when schema version increases
- Add FinishLoad api for loading progress
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-26 08:48:39 +08:00
Sungyun Hur
52f255c958
fix: mark CacheMemoryLimit as deprecated in configuration ( #41451 )
...
`CacheMemoryLimit` configuration is never used in current code. Thus
mark it as deprecated
---------
Signed-off-by: lambert <lambert@daangn.com>
2025-04-24 20:38:40 +08:00
Julien Salleyron
9de0c84576
fix: Allow to compile on windows ( #41448 )
...
This PR fixes #41384 .
When using milvus client and compile on windows, the compilation failed
with the undefined RSS error.
On windows, the way to get memory used is the same as on darwin.
Signed-off-by: Julien Salleyron <julien.salleyron@gmail.com>
2025-04-24 20:34:38 +08:00
junjiejiangjjj
e56adc121b
enhance: refactor embedding credentials manager ( #41442 )
...
https://github.com/milvus-io/milvus/issues/35856
Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com>
2025-04-24 14:34:38 +08:00
Zhen Ye
5fd47c3c89
fix: mockery too unavailable after upgrade golang version ( #41481 )
...
issue: #41291
pr: #41318
Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-24 10:46:43 +08:00
aoiasd
f52c2909c4
feat: support multi analyzer for bm25 function ( #41351 )
...
relate: https://github.com/milvus-io/milvus/issues/41213
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-04-23 18:22:38 +08:00
SimFG
91d40fa558
fix: Update logging context and upgrade dependencies ( #41318 )
...
- issue: #41291
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-23 10:52:38 +08:00
congqixia
b36c88f3c8
enhance: [AddField] Broadcast schema change via WAL ( #41373 )
...
Related to #39718
Add Broadcast logic for collection schema change and notifies:
- Streamnode - Delegator
- Streamnode - Flush component
- QueryNodes via grpc
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-22 16:28:37 +08:00
Chun Han
016920b023
fix: solve incompitable problem for none-encoding index( #40838 ) ( #41369 )
...
related: #40838
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-04-20 22:56:44 +08:00
tinswzy
6fa68c1f16
enhance: Support Woodpecker as a WAL storage option for Milvus ( #41095 )
...
#40916 Support Woodpecker as a WAL storage option for Milvus
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-04-20 22:22:42 +08:00
Ted Xu
93788255eb
fix: errorous deadlock report in unittests ( #41350 )
...
See #41349
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2025-04-17 13:16:31 +08:00
congqixia
52c7d62012
enhance: Adapt hyphen in grpc metadata header ( #41358 )
...
Related to #41357
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-17 10:52:33 +08:00
Sungyun Hur
fbd595eae1
fix: correct a minor typo in log message of kafka client ( #41216 )
...
fixes a minor typo that notify users that some special kafka configs are
overwritten in Milvus.
Signed-off-by: lambert <lambert@daangn.com>
2025-04-16 21:14:20 +08:00
Xianhui Lin
deb610e5d3
fix: update MixCoord registration in MilvusRoles ( #41337 )
...
enhance: update MixCoord registration in MilvusRoles
The `runMixCoord` function in `MilvusRoles` was updated to use the
`RegisterMixCoord` function from the `rootcoord_metrics` package instead
of `RegisterRootCoord`. This change aligns with the recent modifications
made to the `rootcoord_metrics` package.
issue:https://github.com/milvus-io/milvus/issues/41338
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-04-16 19:49:54 +08:00
yihao.dai
dccfc69660
enhance: Get compaction params from request ( #41125 )
...
Make DataNode use compaction parameters from request instead of
configuration.
issue: https://github.com/milvus-io/milvus/issues/41123
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-04-15 10:28:53 +08:00
Zhen Ye
1f2077b68f
fix: remove dead config queryNode.grouping.enabled ( #41244 )
...
issue: #41243
Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-14 14:36:32 +08:00
Bingyi Sun
bf617115ca
enhance: Remove single chunk segment related codes ( #39249 )
...
https://github.com/milvus-io/milvus/issues/39112
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-04-11 18:56:29 +08:00
Xianhui Lin
f9febe3bae
enhance: Merge RootCoord, DataCoord And QueryCoord into MixCoord ( #41006 )
...
Merge RootCoord, DataCoord And QueryCoord into MixCoord
Make Session into one
issue : https://github.com/milvus-io/milvus/issues/37764
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-04-11 16:36:30 +08:00
Xianhui Lin
144911aec6
fix: CreateStatsRequest change storage_version to 25 consistent with 2.5 ( #41217 )
...
fix: CreateStatsRequest change storage_version to 25 consistent with 2.5
relate-pr:https://github.com/milvus-io/milvus/pull/38039
issue: https://github.com/milvus-io/milvus/issues/36995
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-04-11 11:16:43 +08:00
congqixia
486141cc12
enhance: [Restful] Make default timeout configurable ( #41211 )
...
The restful API default timeout was hard-coded. This PR make this
timeout value configurable via paramtable.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-10 16:14:28 +08:00
Xianhui Lin
3bc24c264f
enhance: Add json key inverted index in stats for optimization ( #38039 )
...
Add json key inverted index in stats for optimization
https://github.com/milvus-io/milvus/issues/36995
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-04-10 15:20:28 +08:00
SimFG
a308d2c886
fix: get replicate channel position ( #41188 )
...
- issue: #41187
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-04-10 15:14:28 +08:00
cai.zhang
8a77fb9cdc
enhance: Support slot for index task and stats task ( #39084 )
...
issue: #39101
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-04-08 20:46:25 +08:00
congqixia
1afa7433f0
enhance: Rectify client_request_id logic ( #41089 )
...
The traceID is not initialized by client_request_id in context. If the
client sent valid traceID, milvus log will print two different traceID
which is wierd.
This PR add the logic to tray parsing incoming `client_request_id` into
traceID. If it works just use it the request traceID, otherwise set it
to a different field named `client_request_id`.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-08 10:32:26 +08:00
Ted Xu
1bcea2a775
fix: assigning the correct storage version in sync and index tasks ( #41093 )
...
See #39663 #40667
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2025-04-08 10:14:25 +08:00
cai.zhang
05e25431d9
enhance: Deprecate disk params about indexing ( #41045 )
...
issue: #40863
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-04-07 11:36:34 +08:00
Zhen Ye
f18aa85083
enhance: vchannel fair balance policy for streaming ( #40959 )
...
issue: #40638
- Add `ChannelID` for streaming replica in future.
- Remove the pchannel count fair balance policy for streaming.
- Add Score based vchannel fair balance policy for streaming.
- Add pchannel stats manager to collect the stats of pchannel for
balancer.
- Add configuration and metrics for new balance policy
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-04 10:12:22 +08:00
wei liu
bf8547578f
fix: Address manual balance and balance check issues ( #41037 )
...
issue: #37651
- Fix context propagation for manual balance segment task creation from
PR #38080 .
- Optimize stopping balance by preventing redundant checks per round,
addressing performance regression from PR #40297 .
- Decrease default `checkBalanceInterval` from 3000ms to 300ms.
- Correct minor log messages in `BalanceChecker`.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-04-03 15:48:27 +08:00
smellthemoon
cb1e86e17c
enhance: support add field ( #39800 )
...
after the pr merged, we can support to insert, upsert, build index,
query, search in the added field.
can only do the above operates in added field after add field request
complete, which is a sync operate.
compact will be supported in the next pr.
#39718
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-04-02 14:24:31 +08:00
yihao.dai
b4cb8a4b13
enhance: Add UTF-8 string validation for import ( #40694 )
...
issue: https://github.com/milvus-io/milvus/issues/40684
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-04-01 19:04:21 +08:00
Zhen Ye
b03e60558a
enhance: add proxy and datanode checker when wal balance startup ( #40877 )
...
issue: #40532
- balance should enable only when there's no proxy and datanode which
version is lower than 2.6.0
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-01 11:24:22 +08:00
yihao.dai
5b78ef0a49
fix: Fix delete data loss due to duplicate binlogID ( #40960 )
...
With concurrenct L0 compaction
(https://github.com/milvus-io/milvus/pull/36816 ), delta logs might be
written to the same L1 segment, causing logID duplication when using the
incremental beginLogID. This PR removes the beginLogID mechanism and
instead passes a log ID range, where the number of IDs in the range
equals the number of compaction segment binlogs multiplied by an
expansion factor.
issue: https://github.com/milvus-io/milvus/issues/40207
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-04-01 10:36:22 +08:00
wei liu
c02892e9fb
enhance: Balance the collection with the largest row count first ( #40297 )
...
issue: #37651
this PR enable to balance the collection with largest row count first,
to avoid temporary migration of small table data to new nodes during
their onboarding, only to be moved out again after the large table
balance, which would cause unnecessary load.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-03-31 16:00:19 +08:00
yihao.dai
d8d1dcf076
fix: Fix dispatcher deregister and seek ( #40860 )
...
1. Fix deregister dispatcher concurrency. (Keep sure the same logic as
2.5 branch)
2. Fix seek if includeCurrentMsg. (This is only needed by CDC, so
there's no need to pick to 2.5 branch)
issue: issue: https://github.com/milvus-io/milvus/issues/39862
pr: https://github.com/milvus-io/milvus/pull/39863
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-03-31 11:38:19 +08:00
dependabot[bot]
b1e092673b
enhance: bump github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 ( #40923 )
...
Bumps [github.com/golang-jwt/jwt/v5](https://github.com/golang-jwt/jwt )
from 5.2.1 to 5.2.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/golang-jwt/jwt/releases ">github.com/golang-jwt/jwt/v5's
releases</a>.</em></p>
<blockquote>
<h2>v5.2.2</h2>
<h2>What's Changed</h2>
<ul>
<li>Fixed <a
href="https://github.com/golang-jwt/jwt/security/advisories/GHSA-mh63-6h87-95cp ">https://github.com/golang-jwt/jwt/security/advisories/GHSA-mh63-6h87-95cp </a>
by <a
href="https://github.com/mfridman "><code>@mfridman</code></a></li>
<li>Fixed some typos by <a
href="https://github.com/Ashikpaul "><code>@Ashikpaul</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/382 ">golang-jwt/jwt#382</a></li>
<li>build: add go1.22 to ci workflows by <a
href="https://github.com/mfridman "><code>@mfridman</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/383 ">golang-jwt/jwt#383</a></li>
<li>Bump golangci/golangci-lint-action from 4 to 5 by <a
href="https://github.com/dependabot "><code>@dependabot</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/387 ">golang-jwt/jwt#387</a></li>
<li>Bump golangci/golangci-lint-action from 5 to 6 by <a
href="https://github.com/dependabot "><code>@dependabot</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/389 ">golang-jwt/jwt#389</a></li>
<li>chore: bump ci tests to include go1.23 by <a
href="https://github.com/mfridman "><code>@mfridman</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/405 ">golang-jwt/jwt#405</a></li>
<li>Fix jwt -show by <a
href="https://github.com/AlexanderYastrebov "><code>@AlexanderYastrebov</code></a>
in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/406 ">golang-jwt/jwt#406</a></li>
<li>docs: typo by <a
href="https://github.com/kvii "><code>@kvii</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/407 ">golang-jwt/jwt#407</a></li>
<li>Update SECURITY.md by <a
href="https://github.com/oxisto "><code>@oxisto</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/416 ">golang-jwt/jwt#416</a></li>
<li>Update <code>jwt.Parse</code> example to use
<code>jwt.WithValidMethods</code> by <a
href="https://github.com/mattt "><code>@mattt</code></a> in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/425 ">golang-jwt/jwt#425</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/Ashikpaul "><code>@Ashikpaul</code></a>
made their first contribution in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/382 ">golang-jwt/jwt#382</a></li>
<li><a href="https://github.com/kvii "><code>@kvii</code></a> made their
first contribution in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/407 ">golang-jwt/jwt#407</a></li>
<li><a href="https://github.com/mattt "><code>@mattt</code></a> made
their first contribution in <a
href="https://redirect.github.com/golang-jwt/jwt/pull/425 ">golang-jwt/jwt#425</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/golang-jwt/jwt/compare/v5.2.1...v5.2.2 ">https://github.com/golang-jwt/jwt/compare/v5.2.1...v5.2.2 </a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="0951d18428c035977d9ehttps://redirect.github.com/golang-jwt/jwt/issues/425 ">#425</a>)</li>
<li><a
href="bc8bdca5cchttps://redirect.github.com/golang-jwt/jwt/issues/416 ">#416</a>)</li>
<li><a
href="5ec246c074https://redirect.github.com/golang-jwt/jwt/issues/407 ">#407</a>)</li>
<li><a
href="0123f1ad66https://redirect.github.com/golang-jwt/jwt/issues/406 ">#406</a>)</li>
<li><a
href="f961c72abdhttps://redirect.github.com/golang-jwt/jwt/issues/405 ">#405</a>)</li>
<li><a
href="62e504c281https://redirect.github.com/golang-jwt/jwt/issues/389 ">#389</a>)</li>
<li><a
href="1a56dcf532https://redirect.github.com/golang-jwt/jwt/issues/387 ">#387</a>)</li>
<li><a
href="c8043eab61https://redirect.github.com/golang-jwt/jwt/issues/383 ">#383</a>)</li>
<li><a
href="7c3f6dc563https://redirect.github.com/golang-jwt/jwt/issues/382 ">#382</a>)</li>
<li>See full diff in <a
href="https://github.com/golang-jwt/jwt/compare/v5.2.1...v5.2.2 ">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/milvus-io/milvus/network/alerts ).
</details>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2025-03-27 00:56:21 +08:00
cai.zhang
4220e3355d
fix: Update milvus key words ( #40869 )
...
issue: #40290
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-03-26 19:44:32 +08:00
