437 Commits

Author SHA1 Message Date
tinswzy
754ca58469
enhance: improve WP parameters according to performance testing (#42666) 2025-06-13 20:33:41 +08:00
Zhen Ye
16d6af85c3
fix: panic when use kafka/pulsar/wp for standalone (#41683)
issue: #41682

- There's a bug in rocksmq cleanup, #41565 remove the support of nats.

Signed-off-by: chyezh <chyezh@outlook.com>
2025-05-09 11:16:54 +08:00
Zhen Ye
f1cba24b94
fix: stop worknode concurrently (#41608)
issue: #41445

- make multiple node stop concurrently (otherwise streamingnode stop
will be blocked by querynode).
- change vchannel count updating when collection is dropping.

Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-29 23:16:53 +08:00
Zhen Ye
9cb5271027
enhance: remove support of embeded nats mq (#41565)
issue: #41564

Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-28 23:06:49 +08:00
junjiejiangjjj
e56adc121b
enhance: refactor embedding credentials manager (#41442)
https://github.com/milvus-io/milvus/issues/35856

Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com>
2025-04-24 14:34:38 +08:00
SimFG
91d40fa558
fix: Update logging context and upgrade dependencies (#41318)
- issue: #41291

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-23 10:52:38 +08:00
tinswzy
6fa68c1f16
enhance: Support Woodpecker as a WAL storage option for Milvus (#41095)
#40916 Support Woodpecker as a WAL storage option for Milvus

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-04-20 22:22:42 +08:00
Xianhui Lin
deb610e5d3
fix: update MixCoord registration in MilvusRoles (#41337)
enhance: update MixCoord registration in MilvusRoles

The `runMixCoord` function in `MilvusRoles` was updated to use the
`RegisterMixCoord` function from the `rootcoord_metrics` package instead
of `RegisterRootCoord`. This change aligns with the recent modifications
made to the `rootcoord_metrics` package.
issue:https://github.com/milvus-io/milvus/issues/41338

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-04-16 19:49:54 +08:00
Xianhui Lin
5318f3085c
feat: Add configuration for mixCoord (#41246)
Add configuration for mixCoord
relate-pr:https://github.com/milvus-io/milvus/pull/41006
issue : https://github.com/milvus-io/milvus/issues/37764

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-04-11 18:04:49 +08:00
Xianhui Lin
f9febe3bae
enhance: Merge RootCoord, DataCoord And QueryCoord into MixCoord (#41006)
Merge RootCoord, DataCoord And QueryCoord into MixCoord
Make Session into one
issue : https://github.com/milvus-io/milvus/issues/37764

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-04-11 16:36:30 +08:00
junjiejiangjjj
fe81c7baae
feat: Add function config (#40534)
#35856 
1. Add function-related configuration in milvus.yaml
2. Add null and empty value check to TextEmbeddingFunction

Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com>
2025-03-25 10:06:24 +08:00
yihao.dai
b2a8694686
enhance: Merge IndexNode and DataNode (#40272)
Merge DataNode and IndexNode into DataNode.

issue: https://github.com/milvus-io/milvus/issues/39115

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-03-13 14:26:11 +08:00
wei liu
972e47043a
enhance: Remove CPU profile to prevent blocking stop progress (#40460)
issue: #39735
related to #39726

- Removed CPU profile dump from util.go's pprof collection
- Avoid potential blocking in StopCPUProfile() during shutdown
- Maintain goroutine/heap/block/mutex profiles for diagnostics
- Ensure safe shutdown timeout handling without profile stalls

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-03-13 10:00:09 +08:00
Zhen Ye
bc8e02df44
enhance: always enable streaming service (#40253)
issue: #38399

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-28 15:38:01 +08:00
congqixia
35f188b329
enhance: Unify milvus version def (#40227)
Related to #40216

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-27 14:15:57 +08:00
congqixia
e0b028ade5
enhance: Integrate holmes as pprof dumper (#40151)
Related to #40150

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-25 19:01:55 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990)
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
wei liu
41dc1a705b
enhance: dump pprof info if component stop progress timeout (#39726)
issue: #39735

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-13 15:32:49 +08:00
Zhen Ye
c84a0748c4
enhance: add rw/ro streaming query node replica management (#38677)
issue: #38399

- Embed the query node into streaming node to make delegator available
at streaming node.
- The embedded query node has a special server label
`QUERYNODE_STREAMING-EMBEDDED`.
- Change the balance strategy to make the channel assigned to streaming
node as much as possible.

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-24 16:55:07 +08:00
yihao.dai
38f813bed3
enhance: Read metadata concurrently to accelerate recovery (#38403)
Read metadata such as segments, binlogs, and partitions concurrently at
the collection level.

issue: https://github.com/milvus-io/milvus/issues/37630

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-23 14:27:27 +08:00
Zhen Ye
fd84ed817c
enhance: add broadcast operation for msgstream (#39040)
issue: #38399

- make broadcast service available for msgstream by reusing the
architecture streaming service

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-14 15:14:59 +08:00
Zhen Ye
3e788f0fbd
enhance: record memory size (uncompressed) item for index (#38770)
issue: #38715

- Current milvus use a serialized index size(compressed) for estimate
resource for loading.
- Add a new field `MemSize` (before compressing) for index to estimate
resource.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-14 10:33:06 +08:00
Zhen Ye
bb8d1ab3bf
enhance: make new go package to manage proto (#39114)
issue: #39095

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-10 10:49:01 +08:00
Francisco Arceo
7f05d7a40f
doc: Update typo in "Welcome to Milvus" (#38995)
enhance: Updating a minor typo.

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
2025-01-05 19:00:54 +08:00
tinswzy
11f8f4a378
enhance: add intent ctx to rootcoord init (#38439)
issue: #35917 
add intent ctx to rootcoord initialization process

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-31 18:22:53 +08:00
Zhen Ye
118678bd91
enhance: make runtime config into a global environment table (#38671)
issue: #38399

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-24 21:00:49 +08:00
tinswzy
27229f7907
enhance: refine exists log print with ctx (#38080)
issue: #35917 
Refines exists log print with ctx

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-14 22:36:44 +08:00
tinswzy
7944538ade
enhance: Add ctx param to KV operation interfaces (#38154)
issue: #35917 
Refine KV operation interfaces by adding a ctx param

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-05 15:16:41 +08:00
shaoyue
1f66b9ebfb
feat: add config field to set internal tls sni (#38124)
/cc @xiaofan-luan @jaime0815 @nish112022

part of https://github.com/milvus-io/milvus/issues/36864

Signed-off-by: haorenfsa <haorenfsa@gmail.com>
2024-12-04 14:56:47 +08:00
SimFG
2208b7c2ef
fix: the too long default root password does not take effect (#37983)
- issue: #36987

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-11-26 17:24:35 +08:00
Zhen Ye
2b4f211d84
enhance: add switch for local rpc enabled (#37985)
issue: #33285

- Add switch for local rpc

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-26 17:00:54 +08:00
Zhen Ye
b9a10a2f68
enhance: remove the rpc layer of coordinator when enabling standalone or mixcoord (#37815)
issue: #37764

- add a local client to call local server directly for
querycoord/rootcoord/datacoord.
- enable local client if milvus is running mixcoord or standalone mode.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-23 21:36:33 +08:00
congqixia
cfa1f1f141
enhance: Add thread watcher to provide actual thread num (#37905)
Related to #37904

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-22 10:20:33 +08:00
nish112022
484c6b5c44
feat: Added code for Internal-tls (#36865)
issue : https://github.com/milvus-io/milvus/issues/36864

I have a few questions regarding my approach.I will consolidate them
here for feedback and review.Thanks

---------

Signed-off-by: Nischay Yadav <nischay.yadav@ibm.com>
Signed-off-by: Nischay <Nischay.Yadav@ibm.com>
2024-11-20 06:00:32 +08:00
jaime
1d06d4324b
fix: Int64 overflow in JSON encoding (#37657)
issue: ##36621

- For simple types in a struct, add "string" to the JSON tag for
automatic string conversion during JSON encoding.
- For complex types in a struct, replace "int64" with "string."

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-14 22:52:30 +08:00
Ted Xu
a9538f6e96
fix: config file validator missed the file tail validation (#37608)
See: #32168

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-11-13 15:14:30 +08:00
foxspy
d7b2ffe5aa
enhance: add an unify vector index config checker (#36844)
issue: #34298

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-10-28 10:11:37 +08:00
foxspy
3de57ec4fa
enhance: add vector index mgr to remove vector index type dependency (#36843)
issue: #34298

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-10-17 22:15:25 +08:00
yihao.dai
3685edb264
enhance: Use common gc config (#36668)
Use the GC config from `common` and remove the GC config from
`queryNode`.

issue: https://github.com/milvus-io/milvus/issues/36667

related pr: https://github.com/milvus-io/milvus/pull/34949

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-09 19:47:19 +08:00
Zhen Ye
a6545b2e29
fix: refactor milvus config and change default txn timeout (#36522)
issue: #36498

Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-29 11:01:15 +08:00
Zhen Ye
d29e01e284
fix: port listen racing in mix or standalone mode (#36442)
issue: #36441

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-26 21:23:16 +08:00
Zhen Ye
f65261215b
fix: streaming node health check panic (#36336)
issue: #36335

Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-19 17:11:12 +08:00
Jiquan Long
89bf226f0b
feat: support keyword text match (#35923)
fix: #35922

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-09-10 15:11:08 +08:00
wei liu
5aedc169cd
fix: Revert "enhance: avoid the coexistence of the old coordinator and the new node/proxy (#35720)" (#36113)
issue: #36100
This reverts commit 73be0ba9415192130b013fa118cd31dac63ca751.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-10 10:23:07 +08:00
SimFG
99817953eb
fix: delay to start the metric server port (#36080)
- issue: #36083
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-09-09 14:01:05 +08:00
congqixia
9e96ed4873
fix: Fix tracing config update logic (#35928)
Related to #35927

There are serveral issue this PR addresses:
- Use `ResetTraceConfig` method instead init one in update event handler
- Implement dynamic stats.Handler to receive tracing config update event
- Update `enable_trace` flag when `ResetTraceConfig` is invoked
- Change `enable_trace` to `std::atomic<bool>` in case of data race

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-05 14:27:04 +08:00
wei liu
73be0ba941
enhance: avoid the coexistence of the old coordinator and the new node/proxy (#35720)
issue: #35719
In standalone mode, block the start process until the new coordinator is
active to avoid the coexistence of the old coordinator and the new
node/proxy
1. In the start/restart process, the new coordinator will become active
immediately and will not be blocked
2. In the rolling upgrade process, the new coordinator will not be
active until the old coordinator is down, and it will be blocked

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-29 13:55:01 +08:00
Zhen Ye
99dff06391
enhance: using streaming service in insert/upsert/flush/delete/querynode (#35406)
issue: #33285

- using streaming service in insert/upsert/flush/delete/querynode
- fixup flusher bugs and refactor the flush operation
- enable streaming service for dml and ddl
- pass the e2e when enabling streaming service
- pass the integration tst when enabling streaming service

---------

Signed-off-by: chyezh <chyezh@outlook.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-29 10:03:08 +08:00
Zhen Ye
75da36d1aa
enhance: enable asan for milvus (#35627)
issue: #35626

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-23 21:06:58 +08:00
SimFG
731d45abbe
enhance: provide more general configuration to control mmap behavior (#35359)
- issue: #35273

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-21 00:22:54 +08:00