yihao.dai
ee9d08375d
enhance: Accelerate dispatcher building ( #42500 )
...
Reduce check interval to accelerate dispatcher building.
issue: https://github.com/milvus-io/milvus/issues/42067
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-06-05 16:36:32 +08:00
SimFG
a308d2c886
fix: get replicate channel position ( #41188 )
...
- issue: #41187
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-04-10 15:14:28 +08:00
yihao.dai
d8d1dcf076
fix: Fix dispatcher deregister and seek ( #40860 )
...
1. Fix deregister dispatcher concurrency. (Keep sure the same logic as
2.5 branch)
2. Fix seek if includeCurrentMsg. (This is only needed by CDC, so
there's no need to pick to 2.5 branch)
issue: issue: https://github.com/milvus-io/milvus/issues/39862
pr: https://github.com/milvus-io/milvus/pull/39863
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-03-31 11:38:19 +08:00
yihao.dai
a33c9372ce
fix: Fix channel not balance on datanodes ( #40422 )
...
1. Prevent channels from being assigned to only one datanode during
datacoord startup.
2. Optimize the channel assignment policy by considering newly assigned
channels.
3. Make msgdispatcher manager lock-free.
issue: https://github.com/milvus-io/milvus/issues/40421 ,
https://github.com/milvus-io/milvus/issues/37630
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-03-11 14:56:16 +08:00
yihao.dai
004a1875dc
enhance: Introduce batch subscription in msgdispatcher ( #39863 )
...
Introduce a batch subscription mechanism in msgdispatcher: the
msgdispatcher now includes a vchannel watch task queue, where all
vchannels in the queue will subscribe to the MQ only once and pull
messages from the oldest vchannel checkpoint to the latest.
issue: https://github.com/milvus-io/milvus/issues/39862
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-03-05 14:38:02 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module ( #39990 )
...
Related to #39095
https://go.dev/doc/modules/version-numbers
Update pkg version according to golang dep version convention
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
aoiasd
24d2bbc441
enhance: unmashall ts msg in dispatcher instead in msgstream ( #38656 )
...
relate: https://github.com/milvus-io/milvus/issues/38655
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-02-14 12:04:13 +08:00
yihao.dai
8288c06e3d
fix: Fix flowgraph leak ( #39656 )
...
Consider lag targets when determining whether to remove the dispatcher
manager.
issue: https://github.com/milvus-io/milvus/issues/39642
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-07 16:16:44 +08:00
SimFG
2afe2eaf3e
feat: support to replicate collection when the services contains the system tt msg ( #37559 )
...
- issue: #37105
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-17 09:08:46 +08:00
tinswzy
5768dbbb5d
enhance: refine pular related mq interfaces ( #38007 )
...
issue: #35917
Refines the pulsar-related mq APIs to allow the ctx to be passed down
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-04 20:50:39 +08:00
Zhen Ye
cd181e4c6d
fix: make NewDataSyncService idempotent of dispatcher ( #37576 )
...
issue: #37547
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-14 11:00:31 +08:00
yihao.dai
7721a28584
fix: Fix unstable TestDispatchToVchannels ut ( #35163 )
...
issue: https://github.com/milvus-io/milvus/issues/35150
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-01 14:56:13 +08:00
yihao.dai
aef7664fc6
fix: Prevent dispatcher merging if curTs is 0 ( #34562 )
...
When the main dispatcher has not yet consumed data, curTs is 0. During
this time, merging dispatchers should not be allowed; otherwise, the
data of the solo dispatcher will be skipped.
issue: https://github.com/milvus-io/milvus/issues/34255
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-11 00:02:50 +08:00
yihao.dai
391d2aa5b5
fix: Fix unstable ut of msgdispatcher ( #34485 )
...
issue: https://github.com/milvus-io/milvus/issues/34484
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-09 10:08:11 +08:00
yihao.dai
ff51c7e628
fix: Fix panic due to failed to seek ( #34229 )
...
Converting the same msgposition's vchannel to a pchannel multiple times
would result in an invalid pchannel, leading to seek failure and panic.
This PR:
1. Make a copy of msgposition in msgdispatcher.
2. Check if channel is already a pchannel, no further channel conversion
is performed.
issue: https://github.com/milvus-io/milvus/issues/34221
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-01 16:08:12 +08:00
jaime
9630974fbb
enhance: move rocksmq from internal to pkg module ( #33881 )
...
issue: #33956
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-06-25 21:18:15 +08:00
XuanYang-cn
1629833060
enhance: Add consts of MsgDispatcher to configs ( #33679 )
...
See also: #33676
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-06-07 14:21:59 +08:00
yihao.dai
8febbc79bb
fix: Fix unstable msg dispatcher ut ( #29925 )
...
This fix will not augment the execution time of unit tests, but solely
enhances tolerance for waiting for failure.
issue: https://github.com/milvus-io/milvus/issues/29921
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-01-14 10:40:52 +08:00
SimFG
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
XuanYang-cn
7f1ae35e72
Add timeout in dispatcher, AsConsumer and Seek ( #26686 )
...
See also: #25309
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-08 09:51:17 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
congqixia
5276529524
Make pkg code change trigger ci workflow ( #23312 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-11 10:32:31 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
