51 Commits

Author SHA1 Message Date
Zhen Ye
af0881ee5d
fix: timetick cannot push forward when upgrading (#42567)
issue #42492

- streamingcoord start before old rootcoord.
- streaming balancer will check the node session synchronously to avoid
redundant operation when cluster startup.
- ddl operation will check if streaming enabled, if the streaming is not
enabled, it will use msgstream.
- msgstream will initialize if streaming is not enabled, and stop when
streaming is enabled.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-06-10 14:52:42 +08:00
Zhen Ye
f6fb4bc442
fix: backoff will retry infinitely after reaching max elapse (#40589)
issue: #40588

Signed-off-by: chyezh <chyezh@outlook.com>
2025-03-13 16:24:06 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990)
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
Zhen Ye
fd84ed817c
enhance: add broadcast operation for msgstream (#39040)
issue: #38399

- make broadcast service available for msgstream by reusing the
architecture streaming service

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-14 15:14:59 +08:00
tinswzy
11f8f4a378
enhance: add intent ctx to rootcoord init (#38439)
issue: #35917 
add intent ctx to rootcoord initialization process

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-31 18:22:53 +08:00
tinswzy
27229f7907
enhance: refine exists log print with ctx (#38080)
issue: #35917 
Refines exists log print with ctx

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-14 22:36:44 +08:00
tinswzy
5768dbbb5d
enhance: refine pular related mq interfaces (#38007)
issue: #35917 
Refines the pulsar-related mq APIs to allow the ctx to be passed down

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-04 20:50:39 +08:00
yihao.dai
ebc3c82761
fix: Fix subscription leak (#37382)
Close (unsubscribe) the msg stream after completing the
`PreCreatedTopic` check to prevent backlog issue.

issue: https://github.com/milvus-io/milvus/issues/36021

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-08 16:16:26 +08:00
jaime
9630974fbb
enhance: move rocksmq from internal to pkg module (#33881)
issue: #33956

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-06-25 21:18:15 +08:00
smellthemoon
a9982aa7c6
Remove topic empty restriction when preCreated topic (#27351)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-10-10 18:53:32 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
XuanYang-cn
7f1ae35e72
Add timeout in dispatcher, AsConsumer and Seek (#26686)
See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-08 09:51:17 +08:00
congqixia
3c503afe7c
Use typeutil.ConcurrentMap instead of sync.Map (#25846)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-24 10:23:01 +08:00
Enwei Jiao
9fe62cb5f3
Fix kafka producer panic by nil (#25691)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-18 18:24:57 +08:00
Enwei Jiao
66fdc71479
Refactor logs in DataCoord & DataNode (#25574)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-14 15:56:31 +08:00
SimFG
0c3f92d7d7
Improve the panic code about the rootcoord/session/rocksmq (#24859) (#25024)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-06-21 11:24:42 +08:00
smellthemoon
ab95754069
Remove debug log (#24030)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-11 14:09:19 +08:00
smellthemoon
f1adb1c06d
can only use the topic in the list when preCreatedTopicEnabled (#23913)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-10 13:47:19 +08:00
smellthemoon
146050db82
Fix some wrong ut (#23990)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-10 09:31:19 +08:00
Xiaofan
87d790f052
Fix upgrade casue panic (#23833)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-05-02 14:06:37 +08:00
smellthemoon
912cf4ef0f
Change some configurations, include change the defaultChannelNum to 16 (#23617)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-04-27 14:26:35 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
smellthemoon
ce91f61e5a
Pre-create topics (#21907)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-04-03 16:44:23 +08:00
Jiquan Long
270c5b0502
Refine log when initialize dml channels (#21863)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-30 14:05:54 +08:00
jaime
039e9ce4bb
Refine msgstream interface (#20832)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-12-05 20:55:17 +08:00
Jiquan Long
1817627316
Fix create collection failed to assign channels (#19551)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-09-29 15:54:54 +08:00
congqixia
b62cb82ebe
Simplify Dml-DeltaChannel mapping logic (#18702)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-18 14:24:50 +08:00
congqixia
318cdbebe9
Add heap for dmlChannels to get next channel (#18451)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-01 16:36:33 +08:00
zhenshan.cao
0baeb609dc
Fix bugs in monitoring metrics (#17894)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-30 17:26:19 +08:00
Jiquan Long
8e2797fe30
Print error when failed to broadcast message (#17595)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-16 18:32:10 +08:00
Xiaofan
7f7379d55f
Fix DML channel AsProducer after Close (#16352)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-02 13:09:29 +08:00
jaime
7a3a721380
Reconstruct mqstream module (#15784)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-03 21:57:56 +08:00
cai.zhang
9f4c59e621
Add prometheus metrics for RootCoord (#15684)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-02 21:11:57 +08:00
XuanYang-cn
c544bb21ac
[skip ci]Update license for rc dml channels (#13497)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-16 10:03:23 +08:00
Bingyi Sun
92eff1565c
Fix unlocked mutex in rootcoord (#12772)
issue: #12297
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-06 17:25:35 +08:00
Cai Yudong
858e95f377
Move dmlChannel and deltaChannel into timeticksync (#12254)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-11-25 10:07:15 +08:00
Cai Yudong
d4c297b1a8
Enhance dml channel operations (#12143)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-11-21 21:33:13 +08:00
congqixia
27715c15f4
Fix 0 len slice declaration for dml_channels.go (#12094)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-18 22:33:26 +08:00
Cai Yudong
628a3274bf
Use lazy mode to set dml msgstream as producer (#10993)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-11-03 20:18:48 +08:00
Xiangyu Wang
a7d55229f0
Improve error message in rootcoord (#10836)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-10-28 20:03:29 +08:00
congqixia
88e319acda
Save start position in collection meta (#8682)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-27 18:10:00 +08:00
Cai Yudong
d10c6abc9e
Apply for msgstream from pool when creating collection (#7738)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-18 09:13:50 +08:00
Cai Yudong
4e23ed9a18
Revert apply physical channels from pool (d5f7e358 and 3b960969) (#7687)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-10 13:00:00 +08:00
Cai Yudong
d5f7e358aa
Let dml tolerant physical channels generated by old version (#7644)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-09 18:08:02 +08:00
Cai Yudong
3b9609692b
Apply for msgstream from pool when creating collection (#7377)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-31 18:35:58 +08:00
Cai Yudong
f4b53df727
Revert "Apply for msgstream from pool when creating collection (#7323)" (#7376)
This reverts commit e6de86a4336e623861fd476afa458383b9883662.
2021-08-31 11:07:58 +08:00
Cai Yudong
e6de86a433
Apply for msgstream from pool when creating collection (#7323)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-31 09:47:58 +08:00
congqixia
ec9ccd8bd1
Fix DropCollection not processed by datanode (#7151)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-19 15:06:12 +08:00
Cai Yudong
305fa9c17e
Fix timestamp go back issue for timeticksync (#7123)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-18 14:36:10 +08:00
XuanYang-cn
47df4a2e57
Fix DataNode BackGroudGC problem (#6551)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-08-03 09:59:24 +08:00