12 Commits

Author SHA1 Message Date
Zhen Ye
070aabd27e
enhance: fix remove flushing state of segment (#43560)
issue: #43559, #42884

- also fix the data lost when streaming resuming from old arch message.

Signed-off-by: chyezh <chyezh@outlook.com>
2025-07-25 18:08:54 +08:00
tinswzy
7da62698e0
enhance: improve WP parallel sync mechanism and fencing logic (#42892)
related: #42595 
improve WP parallel sync mechanism and fencing logic; remove redundant
metrics and labels

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-07-13 23:04:49 +08:00
Zhen Ye
b94cee2413
fix: growing segment from old arch is not flushed after upgrading (#42164)
issue: #42162

- enhance: add read ahead buffer size issue #42129
- fix: rocksmq consumer's close operation may get stucked
- fix: growing segment from old arch is not flushed after upgrading

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-05-29 23:00:28 +08:00
Zhen Ye
52950ce392
enhance: add pulsar truncate api to protect pulsar unconsumed message (#41724)
issue: #41465

- implement truncate api for pulsar based on durable subscription.
- truncate api can only be called if wal is read-write.

Signed-off-by: chyezh <chyezh@outlook.com>
2025-05-11 20:50:55 +08:00
Zhen Ye
6982f007e2
enhance: add walimpls access mode options (#40591)
issue: #40532

Signed-off-by: chyezh <chyezh@outlook.com>
2025-03-14 10:58:11 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990)
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
chyezh
14051fed7d
enhance: streaming service client (#34656)
issue: #33285

- implement streaming service client.
- implement producing and consuming service client by streaming coord
client and streaming node client.

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-05 21:38:15 +08:00
chyezh
4f6cbfd520
enhance: specialized immutable and mutable message (#34951)
issue: #33285

- add specialized mutable and immutable message, make type safe.
- add version based constructor and type.

Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-25 11:57:45 +08:00
chyezh
39c7e06bc5
enhance: add message and msgstream msgpack adaptor (#34874)
issue: #33285

- make message builder and message conversion type safe
- add adaptor type and function to adapt old msgstream msgpack and
message interface

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-22 20:59:42 +08:00
chyezh
dfe0416a70
enhance: implement streaming node server service (#34166)
issue: #33285

- implement producing and consuming server of message
- implement management operation for streaming node server

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-09 07:58:14 +08:00
chyezh
3563136c2a
enhance: timetick interceptor implementation (#34238)
issue: #33285

- optimize the message package
- add interceptor package to achieve append operation intercepting.
- add timetick interceptor to attach timetick properties for message.
- add timetick background task to send timetick message.

Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-02 14:42:08 +08:00
chyezh
d2bc4a53be
enhance: implement rmq and pulsar as wal (#34046)
issue: #33285

- use reader but not consumer for pulsar
- advanced test framework
- move some streaming related package into pkg

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-06-27 15:11:05 +08:00