chyezh
16b0aee97f
enhance: timetick interceptor optimization ( #35287 )
...
issue: #33285
- remove redundant goroutine by using insepctor.
- remove the coutinous non-message timetick persistence
- periodically push the time tick forward without persistent timetick
message.
- add 'message type filter' deliver filter.
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-12 18:58:25 +08:00
chyezh
c725416288
enhance: move streaming proto into pkg ( #35284 )
...
issue: #33285
- move streaming related proto into pkg.
- add v2 message type and change flush message into v2 message.
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-07 10:34:16 +08:00
chyezh
14051fed7d
enhance: streaming service client ( #34656 )
...
issue: #33285
- implement streaming service client.
- implement producing and consuming service client by streaming coord
client and streaming node client.
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-05 21:38:15 +08:00
chyezh
9871966415
enhance: segment alloc interceptor ( #34996 )
...
#33285
- add segment alloc interceptor for streamingnode.
- add add manual alloc segment rpc for datacoord.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-04 07:40:15 +08:00
yihao.dai
a4439cc911
enhance: Implement flusher in streamingNode ( #34942 )
...
- Implement flusher to:
- Manage the pipelines (creation, deletion, etc.)
- Manage the segment write buffer
- Manage sync operation (including receive flushMsg and execute flush)
- Add a new `GetChannelRecoveryInfo` RPC in DataCoord.
- Reorganize packages: `flushcommon` and `datanode`.
issue: https://github.com/milvus-io/milvus/issues/33285
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-02 18:30:23 +08:00
chyezh
86eff6e589
enhance: streaming node client implementation ( #34653 )
...
issue: #33285
- add streaming node grpc client wrapper
- add unittest for streaming node grpc client side
- fix binary unsafe bug for message
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-19 17:37:40 +08:00
chyezh
dfe0416a70
enhance: implement streaming node server service ( #34166 )
...
issue: #33285
- implement producing and consuming server of message
- implement management operation for streaming node server
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-09 07:58:14 +08:00
chyezh
3563136c2a
enhance: timetick interceptor implementation ( #34238 )
...
issue: #33285
- optimize the message package
- add interceptor package to achieve append operation intercepting.
- add timetick interceptor to attach timetick properties for message.
- add timetick background task to send timetick message.
Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-02 14:42:08 +08:00
chyezh
d2bc4a53be
enhance: implement rmq and pulsar as wal ( #34046 )
...
issue: #33285
- use reader but not consumer for pulsar
- advanced test framework
- move some streaming related package into pkg
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-06-27 15:11:05 +08:00
chyezh
b9237280c2
enhance: wal interface definition ( #33745 )
...
issue: #33285
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-06-24 10:34:12 +08:00
