33 Commits

Author SHA1 Message Date
neza2017
0a82c6381f add auto flush for data node (#5609)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:04:48 +08:00
congqixia
d0d845805b Change WatchDmChannelsRequest proto (#5577)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:04:48 +08:00
XuanYang-cn
6c02ae4eb1 Remove Old msgposition logic (#5487)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
XuanYang-cn
7a8b34ee9f Remove not using code in DataNode (#5476)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
XuanYang-cn
55eac3ee7f Complete Flush procedure in DataNode (#5454)
* Complete Flush proceduer in DataNode

Resolves: #5220

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix unittest

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
sunby
c53afee616 DataNode scales flowgraph
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
XuanYang-cn
fc630bc1c8
Correct auto-flush behaviour (#5282)
Before this PR, DataNode considered auto-flush a valid flush
complete. It's wrong. So I open this PR to correct this behaviour
in DataNode.

Now binlog paths from auto-flush will be buffered in replica,
waiting until the manul flush to save into etcd all together.

See also: #5220, #5268
A follow up job of #5271 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-05-18 11:45:00 +00:00
Xiangyu Wang
82ccd4cec0
Rename module (#4988)
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
Cai Yudong
51cae2c9ba
add datanode unittest to improve code coverage (#4982)
add datanode unittest to improve code coverage 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-04-21 18:41:37 +08:00
XuanYang-cn
6deba1faa5 Add license in datanode package
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-04-19 15:16:33 +08:00
XuanYang-cn
91f5c48037 Fix datanode goroutine leak
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-03-30 09:47:27 +08:00
sunby
ca3187d173 Fix data race in datanode tests
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-03-24 16:35:38 +08:00
XuanYang-cn
51f669f1cb Flush segments concurrently & change etcd meta
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-03-23 18:50:13 +08:00
godchen
f3649f0419 Refactor interface and proto
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-03-12 14:22:09 +08:00
xige-16
09ae985daa Fix wrong error code in master_service_test.go
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-10 22:06:22 +08:00
neza2017
7b2d67242d Fix describe segment if index not exist
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-03-10 14:45:35 +08:00
quicksilver
455cc59dbb Update publish-test-images.yaml
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-03-09 13:55:35 +08:00
neza2017
2835bcdc4e Update masterservice unittest
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-03-05 20:41:34 +08:00
godchen
7b9fdd7f29 Add opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-02-26 17:44:24 +08:00
XuanYang-cn
10374f15d7 Update log in datanode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-02-26 10:13:36 +08:00
Xiangyu Wang
fd9e42d3ed Update msgstream/unmarshal interface
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-02-03 17:30:10 +08:00
bigsheeper
abffdbff62 Remove partition tag
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 16:44:38 +08:00
XuanYang-cn
73a8e7de84 Fix datanode insertchannel refresh bug
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-02-03 15:18:05 +08:00
Xiangyu Wang
99cef4b5c8 Add Distributed main entrypoint
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-01-29 17:08:31 +08:00
bigsheeper
73d2b6a101 Get index param from minio and filter by vector fields
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-29 15:22:24 +08:00
neza2017
bdf84f08ab Add watcher_test and fix some bugs
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-29 14:40:19 +08:00
sunby
f088467dde Add watcher_test and fix some bugs
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-28 10:14:31 +08:00
XuanYang-cn
b597a637f3 Add datanode metatable tests
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-29 12:03:10 +08:00
XuanYang-cn
b445587e11 Add datanode main
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-26 14:46:54 +08:00
XuanYang-cn
9fe672ab86 Enchance datanode interface
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-24 21:20:11 +08:00
XuanYang-cn
50e2369000 Change name and tag to ID
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-22 19:36:09 +08:00
XuanYang-cn
ec27788628 Add segment statistics updates logic for datanode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-21 09:55:25 +08:00
XuanYang-cn
6a13386393 Add DataNode package
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-19 11:37:16 +08:00