173 Commits

Author SHA1 Message Date
sunby
4c18910acd
Improve log print in rootcoord (#7354)
issue: #7306
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-13 11:56:42 +08:00
groot
c8f860dcfb
Add unittest for proxy_client_manager (#7729)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-10 18:10:00 +08:00
groot
020f109dd8
Add unittest for rootcoord util (#7688)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-10 16:41:48 +08:00
Cai Yudong
4e23ed9a18
Revert apply physical channels from pool (d5f7e358 and 3b960969) (#7687)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-10 13:00:00 +08:00
Cai Yudong
d5f7e358aa
Let dml tolerant physical channels generated by old version (#7644)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-09 18:08:02 +08:00
zhenshan.cao
29e5138274
Fix unittest: timeout caused by excessive use of resources (#7653)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-09 17:39:30 +08:00
zhenshan.cao
b0524adbd8
Support specified shard number when create collection (#7482)
Add shards_num to CollectionInfo

Modify dmlChannelNum config

Compatible with old meta

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-08 15:00:00 +08:00
Cai Yudong
d15d3fa0ac
Improve rootcoord codecov (#7518)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-07 11:16:37 +08:00
zhenshan.cao
86afde4598
Fix unittest for rootcoord, should filter out timetick msg first (#7490)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-06 11:52:41 +08:00
dragondriver
960741ee16
Support more detailed metrics information (#7472)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-03 19:35:47 +08:00
dragondriver
1db573731b
Cache the system information metrics (#7436)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-03 17:15:26 +08:00
Xiangyu Wang
acf4354b83
Add logs in rootcoord (#7457)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-03 14:10:10 +08:00
dragondriver
3721cbfea1
Proxy metrics integrates with other coordinators (#7400)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-01 17:35:00 +08:00
Cai Yudong
3b9609692b
Apply for msgstream from pool when creating collection (#7377)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-31 18:35:58 +08:00
dragondriver
f0dbdd7f13
Expose metrics of root coordinator (#7228)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-08-31 11:45:59 +08:00
Cai Yudong
f4b53df727
Revert "Apply for msgstream from pool when creating collection (#7323)" (#7376)
This reverts commit e6de86a4336e623861fd476afa458383b9883662.
2021-08-31 11:07:58 +08:00
Cai Yudong
e6de86a433
Apply for msgstream from pool when creating collection (#7323)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-31 09:47:58 +08:00
Xiangyu Wang
c62f23e201
[skip ci]Remove neza2017 from reviewers (#7324)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-08-27 00:27:36 +08:00
congqixia
4d9f89df09
Add unit test for 7150 fix (#7276)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-25 14:41:52 +08:00
congqixia
d83bc7fb54
Fix 0 ts when build index (#7272)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-24 21:15:52 +08:00
Xiaofan
58b2089692
Import Embedded Etcd Kv Part 1(#7112) (#7174)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-08-24 09:45:51 +08:00
Cai Yudong
8405d90f5e
proxy add illegal check for search result (#7227)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-23 17:03:51 +08:00
congqixia
ec9ccd8bd1
Fix DropCollection not processed by datanode (#7151)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-19 15:06:12 +08:00
Cai Yudong
305fa9c17e
Fix timestamp go back issue for timeticksync (#7123)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-18 14:36:10 +08:00
Cai Yudong
21117241ec
Let system panic if kv operation fail (#7143)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-17 23:06:10 +08:00
yukun
847586eb95
Add rocksmq retention (#6617)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-08-16 18:46:10 +08:00
XuanYang-cn
6c5be83ea1
Refactor NewEtcdKV API (#7075)
This PR mainly did the following 3 things:

- Refactor NewEtcdKV API
- Add unittests for `kv/etcd/`, raise the coverage
up to 94%
- Correct some APIs from version to revision

The old NewEtcdKV() API took `clientv3.Client` as
one of the input, making everyone who using this API
had to create a new `clientv3.Client`. So there're a lot
of repeated codes everywhere.

Meanwhile, someplace didn't even check whether the
`clientv3.New()` returned error or not, and the old API
also didn't check whether the clientv3.Client was Nil.
This caused issue#6955.

Ectd's version and revision have nothing alike, they
represent different concepts, but our APIs kind of
mixed them all up. So I correct those APIs' names.

Resolves: #6955

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-08-13 11:04:09 +08:00
Xiangyu Wang
5e78fd6a97
[skip ci]Update approvers (#7063)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-08-12 19:36:09 +08:00
xige-16
09199170a4
sync call querycoord releaseXXXX (#6978)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-08-03 21:47:24 +08:00
XuanYang-cn
47df4a2e57
Fix DataNode BackGroudGC problem (#6551)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-08-03 09:59:24 +08:00
neza2017
5adb69161f
Set CollectionInfo.CreateTime to the timestamp when save the metadata into etcd, (#6763)
not the timestamp when RootCoord receive the CreateCollection Request

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-23 14:36:12 +08:00
dragondriver
67c1fdd2c0
Show/Describe collections/partitions with created timestamp (#6700)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-21 18:00:14 +08:00
Cai Yudong
cbb4052fdb
Support search and query output fields using wildcard (#6671)
* update wildcard polocy for search and query

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* mark xfail cases

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* mark xfail cases

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize debug log

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-21 14:52:12 +08:00
neza2017
fac350cc58
fix tso (#6500)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-14 17:11:54 +08:00
neza2017
b66157358c
add more log for create collection (#6478)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-13 22:19:55 +08:00
neza2017
d4239f0cd8
masterservice -> rootcoord (#6447)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-13 09:37:40 +08:00
neza2017
8b89744c6b
search with exp (#6409)
* process float vector and binary vector field
on the search result

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add more debug info on etcd

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add debug info when grpc failed

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-10 10:21:52 +08:00
Cai Yudong
21a9bcd844
Create default partition when receive CreateCollectionMsg (#6305)
* create default partition when create collection

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* adjust sleep time for unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-06 09:16:03 +08:00
neza2017
ca4a562a72
simplify AddIndex, remove collID,PartID from parameters (#6291)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-05 10:08:02 +08:00
Cai Yudong
bc8392eb1e
Add collectionID and partitionID into SegmentIndexInfo (#6289)
* add collectionID and partitionID into SegmentIndexInfo

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add partID2SegID into meta table

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update DropIndex

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-03 20:00:40 +08:00
neza2017
f8fa025dcc
start background go routine to check if all the segments have indexed (#6290)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-03 17:54:25 +08:00
neza2017
111a24a49f
remove msgstream of segment info (#6280)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-03 14:36:18 +08:00
sunby
07b0989628
Add GetFlushedSegments in data coordinator (#6253)
Send segment flush message by grpc and add a new interface
`GetFlushedSegments`

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-02 11:16:20 +08:00
neza2017
6019c193fc
add grpc interface to process NewSegment And FlushSegmentCompleted (#6247)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-01 14:58:17 +08:00
godchen
49a0528d53
Fix trace (#6218)
* Fix trace

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-30 16:18:13 +08:00
neza2017
95ad498822
check proxy if exist before connect to it (#6165)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-28 15:08:12 +08:00
Cai Yudong
a9a806ace0
optimize WatchProxy (#6153)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-28 11:04:14 +08:00
neza2017
b22ab71222
fix rmq (#6147)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-26 22:26:20 +08:00
neza2017
6d7edc6015
fix typo (#6142)
* fix typo

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add log

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-26 18:04:12 +08:00
zhenshan.cao
42be531f9c
Add InsertCnt (#6131)
* Add InsertCnt

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Enhance log

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-26 14:02:12 +08:00