admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-03 01:12:25 +08:00
Code Issues Packages Projects Releases Wiki Activity
7,053 Commits 138 Branches 188 Tags
Commit Graph

1044 Commits

Author SHA1 Message Date
congqixia
a2dd16472d Implement random reassign and add test case (#5474) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 15:59:04 +08:00
congqixia
8657251f41 DataNode stores segment msgpack positions (#5472) 
* DataNode stores segment msgpack positions

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix typo caused deadlock

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Use MsgPack EndPositions for flush pos

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 15:59:04 +08:00
sunby
43534ef63f Add unit tests in dataservice (#5478) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 15:59:04 +08:00
XuanYang-cn
7a8b34ee9f Remove not using code in DataNode (#5476) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 15:59:04 +08:00
sunby
a14c35274f Fix bugs in dataservice (#5457) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 15:59:04 +08:00
XuanYang-cn
55eac3ee7f Complete Flush procedure in DataNode (#5454) 
* Complete Flush proceduer in DataNode

Resolves: #5220

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix unittest

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 15:59:04 +08:00
dragondriver
59a3d83f97 Replace insertChannelsMap with channelsMgr (#5453) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-15 15:59:04 +08:00
congqixia
607a8ffa1f Update WatchDmChannels signature (#5447) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 15:59:04 +08:00
sunby
cfdd98b70f Fix recovery branch after rebasing master (#5444) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 15:59:04 +08:00
sunby
07c6a4a669 Add dataservice register and discovery (#5435) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 15:59:04 +08:00
congqixia
e3956ad13f Change SaveBinLogPaths field2Path to repeated (#8) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 15:59:04 +08:00
yangxuan
ef563fda8f Updata flushe procedure 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 15:59:04 +08:00
sunby
c53afee616 DataNode scales flowgraph 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 15:59:04 +08:00
sunby
b9cf4c1095 Fix recovery (#5417) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 15:59:04 +08:00
sunby
81ac20143b Add persistency of datanode cluster (#5387) 
We save channels registered in datanode in etcd and restore the cluster info after
restarting.

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 15:59:04 +08:00
sunby
3662b3f0e5 Refactor dataservice (#5400) 
Pick up grpc calls to grpc_handler.go and refactor code format

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 15:59:01 +08:00
congqixia
ef7339ceda DataService stores flush msgstream pos at VChannel granularity (#5397) 
SaveBinLogPaths saves stream positions at VChannel granularity

resolves #5396

Signed-off-by: Congqi Xia congqi.xia@zilliz.com
 2021-06-15 15:54:57 +08:00
FluorineDog
bec9f2c182
Split segcore and plan proto for future feature (#5767) 
* Split segcore and plan proto for future feature

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* lint

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-06-15 14:43:57 +08:00
bigsheeper
cdbc6d2c94
Refactor query node and query service (#5751) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-15 12:41:40 +08:00
FluorineDog
4165b761c9
fix not in bug (#5741) 
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-06-11 17:08:25 +08:00
FluorineDog
006dae35c3
fix retrieve bug (#5727) 
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-06-11 00:06:55 +08:00
yukun
e97936ba1c
Fix query hang bug (#5709) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-10 14:33:49 +08:00
bigsheeper
8aae0f7cc9
Refactor flow graph and load/watchDML in query node (#5682) 
* Refactor flow graph and load/watchDML in query node

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-09 11:37:55 +08:00
yukun
313658767d
Add field_name and type in retrieve result (#5665) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-08 16:26:36 +08:00
Xiangyu Wang
68f703f983
Add Query impl in proxynode (#5644) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-06-07 17:21:36 +08:00
sunby
50b3d39370
Add GetRecoveryInfo interface and proto (#5598) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-07 14:16:36 +08:00
zhenshan.cao
bc46e4780d
Fixbug:index is not loaded in some cases (#5633) 
* Fixbug:index is not created in some cases

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Delete unecessary print

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* fix create index

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-06 09:41:35 +08:00
shengjun.li
8ead67ef95
IVF_SQ8 and IVF_PQ cannot be built on multiple GPUs (#5624) 
* IVF_SQ8 and IVF_PQ cannot be built on multiple GPUs

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* index created by std::make_shared

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2021-06-05 17:19:34 +08:00
zhenshan.cao
7dac20c35c
Add log for poxynode transform data and querynode segment insert (#5627) 
* Add log for poxynode transform data and querynode segment insert

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Decrease print times

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-05 16:21:36 +08:00
cai.zhang
5037646cbd
Check dsl whether is None (#5616) 
* Check dsl whether is None

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Format code

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
 2021-06-05 14:57:34 +08:00
yukun
45e7d26b01
Fix retrieve when output_fields is nil (#5623) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-05 10:57:34 +08:00
FluorineDog
1ff4764ba6
Fix wrong primary key (#5604) 
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-06-04 18:35:36 +08:00
zhenshan.cao
03e3b4beb4
Add log to startup process (#5594) 
* Add log to startup process

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fixbug: avoid ctx exceed deadline

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fmt code

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fix bug: wrong time unit

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fixbug: datanservice client reconnect use time ctx

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-04 16:29:35 +08:00
Cai Yudong
2ba93deb5e
Support master get segment info channel name from config file (#5606) 
* optimize msgstream unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* get data service segment info channel name from config file directly

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-04 16:28:34 +08:00
Xiangyu Wang
afa5da7df0
Add query rpc into proto (#5592) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-06-04 12:02:34 +08:00
FluorineDog
9a90313390
Support GetEntityByIDs in CGo, fix segcore bugs (#5563) 
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-06-04 10:38:34 +08:00
Xiangyu Wang
e80f06c030
Add retrieve process log (#5561) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-06-03 20:56:33 +08:00
Xiangyu Wang
f791ccb246 Update OWNERS file 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-06-03 20:01:24 +08:00
yukun
57ece4a771
Fix retrieve_collection bugs (#5575) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-03 19:41:33 +08:00
dragondriver
99699b2021
Support to show collections which are loaded into memory (#5567) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-03 19:09:33 +08:00
godchen
0610ceb3dc
Fix ut error (#5573) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-03 18:44:33 +08:00
dragondriver
e76830ab09
Support column-based insert request in proxy (#5560) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-03 15:03:34 +08:00
zhenshan.cao
b0b8f58192
Fix bug: search timeout (#5557) 
* Fix bug: search timeout

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Add log and fix unittest bug

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-03 14:58:34 +08:00
Cai Yudong
4018dfc5b7
Support data segment channel seek to the break point when system start (#5540) 
* add DecodeDdOperation

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* reload MsgPosition from etcd and seek when system start

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename const string

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize master unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add msg into dataSegmentChannel to test seek

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix seek test

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* roll back distributed/masterservice/masterservice_test.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-02 22:36:41 +08:00
yukun
dcb4161c9f
Fix retrieve_service (#5531) 
* Fix retrieve_service

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix retrieve_collection

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-02 19:18:33 +08:00
Cai Yudong
d5cd561449
Add metrics for master (#5515) 
* metric

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* metric

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add MasterDDChannelTimeTick for metrics

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add metrics for all master grpc

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* register metrics

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add metrics register

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add comments for codacy check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix format issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-01 11:04:31 +08:00
yukun
48f35ffa97
Add publishRetrieveResult (#5501) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-05-31 18:08:32 +08:00
godchen
918458a1be
Add msgstream seek (#5483) 
* Add msgstream seek

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change test name

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove file

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* resolve conversation

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-31 17:46:30 +08:00
Cai Yudong
5ac726b7b9
use NewTicker instead of NewTimer (#5503) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-31 14:44:30 +08:00
Cai Yudong
c468481a36
Optimize MqTtMsgStream and unittest (#5498) 
* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add getTsMsgFromConsumerMsg

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename some variables for better readability

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add chanWaitGroup

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add chanTtMsgTime

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename lastTimeStamp to currTimeStamp

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add comments

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* support unsynced ttMsg

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add TestStream_PulsarTtMsgStream_3

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove debug log

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* change ttMsg logic back

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix ci block issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix codacy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-31 11:31:30 +08:00