5292 Commits

Author SHA1 Message Date
congqixia
e06d6d7cc6
Returns collection instead of copy to make leaderMutex work (#17206)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-25 15:20:00 +08:00
yah01
de0ba6d495
Fix GetQuerySegmentInfo() returns incorrect result after LoadBalance (#17190)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-25 15:17:59 +08:00
Ten Thousand Leaves
5e1e7a6896
Fix an issue that causes flush hanging (#17169)
issue: #16628
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-25 14:34:00 +08:00
congqixia
37d7d7baf8
Fix node state data race in querycoord (#17198)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-25 10:43:59 +08:00
Jiquan Long
4293b640b2
Remove confusing log in segment id assigner (#17195)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-05-25 10:26:00 +08:00
Jiquan Long
75ca64f8c7
Refine task type logs to string (#17196)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-05-25 03:08:00 +08:00
xige-16
a8829554eb
Merge multiple sets of expr's optypes in diff dirs (#17192)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-24 21:56:00 +08:00
zhenshan.cao
ec1103ca27
Simplify the merge logic of searchTask (#17194)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 21:27:59 +08:00
bigsheeper
a4ea2fb18a
Handle errors in DataNode and QueryNode flow graph (#17096)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-24 21:11:59 +08:00
Bingyi Sun
ffaead6ad9
Add load meta in hand off task. (#17179)
issue: #16842

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-05-24 18:24:00 +08:00
congqixia
a0cef3a0a2
Fix sessionutil log always datanode (#17184)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-24 16:05:59 +08:00
zhenshan.cao
d05c24a0ff
Delete unused code (#17182)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 14:49:59 +08:00
zhenshan.cao
2512e668f7
Update timestmap according to gracefultime for Bounded Consistency (#17171)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 12:05:59 +08:00
zhenshan.cao
ea8e1623cd
Acquire nq from placeholdergroup incase of nq is zero (#17168)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 10:05:58 +08:00
zhenshan.cao
cce533c246
Fix unitest fail, change default index slice to 16 (#17162)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-23 17:44:25 +08:00
zhenshan.cao
08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-23 16:41:58 +08:00
cai.zhang
3685ff9e0c
Mark indexing task to retry when minio occurs error (#17153)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-23 15:47:58 +08:00
Xiaofan
65269a4744
Remove rocksmq reader (#17149)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-23 13:55:59 +08:00
jaime
a20e0dfc74
Refactor kafka msgstream implementation (#17069)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-05-23 12:39:58 +08:00
Enwei Jiao
d28a2db46c
move arrow from storage to core (#17061)
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
2022-05-22 20:03:58 +08:00
Bingyi Sun
86728490a2
Fix partition not found (#17132)
issue: #16842

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-05-20 19:37:57 +08:00
Jiquan Long
e756fdccd4
Fix pool of lexer and parser (#17129)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-05-20 18:47:57 +08:00
yah01
7746a5b742
Add NodeIds field for QuerySegmentInfo (#17121)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-20 18:03:58 +08:00
congqixia
90ee23df22
Fix shardLeader cache concurrent access (#17120)
Fix  write map without mutex control
Also GetShards returns a copy of leader list instead of original one

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-20 14:33:57 +08:00
ownbylichaobao
2f8a7e7793
Support tls/ssl connection to etcd (#7703) (#17012)
Signed-off-by: lichaobao <1527563274@qq.com>
2022-05-20 12:29:19 +08:00
groot
d525e95550
Bulkload not allow duplicate numpy file (#17105)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-05-20 10:27:56 +08:00
Letian Jiang
70825a35cf
Make querynode LoadSegments interface idempotent (#17109)
This PR makes the following changes:
* separate LoadSegmentsTask into two phases: PreExecute and Execute
* filters out segments that are already loaded in PreExecute phase

Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-05-20 09:15:57 +08:00
congqixia
599763d9bf
Fix replicas info is not removed after release (#17111)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-19 20:05:57 +08:00
cai.zhang
e05cfe194e
Fix data race for indexcoord test (#17057)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-19 17:25:56 +08:00
Jiquan Long
276ba69819
Replace plan parser with antlr (#17090)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-05-19 17:15:57 +08:00
yah01
dcfe472586
Fix LoadBalance doesn't save the modification to replicas' shards (#17064)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-19 16:51:57 +08:00
congqixia
ae2864639e
Fix replica freeAll order (#17106)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-19 14:53:56 +08:00
congqixia
7eb6fcfb34
Limit the query node timetick log rate (#17065)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-19 10:19:56 +08:00
bigsheeper
9eeec4a2d5
Add collection load cache and InvalidateCollMetaCache by collID (#16882)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-19 10:13:56 +08:00
congqixia
b95681770b
Add deferred clean up for ShardCluster (#17094)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-18 21:51:56 +08:00
SimFG
cf08b5aa11
Fix 'TestChannelManager_StateTransfer' ut when etcd is slow (#17082)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-05-18 17:19:57 +08:00
Jiquan Long
99b4042f8c
Disable segment small index (#16935)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-05-18 15:27:56 +08:00
yah01
33c855dcd2
Fix LoadBalance doesn't remove the source nodes from segment (#17051)
If the triggerCondition isn't NodeDown, the removing won't happen.

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-18 11:55:56 +08:00
XuanYang-cn
127dd34b37
Make proxy use roundrobin to choose replica (#17063)
Fixes: #17055

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-05-17 22:35:57 +08:00
cai.zhang
b37b87eb97
The GetIndexFilePaths interface judges whether the index is complete (#17052)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-17 20:41:56 +08:00
Jiquan Long
77b0f290a8
Fix prefix match on sealed segment (#17050)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-05-17 20:15:55 +08:00
congqixia
5282397c1b
Fix VectorChunkManager cache enable parameter (#17058)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-17 17:37:55 +08:00
cai.zhang
d6bb86dd46
Return buildID when creating index if the index alreay exists (#17045)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-17 15:21:57 +08:00
xige-16
c2f1d65fa1
Print err info when insert msg not aligned (#16305)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-17 13:01:57 +08:00
yah01
960d35e517
Add MockCluster, make unit tests reliable (#17032)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-17 12:57:56 +08:00
XuanYang-cn
a1480aaf76
Clear shard leaders cache when releasing (#16942)
See also: #16926

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-05-17 11:11:56 +08:00
Xiaofan
5355153805
Minio Error is not Handled gracefully (#17003)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-16 19:25:55 +08:00
Xiaofan
327bd6abbe
Support Txn interface in rockskv (#17027)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-16 19:17:55 +08:00
xige-16
935b729a0c
Fix multiple deletes not taking effect (#17033)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-16 18:39:56 +08:00
congqixia
ffff291018
Fix Handoff log in shardCluster service (#17039)
Fix #17035

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-16 18:29:56 +08:00