1318 Commits

Author SHA1 Message Date
XuanYang-cn
dca6c6afde
Add unittest for BackGroundGC in DataNode (#6522)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-07-15 10:05:55 +08:00
dragondriver
5016e509be
Support wildcard match on search/query output fields (#6510)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-14 18:51:54 +08:00
neza2017
fac350cc58
fix tso (#6500)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-14 17:11:54 +08:00
dragondriver
7f99d9d714
Loose the limit of dimension when searching on a binary vector field (#6503)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-14 14:33:54 +08:00
cai.zhang
f469a315d6
Refactor the logic of assign tasks in IndexCoord (#6328)
* Refactor the logic of assign tasks in IndexCoord

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bugs

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug for unittest

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Add lock for map

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Improve code

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix unittest bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reduce duriation for timetick

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Update orm version

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reset sdk version

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reset orm version

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reset test ip

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug for unissued

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Rename some variables

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Use break instead of continue in select::case

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-14 14:15:55 +08:00
Cai Yudong
1c29b42434
change output_fields to out_fields_id for RetrieveRequest (#6506)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-14 13:11:54 +08:00
bigsheeper
e50d39dfe1
Query result get global sealed segments from ETCD (#6377)
* get global sealed segments from ETCD

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix race error

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-14 11:15:54 +08:00
cai.zhang
f3a5b46288
Fix bug for repeatedly create bucket in minio (#6477)
* Fix bug for repeatedly create bucket in minio

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Simplify code

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Add log

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-14 09:55:46 +08:00
Cai Yudong
724f10b9a0
Unify the usage of query and search (#6467)
Unify the usage of query and search

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-13 22:20:33 +08:00
neza2017
b66157358c
add more log for create collection (#6478)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-13 22:19:55 +08:00
dragondriver
d896e3119e
Fix 6419, check if num_rows is greater than zero in proxy (#6439)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-13 16:33:55 +08:00
dragondriver
25b8c37383
Fix 6421, split raw vectors to serveral smaller one (#6449)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-13 16:05:36 +08:00
congqixia
bc15b74244
Handle datasorter edge cases (#6465)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-07-13 14:37:54 +08:00
xaxys
e5f8a77a2d
Extend grammar of boolean expression (#6460)
* Extend grammar of boolean expression

Signed-off-by: xaxys <tpnnghd@163.com>
2021-07-13 14:37:03 +08:00
xige-16
2b720fd2f7
Fix load hang when restart querynode many time in short time (#6351)
* fix load hang when restart querynode many time in short time

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add multi queryNode ut

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add ut for restart querynode

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* set queryCoord contex to load collection

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-07-13 14:16:00 +08:00
neza2017
d4239f0cd8
masterservice -> rootcoord (#6447)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-13 09:37:40 +08:00
sunby
83be910baa
Refactor segment manager (#6441)
issue: #6440
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-12 17:24:25 +08:00
sunby
a8e5fd2024
Refactor cluster in dataservice (#6356)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-12 11:03:52 +08:00
dragondriver
bd317a5461
Check if field data is empty (#6404)
* Check if field data is empty

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* FixBug: fast fail when output_fields contain fields that are not in schema

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-11 19:39:52 +08:00
neza2017
8b89744c6b
search with exp (#6409)
* process float vector and binary vector field
on the search result

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add more debug info on etcd

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add debug info when grpc failed

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-10 10:21:52 +08:00
FluorineDog
99ed122d11
Remove Dead Code, use signed type (#6398)
* make type signed

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* remove dead code

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* remove code

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-09 18:25:11 +08:00
FluorineDog
2ff072ea7b
Use larger dim for binary test to avoid false positive test failed (#6416)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-09 18:12:40 +08:00
groot
19fc42c9eb
calc_distance typo (#6380)
* calc_distance typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* calc_distance typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-07-09 14:53:52 +08:00
godchen
346e9cba4a
Add minio FGet method (#6386)
* Add minio FGet method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change el append

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-09 11:44:40 +08:00
zhenshan.cao
6b4b71bf03
Fixbug: remove wrong check code (#6372)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-08 20:17:43 +08:00
zhenshan.cao
b53699b846
Fixbug: distinguish the name of the subscription based on ProxyID (#6333)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-08 19:52:39 +08:00
XuanYang-cn
f0a68704e2
Add unittests for getChannelName in DataNode (#6346)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-07-08 15:52:40 +08:00
FluorineDog
d972a0361d
Weaken debug message (#6324)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-08 15:50:57 +08:00
quicksilver
7d920e9c9c
Fix compile error on CentOS (#6359)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-07-08 15:43:45 +08:00
yukun
33162ca40f
Fix query error message (#6344)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-07-08 15:05:52 +08:00
Cai Yudong
3387b07dfd
Optimize code under storage (#6335)
* rename AddOneStringToPayload/GetOneStringFromPayload to AddStringToPayload/GetStringFromPayload

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename print_binglog_test to print_binlog_test

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update chap08_binlog.md

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* use SetEventTimestamp() to replace SetStartTimestamp() and SetEndTimestamp()

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename AddStringToPayload/GetStringFromPayload to AddOneStringToPayload/GetOneStringFromPayload

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-07 19:10:07 +08:00
sunby
017484e093
Refactor meta in data coord (#6300)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-07 14:02:01 +08:00
shengjun.li
a1ba9e9dcd
Fix comile on CentOS (#6334)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-07-07 12:34:02 +08:00
dragondriver
166548447b
Fix 6249, use more clear error message (#6316)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-06 14:40:04 +08:00
FluorineDog
97b1426666
fix benchmark, add support for primary key in schemaHelper (#6281)
* fix benchmark

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* fix tests

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-06 09:50:01 +08:00
sunby
836a45ec26
Remove sending newSegmentMsg to rootcoord (#6301)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-06 09:24:05 +08:00
Cai Yudong
21a9bcd844
Create default partition when receive CreateCollectionMsg (#6305)
* create default partition when create collection

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* adjust sleep time for unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-06 09:16:03 +08:00
shengjun.li
1f42b4ec9d
fix the case that knng is not full (#6308)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-07-05 17:26:07 +08:00
neza2017
ca4a562a72
simplify AddIndex, remove collID,PartID from parameters (#6291)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-05 10:08:02 +08:00
Cai Yudong
bc8392eb1e
Add collectionID and partitionID into SegmentIndexInfo (#6289)
* add collectionID and partitionID into SegmentIndexInfo

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add partID2SegID into meta table

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update DropIndex

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-03 20:00:40 +08:00
neza2017
f8fa025dcc
start background go routine to check if all the segments have indexed (#6290)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-03 17:54:25 +08:00
neza2017
111a24a49f
remove msgstream of segment info (#6280)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-03 14:36:18 +08:00
dragondriver
8dd2051a9a
Fix 5045, decrease the nb and nlist in unittest (#6282)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-02 18:20:14 +08:00
congqixia
76fd9ad339
Fix restart node logic (#6265)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-07-02 16:48:30 +08:00
sunby
07b0989628
Add GetFlushedSegments in data coordinator (#6253)
Send segment flush message by grpc and add a new interface
`GetFlushedSegments`

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-02 11:16:20 +08:00
xige-16
ddb027fb9e
clear proxy sarchStream when releaseCollection (#6258)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-07-02 10:40:13 +08:00
groot
a364d7303b
Implement calculate vectors distance in server (#6219)
* calc distance stage 3

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* calc distance stage 5

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* calc distance stage 6

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* code style

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-07-01 18:56:17 +08:00
godchen
2a07124885
Change default grpc retry config (#6233)
* Change default grpc retry config

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change timeout

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add retry code

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change time

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-01 15:24:17 +08:00
neza2017
6019c193fc
add grpc interface to process NewSegment And FlushSegmentCompleted (#6247)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-01 14:58:17 +08:00
congqixia
580e3a57cf
stop add new node into cluster before validated (#6241)
* stop add new new into cluster

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix startup & test cases

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-07-01 14:32:16 +08:00