1356 Commits

Author SHA1 Message Date
shengjun.li
cecbeacc42
fix annoy l2 (#3235)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-12 19:18:40 +08:00
XuanYang-cn
a891c5beee
Optimize grpc download & build in CMakeLists (#3207)
* change thirdparty make args

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add compile option to googletest

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add ccache to gtest

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* rearrange grpc download &build

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix grpc download repeat problem

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix cannot find openblas.so.0 error

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-08-12 08:44:52 +00:00
yukun
92e2a26a78
Fix test_query_range_valid_ranges bug (#3224)
* Fix TestSearchDSL level 2 bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix QueryTest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add annotation in milvus.proto

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix CreateIndex in C++ sdk

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix C++ sdk range test

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_search_ip_index_partitions

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_query_range_valid_ranges

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix GetCollectionInfo

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-08-12 14:29:33 +08:00
BossZou
3735e3d19d
Improve MysqlEngine (#3227)
* Mysql

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* .

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* MysqlEngine thread aware

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Update case

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Optimize mysqlengine threads

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
2020-08-12 14:18:24 +08:00
chen qingxiang
3ec8c382d6
fix compact bug and change some test case level from 2 to 1 (#3219)
* fix compact bug and change some test case level from 2 to 1

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* change wrong test case

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* fix ping test case bug

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2020-08-12 10:56:33 +08:00
Cai Yudong
18d67772fd
enable -Werror to improve code quality (#3221)
* clean warning

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update faiss make options

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove wrong file

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-11 14:16:42 +00:00
yukun
0b25f4c015
Fix CreateIndex bug in C++ sdk (#3216)
* Fix TestSearchDSL level 2 bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix QueryTest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add annotation in milvus.proto

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix CreateIndex in C++ sdk

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix C++ sdk range test

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_search_ip_index_partitions

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-08-11 17:16:54 +08:00
quicksilver
8c6533a4bb
Enable/Disable output of compile commands for clang-tidy (#3214)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-08-11 11:43:57 +08:00
chen qingxiang
202a8b7244
fix GetEntityBug and part of test case bug (#3200)
* fix GetEntityBug and part of test case bug

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* fix test case bug

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2020-08-11 02:46:59 +00:00
BossZou
70cab9bed5
Fix compact function pass parameters error (#3196)
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
2020-08-11 09:36:32 +08:00
yukun
773236b93e
Fix TestSearchDSL level 2 bugs (#3185)
* Fix TestSearchDSL level 2 bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix QueryTest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add annotation in milvus.proto

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-08-10 15:56:58 +08:00
BossZou
d30f4c8585
Fix sqlite insert error (#3191)
* Fix sqlite transaction error

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Update header reference in ut

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
2020-08-10 15:56:42 +08:00
groot
22ea1e3423
clean unused code (#3190)
Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-10 13:58:45 +08:00
chen qingxiang
eb4eea6251
add getEntity unittest (#3183)
* add getEntity ut

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* add getEntity unittest

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* change getEntity ut uid field name

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-10 09:42:49 +08:00
XuPeng-SH
c0f77a7e47
(db/snapshot): retry remove if remove_all fails due to bug in filesystem remove_all (#3187)
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-08-09 23:19:30 +08:00
BossZou
d37e0a39e5
Make Memsegment create segment explicitly (#3181)
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-08-09 08:46:14 +00:00
op-hunter
ab5a48cfa6
revert hnsw to version 0.10.0 (#3184)
* revert hnsw to version 0.10.0

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-09 00:17:37 +08:00
chen qingxiang
01b8057c63
fix the bug cpu cache not make effect (#3178)
* fix the bug not make effect

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* code format

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* change test case level

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* change test case level

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* change test case level

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2020-08-08 20:35:18 +08:00
XuPeng-SH
36d512da81
(db/snapshot): replace boost filesystem with std::experimental filesystem (#3166)
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-08-08 16:23:08 +08:00
shengjun.li
fd92afc5ed
fix metric type (#3179)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-08 14:17:44 +08:00
chen qingxiang
f9e8c16e29
fix bug caused by bloomfilter file not release (#3173)
* fix bug caused by bloomfilter file not release

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* fix bug caused by apply delete

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* cancel annotation

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-08 09:41:33 +08:00
yukun
237e909e7c
Fix test_search.py::TestSearchDSL bugs (#3170)
* Fix dsl test case nb bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix dsl test case bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add metric_type judge in search

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_search.py

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_db

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix search metric_type

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* ci retry

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_search.py::TestSearchDSL bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-07 20:36:12 +08:00
groot
eca7d3c90c
fix merge hang issue (#3171)
* fix merge hang issue

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* index type case sensitive

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-07 18:14:05 +08:00
quicksilver
db6d4d5b9d
Use ccache with Make (#3164)
* using ccache if defined

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* using ccache if defined

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* using ccache if defined

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* using ccache if defined

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* using ccache if defined

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* using ccache if defined

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* using ccache if defined

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* using ccache if defined

Signed-off-by: jeffoverflow <zhifeng.zhang@zilliz.com>

* call "ccache -z" before build start, call "ccache -s" after build finish

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-08-07 16:13:49 +08:00
groot
15a07951ab
fix build index crash (#3167)
* fix build index crash

Signed-off-by: yhmo <yihua.mo@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-07 14:08:13 +08:00
BossZou
c05f67baf2
Fix delete crash when delete multi time (#3163)
* Fix Delete Bug when delete multi time

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Fix multi delete crash bug

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Add filter field when get active collection

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Fix cmake link error & code format

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
2020-08-07 14:07:45 +08:00
shengjun.li
186f36c794
fix metric type (#3158)
* fix metric type

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix config

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix query

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-07 11:56:01 +08:00
chen qingxiang
aca1aeb5ec
fix bug casued by merge and drop at the same time (#3160)
* add log message

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* fix bug casued by merge and drop at the same time

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2020-08-07 09:37:41 +08:00
XuPeng-SH
596808a376
(db/snapshot): fix DeleteEntity bug (#3165)
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-08-07 09:37:26 +08:00
groot
ce1e528d1c
fix delete entity test case (#3162)
* fix delete eneity bug

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix delete entity test case

Signed-off-by: yhmo <yihua.mo@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-07 09:33:08 +08:00
op-hunter
1042f2e1f1
Remove useless index (#3157)
* update change log and make clang-format

Signed-off-by: cmli <chengming.li@zilliz.com>

* remove sq8nr_nm

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* update changelog

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: sahuang <xiaohai.xu@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-06 19:29:57 +08:00
yukun
cbba262442
Fix metric_type bug in search (#3155)
* Fix dsl test case nb bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix dsl test case bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add metric_type judge in search

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_search.py

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_db

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix search metric_type

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* ci retry

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-06 19:08:42 +08:00
XuanYang-cn
38a3fe766d
Move gtest download and build process to thirdparty/gtest (#3145)
* fix cmake configure warning

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add config tests to ctest

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* rm customed add_test, using default add_test

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add thirdparty/gtest and built-able

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add dependences fiu

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add license, rm not useful comments

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix include dir not exist bug

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add some config

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* gtest download at configure time

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-06 19:07:20 +08:00
groot
d22ff25a35
fix delete eneity bug (#3161)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-08-06 18:18:25 +08:00
groot
c73a58a114
add unittest case FetchTest (#3153)
* add unittest cases

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* add unittest case

Signed-off-by: yhmo <yihua.mo@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-06 14:07:11 +08:00
op-hunter
7688f51343
Faiss hnsw upgrade (#3134)
* combine the hnsw's implemention of faiss and hnswlib

Signed-off-by: cmli <chengming.li@zilliz.com>

* transplant the datastructure of hnsw from hnswlib 2 faiss

Signed-off-by: cmli <chengming.li@zilliz.com>

* basic work finished, pass compile, to be tested

Signed-off-by: cmli <chengming.li@zilliz.com>

* rhnswflat, rhnswsq, rhnswpq pass ut

Signed-off-by: cmli <chengming.li@zilliz.com>

* remove AssertAnns of RHNSWPQ because PQ has accuracy loss

Signed-off-by: cmli <chengming.li@zilliz.com>


Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-06 11:36:41 +08:00
yukun
ef24a8b340
Fix dsl test cases bug (#3130)
* Fix dsl test case nb bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix dsl test case bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add metric_type judge in search

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_search.py

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_db

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-08-05 21:44:17 +08:00
groot
2b2ebc27e5
fix auto-generated id field bug (#3137)
* autogen id

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix bug

Signed-off-by: yhmo <yihua.mo@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-05 18:19:51 +08:00
BossZou
fc393b3aeb
(scalar) Debug delete request (#3107)
* Debug delete request

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Chang previous bloom filter and deleted docs file to stale during ApplyDeletes

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Add delete case

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Add delete ut

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Add Delete test

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Format

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-05 16:27:58 +08:00
chen qingxiang
aad8f82d41
fix index bug caused by json parse (#3129)
* fix index bug caused by json parse

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2020-08-05 16:24:47 +08:00
groot
3dfbe61422
modify proto (#3126)
* modify proto

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix compact

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* print cache

Signed-off-by: yhmo <yihua.mo@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-05 14:02:51 +08:00
groot
362bf7a94e
cache (#3124)
* cache

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-05 10:04:41 +08:00
Wang Xiangyu
f18c95eed3
update grpc commit (#3125)
Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-05 09:44:10 +08:00
yukun
b12a19df34
Fix bug (#3121)
* Fix Block Format Read bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix Search crash bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix CreateCollection bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add db->Query unittest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix index name bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_db error

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-08-04 18:29:50 +08:00
Wang Xiangyu
7861da3e92
[skip ci]update grpc commit and md5 (#3123)
Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-04 18:22:33 +08:00
BossZou
9aa364ce25
Optimize DeletedDocsFormat class file I/O (#3112)
* Optimize DeletedDocsFormat file I/O

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Update DeletedDocsFormat Read

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-04 15:45:44 +08:00
yukun
96d799655a
Add DBImpl::Query unittest (#3119)
* Fix Block Format Read bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix Search crash bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix CreateCollection bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add db->Query unittest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-08-04 15:00:15 +08:00
groot
1f705a2096
define binary data type (#3115)
* define binary data type

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-04 14:58:18 +08:00
Xiaohai Xu
19c5dc079f
add logging (#3092)
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-04 12:09:15 +08:00
groot
8f8fa0aaf2
use index type replace name (#3114)
* use index type replace name

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-08-04 10:45:41 +08:00