1403 Commits

Author SHA1 Message Date
XuanYang-cn
144523b480
Optimize download & compile of Prometheus (#3357)
* runable prometheus

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add copyright

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-08-20 11:13:16 +00:00
Cai Yudong
fb70337c74
fix clang tidy warnings (#3362)
* fix codacy warning

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update codacy rules

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update index unittest build option

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-08-20 18:34:44 +08:00
Xiaohai Xu
b437639b7a
#3309 Error should be raised if search binary collection with metric_type: L2 (#3349)
* add check

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-08-20 18:22:09 +08:00
Wang Xiangyu
1938d6b769
#3265 clear finished task to release some resource (#3360)
Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-20 17:40:46 +08:00
Wang Xiangyu
e24bf8b854
Refactor directory access check and add config system.lock.enable (#3354)
* refactor directory access and add config system.lock.enable

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>

* update namespace

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-20 17:37:49 +08:00
chen qingxiang
f51959bb76
Add a feature of getting milvus all configs (#3347)
* change the header define to make it standardized

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* add a feature of getting milvus all configs

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* change the position of header json

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* remove useless line

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* fix a magic bytes write bug and format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2020-08-20 16:00:18 +08:00
groot
0f191bcb8f
Implement wal (#3345)
* implement wal

Signed-off-by: groot <yihua.mo@zilliz.com>

* wal unittest

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* add unittest

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-08-20 14:08:07 +08:00
XuanYang-cn
157f24b454
Download path can be configured (#3350)
* download path configuration

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add -f to build.sh

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* change mysql cxx_flags to aviod print Wdeprecated warnings

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-08-20 04:46:46 +00:00
shengjun.li
1615e948cb
fix hamming (#3342)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-20 03:21:39 +00:00
Wang Xiangyu
2a6f797c29
refactor Server::Start() (#3343)
* refactor Server::Start()

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>

* remove unused file

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-20 09:41:56 +08:00
Cai Yudong
6cb65a3a19
fix clang-tidy warnings (#3344)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-08-20 09:37:57 +08:00
XuanYang-cn
fd7c25546e
Optimize download & compile of mysqlpp (#3337)
* runable mysqlpp

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* rm not using comments

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix format

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-08-19 15:30:10 +00:00
BossZou
81791fc803
Fix milvus crash when exiting (#3333)
* Fix server crash when exiting

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Simplify web server

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
2020-08-19 15:17:24 +00:00
Cai Yudong
05a602fecc
fix clang tidy warnings (#3340)
* update GrpcRequestHandler namespace

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-tidy warnings

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optmize clang-tidy rules and clean warnings

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-08-19 22:25:35 +08:00
cqy123456
17ad160e4c
Master codacy v3 (#3322)
* codacy fix

Signed-off-by: cqy <yaya645@126.com>

* codacy

Signed-off-by: cqy <yaya645@126.com>

* codacy

Signed-off-by: cqy <yaya645@126.com>

* codacy check

Signed-off-by: cqy <yaya645@126.com>

* codacy

Signed-off-by: cqy <yaya645@126.com>

* codacy

Signed-off-by: cqy <yaya645@126.com>
2020-08-19 12:35:43 +00:00
yukun
68a2918c30
Add WebServer unittest (#3321)
* Add web server interface

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add unittest/server

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add web server ut

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-08-19 18:52:36 +08:00
Cai Yudong
729c6ebdc5
update clang-tidy config (#3314)
* update clang-tidy config

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update .clang-tidy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update clang-tidy config

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update clang-tidy HeaderFilterRegex

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-08-19 16:42:32 +08:00
XuanYang-cn
7b7f62820d
reduce grpc download file size from 380M to 130M (#3326)
* reduce grpc download file size from 380M to 130M

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix bug

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-08-19 08:26:31 +00:00
XuanYang-cn
1cac6388f3
fix include directories not find bug (#3323)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-08-19 08:26:24 +00:00
chen qingxiang
fe905a0062
add file header and tail implementation (#3316)
* add file header and tail implementation

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* remove files and formate code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* fix bugs

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* fix bug in getEntityById

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* remove useless lines and add zip md5 cheksum

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* change crc32c thirdparty reposity

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2020-08-19 08:17:23 +00:00
Xiaohai Xu
07ece74b70
#3295 Server crashed during creating index (#3307)
* fix binary case

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix validation

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* exception by reference

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-08-19 15:43:41 +08:00
quicksilver
55f47defdf
Fix clang-tidy bug (#3324)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-08-19 04:41:39 +00:00
Wang Xiangyu
8e8169d3aa
refactor LogMgr init (#3313)
Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-18 22:50:24 +08:00
Cai Yudong
46b68e2096
fix codacy warnings (#3308)
* improve code quality

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix code quality warnings

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* use vector reference instead

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* clean codacy warnings

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* clean codacy warnings

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-08-18 13:02:24 +00:00
XuanYang-cn
b045200c23
Optimize thirdparty libunwind and gperftools download & compile processes (#3305)
* runable gperftools

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add imported global to targets gperftools

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* optimized runable

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* thirdparty download file path optimize

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* optimize format

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* rm old profilers building

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* change target gperftools to tcmalloc_and_profiler.a

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-08-18 12:48:06 +00:00
Wang Xiangyu
5ab3bf887f
Update compile option with fiu (#3293)
* always disable fiu in milvus_server and enable in unittest, make it work in config directory

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>

* always enable fiu to keeping behavior

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>

* always enable fiu to keeping behavior

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-18 14:09:26 +08:00
groot
aa2fe7263d
prepare for wal (#3297)
* prepare for wal

Signed-off-by: groot <yihua.mo@zilliz.com>

* prepare for wal

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-08-18 06:07:35 +00:00
shengjun.li
2f6920f840
conf build_index_threshold (#3284)
* conf build_index_threshold

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* add member

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* add log

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-18 04:35:50 +00:00
Cai Yudong
89ca954e19
fix storage clang tidy warnings (#3285)
* update storage interface Read

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable storage unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* use THROW_ERROR in more places

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename read() to Read()

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update storage interface names

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* roll back to void Read()

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-08-18 03:17:25 +00:00
BossZou
788b015667
Check stale segment file when flush delete ids (#3283)
* Check stale file modified

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Modify test

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Add more detail for debuging

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Change del ids type

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Remove surplus code

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Allow to delete id-deulicated entities

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
2020-08-17 12:35:10 +00:00
cqy123456
b8e96a7fb9
Master fix (#3279)
* clang-tiny fix

Signed-off-by: cqy <yaya645@126.com>

* clang-tiny fix

Signed-off-by: cqy <yaya645@126.com>

* clang-tiny fix

Signed-off-by: cqy <yaya645@126.com>

* clang-tiny fix

Signed-off-by: cqy <yaya645@126.com>

* clang-tiny fix

Signed-off-by: cqy <yaya645@126.com>
2020-08-17 11:08:32 +00:00
XuanYang-cn
317ec86d31
Optimize download&compilation of opentracing (#3276)
* opentracing runable

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* optimize opentracing

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* rm opentracing from Thirdpartypackages.cmake

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* rm dup links

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fit unittest cann't compile bug

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix format

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-08-17 07:23:54 +00:00
Wang Xiangyu
ddd4c42b09
[skip ci]fix typo (#3275)
Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-17 11:19:05 +08:00
BossZou
e3f4ab92a7
Upgrade third-party oatpp up to v1.1.0 (#3272)
* Update oatpp link

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Update DATo

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Upgrade oatpp to v1.1.0 (fix #2568)

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Fix complie error in GPU version

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
2020-08-17 11:17:10 +08:00
shengjun.li
9554a38614
fix gpu (#3270)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-15 19:41:35 +08:00
groot
f61445602c
prepare for wal (#3259)
* prepare for wal

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-08-15 15:42:09 +08:00
XuanYang-cn
7ec3ab25e6
remove ZLIB from our build tree (#3264)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-08-15 06:47:40 +00:00
XuanYang-cn
8eff438a61
rm multiple url for yaml-cpp (#3260)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-08-15 06:47:34 +00:00
yukun
4beb05499d
Add web server interface (#3257)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-08-14 17:30:59 +08:00
groot
6f5be4b54f
remove wal logic (#3245)
* remove wal logic

Signed-off-by: groot <yihua.mo@zilliz.com>

* validate insert data size

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* preload collections

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine code

Signed-off-by: groot <yihua.mo@zilliz.com>

* avoid test fail

Signed-off-by: groot <yihua.mo@zilliz.com>

* db proxy

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-08-14 17:30:24 +08:00
XuanYang-cn
ae5f518f52
Optimize yaml-cpp download and build process (#3252)
* modify  yaml

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* config dependency altering

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* modify format

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-08-14 07:43:21 +00:00
op-hunter
cd38d607f0
fix wrong arguments check of hnsw-pq (#3247)
Signed-off-by: cmli <chengming.li@zilliz.com>
2020-08-14 11:53:56 +08:00
XuanYang-cn
3913cc0fc4
Fix compile option of thirdparty libs (#3246)
* fix some bugs

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* rm not using -f

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-08-13 09:43:22 +00:00
chen qingxiang
6f6a1bfd82
fix cmd req bug and fix test case bug (#3241)
* [skip ci]fix test case bug

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* cancel change config

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* fix Cmd req bug and change config test case

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* change test case in config and throw inmutable params problem

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2020-08-13 06:14:50 +00:00
Cai Yudong
967c971a52
[skip ci]update clang-tidy rules (#3239)
* add more clang-tidy rules

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix typo

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* [skip ci] update build.sh

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-08-13 11:55:17 +08:00
shengjun.li
abe2f21a84
rm find faiss (#3242)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-13 09:31:59 +08:00
groot
997a5b8700
implement loadcollection (#3229)
* implement loadcollection

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine code

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-13 00:08:26 +08:00
shengjun.li
cecbeacc42
fix annoy l2 (#3235)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-12 19:18:40 +08:00
XuanYang-cn
a891c5beee
Optimize grpc download & build in CMakeLists (#3207)
* change thirdparty make args

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add compile option to googletest

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add ccache to gtest

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* rearrange grpc download &build

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix grpc download repeat problem

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix cannot find openblas.so.0 error

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-08-12 08:44:52 +00:00
yukun
92e2a26a78
Fix test_query_range_valid_ranges bug (#3224)
* Fix TestSearchDSL level 2 bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix QueryTest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add annotation in milvus.proto

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix CreateIndex in C++ sdk

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix C++ sdk range test

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_search_ip_index_partitions

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_query_range_valid_ranges

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix GetCollectionInfo

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-08-12 14:29:33 +08:00