16 Commits

Author SHA1 Message Date
Cai Yudong
dc707bb8a3
Caiyd 1858 disable s3 build (#1860)
* #1858 disable s3 build

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable s3 in unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-03 16:35:48 +08:00
Cai Yudong
588ef95d76
Caiyd refactor knowhere (#1687)
* add new knowhere

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* build pass

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>

* update

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* rename algo

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update...

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>

* add archive

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* add new APIs: GetVectorById/SearchVectorById/SetBlacklist/GetBlacklist

* update unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* all unittest pass

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* test_binary pass

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move knowhere into namespace milvus

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update hnsw

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ConfAdapterMgr

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ExecutionEngineImpl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move SetBlacklist/GetBlacklist to VecIndex

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update VectorAdapter and rename SearchById to QueryById

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update interface in ExecutionEngineImpl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* milvus build pass

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* milvus IDMAP sdk_simple pass

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix test_server

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix test_schedule

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix CPU version Milvus build issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update BinaryIVF BuildAll

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update VecIndexFactory

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ConfAdapter

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix knowhere unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix SPTAG unittest

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix CPU version unittest build issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix db_test

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix test_engine

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix test_delete

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix CPU version build issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* change BinarySet key back

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* change IndexType to string, and add compatible API for 0.7.0

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix IndexHNSW build warning

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix test_cache

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix write_index error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code clean

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: Nicky <nicky.xj.lin@gmail.com>
Co-authored-by: xiaojun.lin <xiaojun.lin@zilliz.com>
2020-03-19 19:24:20 +08:00
yudong.cai
6b81dd55df #1536 move Config.cpp to config directory
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-07 17:02:21 +08:00
Cai Yudong
636f5c9cb6
#1234 do service check when Milvus startup (#1404)
* #1234 do service check when Milvus startup

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1234 update error log

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-02-28 10:05:30 +08:00
Jin Hai
e1027e9e65
#1240 Update license declaration of each file (#1241)
* #1240 Update license declaration of each files

Signed-off-by: jinhai <hai.jin@zilliz.com>

* #1240 Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-02-17 23:40:58 +08:00
BossZou
a0faf1a786 Add http server (#956)
* refactoring(create_table done)

* refactoring

* refactor server delivery (insert done)

* refactoring server module (count_table done)

* server refactor done

* cmake pass

* refactor server module done.

* set grpc response status correctly

* format done.

* fix redefine ErrorMap()

* optimize insert reducing ids data copy

* optimize grpc request with reducing data copy

* clang format

* [skip ci] Refactor server module done. update changlog. prepare for PR

* remove explicit and change int32_t to int64_t

* add web server

* [skip ci] add license in web module

* modify header include & comment oatpp environment config

* add port configure & create table in handler

* modify web url

* simple url complation done & add swagger

* make sure web url

* web functionality done. debuging

* add web unittest

* web test pass

* add web server port

* add web server port in template

* update unittest cmake file

* change web server default port to 19121

* rename method in web module & unittest pass

* add search case in unittest for web module

* rename some variables

* fix bug

* unittest pass

* web prepare

* fix cmd bug(check server status)

* update changlog

* add web port validate & default set

* clang-format pass

* add web port test in unittest

* add CORS & redirect root to swagger ui

* add web status

* web table method func cascade test pass

* add config url in web module

* modify thirdparty cmake to avoid building oatpp test

* clang format

* update changlog

* add constants in web module

* reserve Config.cpp

* fix constants reference bug

* replace web server with async module

* modify component to support async

* format

* developing controller & add test clent into unittest

* add web port into demo/server_config

* modify thirdparty cmake to allow build test

* remove  unnecessary comment

* add endpoint info in controller

* finish web test(bug here)

* clang format

* add web test cpp to lint exclusions

* check null field in GetConfig

* add macro RETURN STATUS DTo

* fix cmake conflict

* fix crash when exit server

* remove surplus comments & add http param check

* add uri /docs to direct swagger

* format

* change cmd to system

* add default value & unittest in web module

* add macros to judge if GPU supported

* add macros in unit & add default in index dto & print error message when bind http port fail

* format (fix #788)

* fix cors bug (not completed)

* comment cors

* change web framework to simple api

* comments optimize

* change to simple API

* remove comments in controller.hpp

* remove EP_COMMON_CMAKE_ARGS in oatpp and oatpp-swagger

* add ep cmake args to sqlite

* clang-format

* change a format

* test pass

* change name to

* fix compiler issue(oatpp-swagger depend on oatpp)

* add & in start_server.h

* specify lib location with oatpp and oatpp-swagger

* add comments

* add swagger definition

* [skip ci] change http method options status code

* remove oatpp swagger(fix #970)

* remove comments

* check Start web behavior

* add default to cpu_cache_capacity

* remove swagger component.hpp & /docs url

* remove /docs info

* remove /docs in unittest

* remove space in test rpc

* remove repeate info in CHANGLOG

* change cache_insert_data default value as a constant

* [skip ci] Fix some broken links (#960)

* [skip ci] Fix broken link

* [skip ci] Fix broken link

* [skip ci] Fix broken link

* [skip ci] Fix broken links

* fix issue 373 (#964)

* fix issue 373

* Adjustment format

* Adjustment format

* Adjustment format

* change readme

* #966 update NOTICE.md (#967)

* remove comments

* check Start web behavior

* add default to cpu_cache_capacity

* remove swagger component.hpp & /docs url

* remove /docs info

* remove /docs in unittest

* remove space in test rpc

* remove repeate info in CHANGLOG

* change cache_insert_data default value as a constant

* adjust web port cofig place

* rename web_port variable

* set advanced config name add DEFAULT

Co-authored-by: jielinxu <52057195+jielinxu@users.noreply.github.com>
Co-authored-by: JackLCL <53512883+JackLCL@users.noreply.github.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
2020-01-13 13:55:48 +08:00
Cai Yudong
b9d9b8ecc2 support minio storage (#895)
* #815 add S3 client

* #815 update S3 interface

* #815 add bucket_name_ as class field, add StartService() and StopService()

* #815 add FileIOReader and FileIOWriter

* #815 fix unittest test_storage build error

* #815 add GetObjectStr()

* #815 add S3IOReader/S3IOWriter

* #815 add ListObjects(), support put string with \0, launch S3 service when server start

* #815 read/write s3 index file pass

* #815 add MinIO related config

* #815 fix storage config issuegs

* #815 code clean

* #815 use local file or S3 via minio_enable

* #815 add DeleteObjects() for S3

* #815 print out IO rate when read_index & write_index

* #815 clean code

* #815 fix centos build error

* #815 support MinIO storage

* #815 fix typo
2020-01-04 10:39:52 +08:00
Zhiru Zhu
e99896ac62 Add Opentracing (#729)
* add TracerUtil

* Interceptor ok

* add handler

* add context

* minor update

* keep span in trace context

* add span in search okay

* Update Context.cpp

* refactor

* refactor

* refactor

* format

* add context in SearchJob

* trace search okay

* add back finish span in interceptor

* add namespace

* add tracing config in server config

* add random id

* debug mode okay

* update CMakeLists

* add opentracing to cmake

* update unittest

* add tracing namespace

* remove std::run_time error

* add lock when erasing context_map

* update tracing config

* lint

* update CHANGELOG

* small fix

* fix server unit test

* fix scheduler unit test

* fix db unit test

* lint

* fix db unit test gpu version

* rename to tracing_config

* fix

* update

* trigger ci
2019-12-10 15:24:54 +08:00
yudong.cai
a023511f16 #611 remove MILVUS_CPU_VERSION 2019-11-29 16:02:08 +08:00
groot
2b805489a1 #485 Increase code coverage rate 2019-11-23 11:34:45 +08:00
groot
29586ba725 #353 Rename config.h.in to version.h.in 2019-11-15 19:06:14 +08:00
youny626
4ca6b59b91 add CPU version 2019-11-05 10:23:59 +08:00
starlord
e7876f6343 avoid relative header path
Former-commit-id: d4a01b720ac7011e17076b7c36a20af0ebe2b58a
2019-10-23 15:05:22 +08:00
starlord
d9138e1960 #80 Print version information into log during server start
Former-commit-id: 874acccc6856b18b9f5992a2ee7bf14382f0f3c6
2019-10-22 18:40:42 +08:00
yudong.cai
30534bee23 MS-653 fix when config check fail, Milvus close without message
Former-commit-id: b9ff8c2228d07b6abe9d72f87e06bee823a05363
2019-10-15 15:59:56 +08:00
starlord
41d37f9648 re-organize project
Former-commit-id: 898c2c8b0b2cfdc1b8664062aa0ea910a93566dd
2019-10-14 09:51:48 +08:00