From c2e190e82224692a7c3bf30ae6bf561f4fb575ed Mon Sep 17 00:00:00 2001 From: thywdy <56624359+thywdy@users.noreply.github.com> Date: Tue, 5 Nov 2019 09:58:15 +0800 Subject: [PATCH 001/103] [skip ci] add Japanese readme --- README_JP.md | 200 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 200 insertions(+) create mode 100644 README_JP.md diff --git a/README_JP.md b/README_JP.md new file mode 100644 index 0000000000..41ebf9b514 --- /dev/null +++ b/README_JP.md @@ -0,0 +1,200 @@ +![Milvuslogo](https://github.com/milvus-io/docs/blob/master/assets/milvus_logo.png) + + +![LICENSE](https://img.shields.io/badge/license-Apache--2.0-brightgreen) +![Language](https://img.shields.io/badge/language-C%2B%2B-blue) +[![codebeat badge](https://codebeat.co/badges/e030a4f6-b126-4475-a938-4723d54ec3a7?style=plastic)](https://codebeat.co/projects/github-com-jinhai-cn-milvus-master) + +- [Slack Community](https://join.slack.com/t/milvusio/shared_invite/enQtNzY1OTQ0NDI3NjMzLWNmYmM1NmNjOTQ5MGI5NDhhYmRhMGU5M2NhNzhhMDMzY2MzNDdlYjM5ODQ5MmE3ODFlYzU3YjJkNmVlNDQ2ZTk) +- [Twitter](https://twitter.com/milvusio) +- [Facebook](https://www.facebook.com/io.milvus.5) +- [Blog](https://www.milvus.io/blog/) +- [CSDN](https://zilliz.blog.csdn.net/) +- [中文官网](https://www.milvus.io/zh-CN/) + + +# Milvus へようこそ + +## 概要 + +Milvus は大規模な特徴ベクトルにむかう類似性検索エンジンです。不均質な計算アーキテクチャーに基づいて効率を最大化出来ます。数十億のベクタの中に目標を検索できるまで数ミリ秒しかかからず、最低限の計算資源だけが必要です。 + +Milvus は安定的な Python、Java 又は C++ APIsを提供します。 + +Milvus [リリースノート](https://milvus.io/docs/en/Releases/v0.5.0/)を読んで最新バージョンや更新情報を手に入れます。 + +- 不均質な計算アーキテクチャー + + Milvusは不均質な計算アーキテクチャーに基づいて効率を最大化出来ます。 + +- 様々な索引方法 + + Milvusはさまざまな索引方法を支えます。量子化、 木、 グラフに基づいて索引を作成できます。 + +- 知能てきな資源管理 + + Milvusはデータセットと利用できる資源を基づいて、自動的に検索アルゴリズムと索引作成方法を選びます。 + +- 水平拡張 + + Milvusはオンラインとオフラインに記憶域と計算を簡単な命令で拡張できます。 + +- 高い可用性 + + MilvusはKubernetes枠組みと統合するので、 単一障害点を避けられます。 + +- 高い互換性 + + Milvusはほぼ全ての深層学習モデルと主要なプログラミング言語と互換性があります。 + +- やすい使い方 + + Milvusは簡単にインストールできます。ベクタ以外のことを心配する必要がありますん。 + +- 視覚化てきな監視モード + + Prometheusに基づいてGUIでシステムの性能を監視出来ます。 + +## アーキテクチャー + +![Milvus_arch](https://github.com/milvus-io/docs/blob/master/assets/milvus_arch.png) + +## はじめに + +### ハードウェア要件 + +| コンポーネント | お勧めの配置 | +| --------- | ----------------------------------- | +| CPU | Intel CPU Haswell 以上 | +| GPU | NVIDIA Pascal series 以上 | +| RAM メモリ | 8 GB 以上 (データ規模に関わる) | +| ハードディスク | SATA 3.0 SSD 以上 | + +### Dockerでインストールする + +DockerでMilvusをインストールすることは簡単です。 [Milvusインストール案内](https://milvus.io/docs/en/userguide/install_milvus/) を参考してください。 + +### ソースから構築する + +#### ソフトウェア要件 + +- Ubuntu 18.04 以上 +- CMake 3.14 以上 +- CUDA 10.0 以上 +- NVIDIA driver 418 以上 + +#### コンパイル + +##### 1 依存コンポーネントをインストールする + +```shell +$ cd [Milvus sourcecode path]/core +./ubuntu_build_deps.sh +``` + +##### 2 構築する + +```shell +$ cd [Milvus sourcecode path]/core +$ ./build.sh -t Debug +or +$ ./build.sh -t Release +``` + +構築が完成するとき、 Milvusを実行するために必要なものは全てこのディレクトリにあります： `[Milvus root path]/core/milvus`。 + +#### Milvusサーバーを実行する + +```shell +$ cd [Milvus root path]/core/milvus +``` + +`lib/`　ディレクトリを　`LD_LIBRARY_PATH`　に添付する。 + +```shell +$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/milvus/lib +``` + +Milvusサーバーを実行する。 + +```shell +$ cd scripts +$ ./start_server.sh +``` + +Milvusサーバーを止めるために、次のコードを実行する： + +```shell +$ ./stop_server.sh +``` + +`conf/server_config.yaml` と `conf/log_config.conf` でMilvusをコンフィグするために、 [Milvusコンフィグ](https://github.com/milvus-io/docs/blob/master/reference/milvus_config.md)を読んでください。 + +### 初めてのMilvusプログラムを実行する + +#### Pythonサンプルコードを実行する + +[Python 3.5](https://www.python.org/downloads/)以上のバージョンがインストールされていることを確かめてください。 + +Milvus Python SDK　をインストールする。 + +```shell +# Install Milvus Python SDK +$ pip install pymilvus==0.2.3 +``` + +新しいファイル　`example.py`　を作成し、 [Pythonサンプルコード](https://github.com/milvus-io/pymilvus/blob/master/examples/AdvancedExample.py)を添付してください。 + +サンプルコードを実行する。 + +```shell +# Run Milvus Python example +$ python3 example.py +``` + +#### C++サンプルコードを実行する + +```shell + # Run Milvus C++ example + $ cd [Milvus root path]/core/milvus/bin + $ ./sdk_simple +``` + +#### Javaサンプルコードを実行する + +Java 8以上のバージョンがインストールされていることを確かめてください。 + +[このリンク](https://github.com/milvus-io/milvus-sdk-java/tree/master/examples)でサンプルコードを手に入れます。 + +## 貢献規約 + +本プロジェクトへの貢献に心より感謝いたします。 Milvusを貢献したいと思うなら、[貢献規約](CONTRIBUTING.md)を読んでください。 本プロジェクトはMilvusの[行動規範](CODE_OF_CONDUCT.md)に従います。プロジェクトに参加したい場合は、行動規範を従ってください。 + +[GitHub issues](https://github.com/milvus-io/milvus/issues/new/choose) を使って問題やバッグなとを報告しでください。 一般てきな問題なら, Milvusコミュニティに参加してください。 + +## Milvusコミュニティを参加する + +他の貢献者と交流したい場合は、Milvusの [slackチャンネル](https://join.slack.com/t/milvusio/shared_invite/enQtNzY1OTQ0NDI3NjMzLWNmYmM1NmNjOTQ5MGI5NDhhYmRhMGU5M2NhNzhhMDMzY2MzNDdlYjM5ODQ5MmE3ODFlYzU3YjJkNmVlNDQ2ZTk)に参加してください。 + +## Milvusロードマップ + +[ロードマップ](https://milvus.io/docs/en/roadmap/)を読んで、追加する予定の特性が分かります。 + +## 参考情報 + +[Milvus公式サイト](https://www.milvus.io) + +[Milvusドキュメント](https://www.milvus.io/docs/en/userguide/install_milvus/) + +[Milvusトレーニングセンター](https://github.com/milvus-io/bootcamp) + +[Milvusブロック](https://www.milvus.io/blog/) + +[Milvus CSDN](https://zilliz.blog.csdn.net/) + +[Milvusロードマップ](https://milvus.io/docs/en/roadmap/) + + +## ライセンス + +[Apache 2.0ライセンス](LICENSE) \ No newline at end of file From 1c8d27ec3833f29c9228fd1d42fd616f0e719315 Mon Sep 17 00:00:00 2001 From: thywdy <56624359+thywdy@users.noreply.github.com> Date: Tue, 5 Nov 2019 14:01:35 +0800 Subject: [PATCH 002/103] [skip ci] Update Japanese Readme --- README_JP.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README_JP.md b/README_JP.md index 41ebf9b514..abbbaf7e64 100644 --- a/README_JP.md +++ b/README_JP.md @@ -21,7 +21,7 @@ Milvus は大規模な特徴ベクトルにむかう類似性検索エンジン Milvus は安定的な Python、Java 又は C++ APIsを提供します。 -Milvus [リリースノート](https://milvus.io/docs/en/Releases/v0.5.0/)を読んで最新バージョンや更新情報を手に入れます。 +Milvus [リリースノート](https://milvus.io/docs/en/release/v0.5.1/)を読んで最新バージョンや更新情報を手に入れます。 - 不均質な計算アーキテクチャー @@ -143,7 +143,7 @@ Milvus Python SDK　をインストールする。 $ pip install pymilvus==0.2.3 ``` -新しいファイル　`example.py`　を作成し、 [Pythonサンプルコード](https://github.com/milvus-io/pymilvus/blob/master/examples/AdvancedExample.py)を添付してください。 +新しいファイル　`example.py`　を作成し、 [Pythonサンプルコード]( https://github.com/milvus-io/pymilvus/blob/master/examples/advanced_example.py)を添付してください。 サンプルコードを実行する。 @@ -197,4 +197,4 @@ Java 8以上のバージョンがインストールされていることを確 ## ライセンス -[Apache 2.0ライセンス](LICENSE) \ No newline at end of file +[Apache 2.0ライセンス](LICENSE) From 75d0f10784ecef21660ebfb920af8dbb3c4d3236 Mon Sep 17 00:00:00 2001 From: thywdy <56624359+thywdy@users.noreply.github.com> Date: Tue, 5 Nov 2019 20:11:59 +0800 Subject: [PATCH 003/103] [skip ci] Update Japanese Readme --- README_JP.md | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/README_JP.md b/README_JP.md index abbbaf7e64..a99c37a845 100644 --- a/README_JP.md +++ b/README_JP.md @@ -1,16 +1,12 @@ ![Milvuslogo](https://github.com/milvus-io/docs/blob/master/assets/milvus_logo.png) +[![Slack](https://img.shields.io/badge/Join-Slack-orange)](https://join.slack.com/t/milvusio/shared_invite/enQtNzY1OTQ0NDI3NjMzLWNmYmM1NmNjOTQ5MGI5NDhhYmRhMGU5M2NhNzhhMDMzY2MzNDdlYjM5ODQ5MmE3ODFlYzU3YjJkNmVlNDQ2ZTk) ![LICENSE](https://img.shields.io/badge/license-Apache--2.0-brightgreen) ![Language](https://img.shields.io/badge/language-C%2B%2B-blue) [![codebeat badge](https://codebeat.co/badges/e030a4f6-b126-4475-a938-4723d54ec3a7?style=plastic)](https://codebeat.co/projects/github-com-jinhai-cn-milvus-master) - -- [Slack Community](https://join.slack.com/t/milvusio/shared_invite/enQtNzY1OTQ0NDI3NjMzLWNmYmM1NmNjOTQ5MGI5NDhhYmRhMGU5M2NhNzhhMDMzY2MzNDdlYjM5ODQ5MmE3ODFlYzU3YjJkNmVlNDQ2ZTk) -- [Twitter](https://twitter.com/milvusio) -- [Facebook](https://www.facebook.com/io.milvus.5) -- [Blog](https://www.milvus.io/blog/) -- [CSDN](https://zilliz.blog.csdn.net/) -- [中文官网](https://www.milvus.io/zh-CN/) +![Release](https://img.shields.io/badge/release-v0.5.1-yellowgreen) +![Release_date](https://img.shields.io/badge/release%20date-November-yellowgreen) # Milvus へようこそ From 1b0d4533798796b0fe68425507ac4de36ca4b8ea Mon Sep 17 00:00:00 2001 From: "yudong.cai" Date: Wed, 6 Nov 2019 11:17:19 +0800 Subject: [PATCH 004/103] #207 add more unittest for config set/get --- core/src/server/Config.cpp | 62 +++++-- core/src/server/Config.h | 6 + core/src/utils/StringHelpFunctions.cpp | 44 +++-- core/src/utils/StringHelpFunctions.h | 5 +- core/unittest/server/test_config.cpp | 215 +++++++++++++++++++++++-- core/unittest/server/test_util.cpp | 5 +- 6 files changed, 292 insertions(+), 45 deletions(-) diff --git a/core/src/server/Config.cpp b/core/src/server/Config.cpp index c567bc68ea..20bcb937e7 100644 --- a/core/src/server/Config.cpp +++ b/core/src/server/Config.cpp @@ -25,6 +25,7 @@ #include "config/YamlConfigMgr.h" #include "server/Config.h" #include "utils/CommonUtil.h" +#include "utils/StringHelpFunctions.h" #include "utils/ValidationUtil.h" namespace milvus { @@ -341,6 +342,11 @@ Config::ResetDefaultConfig() { return s; } + s = SetResourceConfigSearchResources(CONFIG_RESOURCE_SEARCH_RESOURCES_DEFAULT); + if (!s.ok()) { + return s; + } + s = SetResourceConfigIndexBuildDevice(CONFIG_RESOURCE_INDEX_BUILD_DEVICE_DEFAULT); if (!s.ok()) { return s; @@ -788,6 +794,17 @@ Config::GetConfigStr(const std::string& parent_key, const std::string& child_key return value; } +std::string +Config::GetConfigSequenceStr(const std::string& parent_key, const std::string& child_key, const std::string& delim) { + std::string value; + if (!GetConfigValueInMem(parent_key, child_key, value).ok()) { + std::vector sequence = GetConfigNode(parent_key).GetSequence(child_key); + server::StringHelpFunctions::MergeStringWithDelimeter(sequence, delim, value); + SetConfigValueInMem(parent_key, child_key, value); + } + return value; +} + Status Config::GetServerConfigAddress(std::string& value) { value = GetConfigStr(CONFIG_SERVER, CONFIG_SERVER_ADDRESS, CONFIG_SERVER_ADDRESS_DEFAULT); @@ -1011,8 +1028,9 @@ Config::GetResourceConfigMode(std::string& value) { Status Config::GetResourceConfigSearchResources(std::vector& value) { - ConfigNode resource_config = GetConfigNode(CONFIG_RESOURCE); - value = resource_config.GetSequence(CONFIG_RESOURCE_SEARCH_RESOURCES); + std::string str = GetConfigSequenceStr(CONFIG_RESOURCE, CONFIG_RESOURCE_SEARCH_RESOURCES, + CONFIG_RESOURCE_SEARCH_RESOURCES_DELIMITER); + server::StringHelpFunctions::SplitStringByDelimeter(str, CONFIG_RESOURCE_SEARCH_RESOURCES_DELIMITER, value); return CheckResourceConfigSearchResources(value); } @@ -1155,7 +1173,7 @@ Config::SetMetricConfigEnableMonitor(const std::string& value) { return s; } - SetConfigValueInMem(CONFIG_DB, CONFIG_METRIC_ENABLE_MONITOR, value); + SetConfigValueInMem(CONFIG_METRIC, CONFIG_METRIC_ENABLE_MONITOR, value); return Status::OK(); } @@ -1166,7 +1184,7 @@ Config::SetMetricConfigCollector(const std::string& value) { return s; } - SetConfigValueInMem(CONFIG_DB, CONFIG_METRIC_COLLECTOR, value); + SetConfigValueInMem(CONFIG_METRIC, CONFIG_METRIC_COLLECTOR, value); return Status::OK(); } @@ -1177,7 +1195,7 @@ Config::SetMetricConfigPrometheusPort(const std::string& value) { return s; } - SetConfigValueInMem(CONFIG_DB, CONFIG_METRIC_PROMETHEUS_PORT, value); + SetConfigValueInMem(CONFIG_METRIC, CONFIG_METRIC_PROMETHEUS_PORT, value); return Status::OK(); } @@ -1189,7 +1207,7 @@ Config::SetCacheConfigCpuCacheCapacity(const std::string& value) { return s; } - SetConfigValueInMem(CONFIG_DB, CONFIG_CACHE_CPU_CACHE_CAPACITY, value); + SetConfigValueInMem(CONFIG_CACHE, CONFIG_CACHE_CPU_CACHE_CAPACITY, value); return Status::OK(); } @@ -1200,7 +1218,7 @@ Config::SetCacheConfigCpuCacheThreshold(const std::string& value) { return s; } - SetConfigValueInMem(CONFIG_DB, CONFIG_CACHE_CPU_CACHE_THRESHOLD, value); + SetConfigValueInMem(CONFIG_CACHE, CONFIG_CACHE_CPU_CACHE_THRESHOLD, value); return Status::OK(); } @@ -1211,7 +1229,7 @@ Config::SetCacheConfigGpuCacheCapacity(const std::string& value) { return s; } - SetConfigValueInMem(CONFIG_DB, CONFIG_CACHE_GPU_CACHE_CAPACITY, value); + SetConfigValueInMem(CONFIG_CACHE, CONFIG_CACHE_GPU_CACHE_CAPACITY, value); return Status::OK(); } @@ -1222,7 +1240,7 @@ Config::SetCacheConfigGpuCacheThreshold(const std::string& value) { return s; } - SetConfigValueInMem(CONFIG_DB, CONFIG_CACHE_GPU_CACHE_THRESHOLD, value); + SetConfigValueInMem(CONFIG_CACHE, CONFIG_CACHE_GPU_CACHE_THRESHOLD, value); return Status::OK(); } @@ -1233,7 +1251,7 @@ Config::SetCacheConfigCacheInsertData(const std::string& value) { return s; } - SetConfigValueInMem(CONFIG_DB, CONFIG_CACHE_CACHE_INSERT_DATA, value); + SetConfigValueInMem(CONFIG_CACHE, CONFIG_CACHE_CACHE_INSERT_DATA, value); return Status::OK(); } @@ -1245,7 +1263,7 @@ Config::SetEngineConfigUseBlasThreshold(const std::string& value) { return s; } - SetConfigValueInMem(CONFIG_DB, CONFIG_ENGINE_USE_BLAS_THRESHOLD, value); + SetConfigValueInMem(CONFIG_ENGINE, CONFIG_ENGINE_USE_BLAS_THRESHOLD, value); return Status::OK(); } @@ -1256,7 +1274,7 @@ Config::SetEngineConfigOmpThreadNum(const std::string& value) { return s; } - SetConfigValueInMem(CONFIG_DB, CONFIG_ENGINE_OMP_THREAD_NUM, value); + SetConfigValueInMem(CONFIG_ENGINE, CONFIG_ENGINE_OMP_THREAD_NUM, value); return Status::OK(); } @@ -1267,7 +1285,7 @@ Config::SetEngineConfigGpuSearchThreshold(const std::string& value) { return s; } - SetConfigValueInMem(CONFIG_DB, CONFIG_ENGINE_GPU_SEARCH_THRESHOLD, value); + SetConfigValueInMem(CONFIG_ENGINE, CONFIG_ENGINE_GPU_SEARCH_THRESHOLD, value); return Status::OK(); } @@ -1279,7 +1297,21 @@ Config::SetResourceConfigMode(const std::string& value) { return s; } - SetConfigValueInMem(CONFIG_DB, CONFIG_RESOURCE_MODE, value); + SetConfigValueInMem(CONFIG_RESOURCE, CONFIG_RESOURCE_MODE, value); + return Status::OK(); +} + +Status +Config::SetResourceConfigSearchResources(const std::string& value) { + std::vector res_vec; + server::StringHelpFunctions::SplitStringByDelimeter(value, CONFIG_RESOURCE_SEARCH_RESOURCES_DELIMITER, res_vec); + + Status s = CheckResourceConfigSearchResources(res_vec); + if (!s.ok()) { + return s; + } + + SetConfigValueInMem(CONFIG_RESOURCE, CONFIG_RESOURCE_SEARCH_RESOURCES, value); return Status::OK(); } @@ -1290,7 +1322,7 @@ Config::SetResourceConfigIndexBuildDevice(const std::string& value) { return s; } - SetConfigValueInMem(CONFIG_DB, CONFIG_RESOURCE_INDEX_BUILD_DEVICE, value); + SetConfigValueInMem(CONFIG_RESOURCE, CONFIG_RESOURCE_INDEX_BUILD_DEVICE, value); return Status::OK(); } diff --git a/core/src/server/Config.h b/core/src/server/Config.h index 45591fb5ec..df43f04077 100644 --- a/core/src/server/Config.h +++ b/core/src/server/Config.h @@ -92,6 +92,8 @@ static const char* CONFIG_RESOURCE = "resource_config"; static const char* CONFIG_RESOURCE_MODE = "mode"; static const char* CONFIG_RESOURCE_MODE_DEFAULT = "simple"; static const char* CONFIG_RESOURCE_SEARCH_RESOURCES = "search_resources"; +static const char* CONFIG_RESOURCE_SEARCH_RESOURCES_DELIMITER = ","; +static const char* CONFIG_RESOURCE_SEARCH_RESOURCES_DEFAULT = "cpu,gpu0"; static const char* CONFIG_RESOURCE_INDEX_BUILD_DEVICE = "index_build_device"; static const char* CONFIG_RESOURCE_INDEX_BUILD_DEVICE_DEFAULT = "gpu0"; @@ -183,6 +185,8 @@ class Config { std::string GetConfigStr(const std::string& parent_key, const std::string& child_key, const std::string& default_value = ""); + std::string + GetConfigSequenceStr(const std::string& parent_key, const std::string& child_key, const std::string& delim); public: /* server config */ @@ -304,6 +308,8 @@ class Config { Status SetResourceConfigMode(const std::string& value); Status + SetResourceConfigSearchResources(const std::string& value); + Status SetResourceConfigIndexBuildDevice(const std::string& value); private: diff --git a/core/src/utils/StringHelpFunctions.cpp b/core/src/utils/StringHelpFunctions.cpp index 230cc1a0ff..2db37829d6 100644 --- a/core/src/utils/StringHelpFunctions.cpp +++ b/core/src/utils/StringHelpFunctions.cpp @@ -39,39 +39,53 @@ StringHelpFunctions::TrimStringQuote(std::string& string, const std::string& qou } } -Status +void StringHelpFunctions::SplitStringByDelimeter(const std::string& str, const std::string& delimeter, std::vector& result) { if (str.empty()) { - return Status::OK(); + return; } - size_t last = 0; - size_t index = str.find_first_of(delimeter, last); - while (index != std::string::npos) { - result.emplace_back(str.substr(last, index - last)); - last = index + 1; - index = str.find_first_of(delimeter, last); + size_t prev = 0, pos = 0; + while (true) { + pos = str.find_first_of(delimeter, prev); + if (pos == std::string::npos) { + result.emplace_back(str.substr(prev)); + break; + } else { + result.emplace_back(str.substr(prev, pos - prev)); + prev = pos + 1; + } } - if (index - last > 0) { - std::string temp = str.substr(last); - result.emplace_back(temp); +} + +void +StringHelpFunctions::MergeStringWithDelimeter(const std::vector& strs, const std::string& delimeter, + std::string& result) { + if (strs.empty()) { + result = ""; + return; } - return Status::OK(); + result = strs[0]; + for (size_t i = 1; i < strs.size(); i++) { + result = result + delimeter + strs[i]; + } } Status StringHelpFunctions::SplitStringByQuote(const std::string& str, const std::string& delimeter, const std::string& quote, std::vector& result) { if (quote.empty()) { - return SplitStringByDelimeter(str, delimeter, result); + SplitStringByDelimeter(str, delimeter, result); + return Status::OK(); } size_t last = 0; size_t index = str.find_first_of(quote, last); if (index == std::string::npos) { - return SplitStringByDelimeter(str, delimeter, result); + SplitStringByDelimeter(str, delimeter, result); + return Status::OK(); } std::string process_str = str; @@ -116,7 +130,7 @@ StringHelpFunctions::SplitStringByQuote(const std::string& str, const std::strin } if (!process_str.empty()) { - return SplitStringByDelimeter(process_str, delimeter, result); + SplitStringByDelimeter(process_str, delimeter, result); } return Status::OK(); diff --git a/core/src/utils/StringHelpFunctions.h b/core/src/utils/StringHelpFunctions.h index cb355332f1..0cb303bb4f 100644 --- a/core/src/utils/StringHelpFunctions.h +++ b/core/src/utils/StringHelpFunctions.h @@ -43,9 +43,12 @@ class StringHelpFunctions { // ,b, | b | // ,, | | // a a - static Status + static void SplitStringByDelimeter(const std::string& str, const std::string& delimeter, std::vector& result); + static void + MergeStringWithDelimeter(const std::vector& strs, const std::string& delimeter, std::string& result); + // assume the table has two columns, quote='\"', delimeter=',' // a,b a | b // "aa,gg,yy",b aa,gg,yy | b diff --git a/core/unittest/server/test_config.cpp b/core/unittest/server/test_config.cpp index 123ddf5265..8fd6de2a8d 100644 --- a/core/unittest/server/test_config.cpp +++ b/core/unittest/server/test_config.cpp @@ -19,10 +19,11 @@ #include #include "config/YamlConfigMgr.h" -#include "utils/CommonUtil.h" -#include "utils/ValidationUtil.h" #include "server/Config.h" #include "server/utils.h" +#include "utils/CommonUtil.h" +#include "utils/ValidationUtil.h" +#include "utils/StringHelpFunctions.h" namespace { @@ -98,19 +99,191 @@ TEST_F(ConfigTest, CONFIG_TEST) { ASSERT_TRUE(seqs.empty()); } -TEST_F(ConfigTest, SERVER_CONFIG_TEST) { +TEST_F(ConfigTest, SERVER_CONFIG_VALID_TEST) { std::string config_path(CONFIG_PATH); - milvus::server::Config& config = milvus::server::Config::GetInstance(); - milvus::Status s = config.LoadConfigFile(config_path + VALID_CONFIG_FILE); - ASSERT_TRUE(s.ok()); + milvus::server::Config &config = milvus::server::Config::GetInstance(); + milvus::Status s; + std::string str_val; + int32_t int32_val; + int64_t int64_val; + float float_val; + bool bool_val; - s = config.ValidateConfig(); + /* server config */ + std::string server_addr = "192.168.1.155"; + s = config.SetServerConfigAddress(server_addr); ASSERT_TRUE(s.ok()); - - config.PrintAll(); - - s = config.ResetDefaultConfig(); + s = config.GetServerConfigAddress(str_val); ASSERT_TRUE(s.ok()); + ASSERT_TRUE(str_val == server_addr); + + std::string server_port = "12345"; + s = config.SetServerConfigPort(server_port); + ASSERT_TRUE(s.ok()); + s = config.GetServerConfigPort(str_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(str_val == server_port); + + std::string server_mode = "cluster_readonly"; + s = config.SetServerConfigDeployMode(server_mode); + ASSERT_TRUE(s.ok()); + s = config.GetServerConfigDeployMode(str_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(str_val == server_mode); + + std::string server_time_zone = "UTC+6"; + s = config.SetServerConfigTimeZone(server_time_zone); + ASSERT_TRUE(s.ok()); + s = config.GetServerConfigTimeZone(str_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(str_val == server_time_zone); + + /* db config */ + std::string db_primary_path = "/home/zilliz"; + s = config.SetDBConfigPrimaryPath(db_primary_path); + ASSERT_TRUE(s.ok()); + s = config.GetDBConfigPrimaryPath(str_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(str_val == db_primary_path); + + std::string db_secondary_path = "/home/zilliz"; + s = config.SetDBConfigSecondaryPath(db_secondary_path); + ASSERT_TRUE(s.ok()); + s = config.GetDBConfigSecondaryPath(str_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(str_val == db_secondary_path); + + std::string db_backend_url = "mysql://root:123456@127.0.0.1:19530/milvus"; + s = config.SetDBConfigBackendUrl(db_backend_url); + ASSERT_TRUE(s.ok()); + s = config.GetDBConfigBackendUrl(str_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(str_val == db_backend_url); + + int32_t db_archive_disk_threshold = 100; + s = config.SetDBConfigArchiveDiskThreshold(std::to_string(db_archive_disk_threshold)); + ASSERT_TRUE(s.ok()); + s = config.GetDBConfigArchiveDiskThreshold(int32_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(int32_val == db_archive_disk_threshold); + + int32_t db_archive_days_threshold = 365; + s = config.SetDBConfigArchiveDaysThreshold(std::to_string(db_archive_days_threshold)); + ASSERT_TRUE(s.ok()); + s = config.GetDBConfigArchiveDaysThreshold(int32_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(int32_val == db_archive_days_threshold); + + int32_t db_insert_buffer_size = 2; + s = config.SetDBConfigInsertBufferSize(std::to_string(db_insert_buffer_size)); + ASSERT_TRUE(s.ok()); + s = config.GetDBConfigInsertBufferSize(int32_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(int32_val == db_insert_buffer_size); + + /* metric config */ + bool metric_enable_monitor = false; + s = config.SetMetricConfigEnableMonitor(std::to_string(metric_enable_monitor)); + ASSERT_TRUE(s.ok()); + s = config.GetMetricConfigEnableMonitor(bool_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(bool_val == metric_enable_monitor); + + std::string metric_collector = "prometheus"; + s = config.SetMetricConfigCollector(metric_collector); + ASSERT_TRUE(s.ok()); + s = config.GetMetricConfigCollector(str_val); + ASSERT_TRUE(str_val == metric_collector); + + std::string metric_prometheus_port = "2222"; + s = config.SetMetricConfigPrometheusPort(metric_prometheus_port); + ASSERT_TRUE(s.ok()); + s = config.GetMetricConfigPrometheusPort(str_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(str_val == metric_prometheus_port); + + /* cache config */ + int64_t cache_cpu_cache_capacity = 5; + s = config.SetCacheConfigCpuCacheCapacity(std::to_string(cache_cpu_cache_capacity)); + ASSERT_TRUE(s.ok()); + s = config.GetCacheConfigCpuCacheCapacity(int64_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(int64_val == cache_cpu_cache_capacity); + + float cache_cpu_cache_threshold = 0.1; + s = config.SetCacheConfigCpuCacheThreshold(std::to_string(cache_cpu_cache_threshold)); + ASSERT_TRUE(s.ok()); + s = config.GetCacheConfigCpuCacheThreshold(float_val); + ASSERT_TRUE(float_val == cache_cpu_cache_threshold); + + int64_t cache_gpu_cache_capacity = 1; + s = config.SetCacheConfigGpuCacheCapacity(std::to_string(cache_gpu_cache_capacity)); + ASSERT_TRUE(s.ok()); + s = config.GetCacheConfigGpuCacheCapacity(int64_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(int64_val == cache_gpu_cache_capacity); + + float cache_gpu_cache_threshold = 0.2; + s = config.SetCacheConfigGpuCacheThreshold(std::to_string(cache_gpu_cache_threshold)); + ASSERT_TRUE(s.ok()); + s = config.GetCacheConfigGpuCacheThreshold(float_val); + ASSERT_TRUE(float_val == cache_gpu_cache_threshold); + + bool cache_insert_data = true; + s = config.SetCacheConfigCacheInsertData(std::to_string(cache_insert_data)); + ASSERT_TRUE(s.ok()); + s = config.GetCacheConfigCacheInsertData(bool_val); + ASSERT_TRUE(bool_val == cache_insert_data); + + /* engine config */ + int32_t engine_use_blas_threshold = 50; + s = config.SetEngineConfigUseBlasThreshold(std::to_string(engine_use_blas_threshold)); + ASSERT_TRUE(s.ok()); + s = config.GetEngineConfigUseBlasThreshold(int32_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(int32_val == engine_use_blas_threshold); + + int32_t engine_omp_thread_num = 8; + s = config.SetEngineConfigOmpThreadNum(std::to_string(engine_omp_thread_num)); + ASSERT_TRUE(s.ok()); + s = config.GetEngineConfigOmpThreadNum(int32_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(int32_val == engine_omp_thread_num); + + int32_t engine_gpu_search_threshold = 800; + s = config.SetEngineConfigGpuSearchThreshold(std::to_string(engine_gpu_search_threshold)); + ASSERT_TRUE(s.ok()); + s = config.GetEngineConfigGpuSearchThreshold(int32_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(int32_val == engine_gpu_search_threshold); + + /* resource config */ + std::string resource_mode = "simple"; + s = config.SetResourceConfigMode(resource_mode); + ASSERT_TRUE(s.ok()); + s = config.GetResourceConfigMode(str_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(str_val == resource_mode); + + std::vector search_resources = {"cpu", "gpu0"}; + std::vector res_vec; + std::string res_str; + milvus::server::StringHelpFunctions::MergeStringWithDelimeter(search_resources, + milvus::server::CONFIG_RESOURCE_SEARCH_RESOURCES_DELIMITER, res_str); + s = config.SetResourceConfigSearchResources(res_str); + ASSERT_TRUE(s.ok()); + s = config.GetResourceConfigSearchResources(res_vec); + ASSERT_TRUE(s.ok()); + for (size_t i = 0; i < search_resources.size(); i++) { + ASSERT_TRUE(search_resources[i] == res_vec[i]); + } + + int32_t resource_index_build_device = 0; + s = config.SetResourceConfigIndexBuildDevice("gpu" + std::to_string(resource_index_build_device)); + ASSERT_TRUE(s.ok()); + s = config.GetResourceConfigIndexBuildDevice(int32_val); + ASSERT_TRUE(s.ok()); + ASSERT_TRUE(int32_val == resource_index_build_device); } TEST_F(ConfigTest, SERVER_CONFIG_INVALID_TEST) { @@ -226,9 +399,29 @@ TEST_F(ConfigTest, SERVER_CONFIG_INVALID_TEST) { s = config.SetResourceConfigMode("default"); ASSERT_FALSE(s.ok()); + s = config.SetResourceConfigSearchResources("gpu10"); + ASSERT_FALSE(s.ok()); + s = config.SetResourceConfigSearchResources("cpu"); + ASSERT_FALSE(s.ok()); + s = config.SetResourceConfigIndexBuildDevice("gup2"); ASSERT_FALSE(s.ok()); s = config.SetResourceConfigIndexBuildDevice("gpu16"); ASSERT_FALSE(s.ok()); } +TEST_F(ConfigTest, SERVER_CONFIG_TEST) { + std::string config_path(CONFIG_PATH); + milvus::server::Config& config = milvus::server::Config::GetInstance(); + milvus::Status s = config.LoadConfigFile(config_path + VALID_CONFIG_FILE); + ASSERT_TRUE(s.ok()); + + s = config.ValidateConfig(); + ASSERT_TRUE(s.ok()); + + config.PrintAll(); + + s = config.ResetDefaultConfig(); + ASSERT_TRUE(s.ok()); +} + diff --git a/core/unittest/server/test_util.cpp b/core/unittest/server/test_util.cpp index 24482740bc..6f180aab72 100644 --- a/core/unittest/server/test_util.cpp +++ b/core/unittest/server/test_util.cpp @@ -117,12 +117,11 @@ TEST(UtilTest, STRINGFUNCTIONS_TEST) { str = "a,b,c"; std::vector result; - auto status = milvus::server::StringHelpFunctions::SplitStringByDelimeter(str, ",", result); - ASSERT_TRUE(status.ok()); + milvus::server::StringHelpFunctions::SplitStringByDelimeter(str, ",", result); ASSERT_EQ(result.size(), 3UL); result.clear(); - status = milvus::server::StringHelpFunctions::SplitStringByQuote(str, ",", "\"", result); + auto status = milvus::server::StringHelpFunctions::SplitStringByQuote(str, ",", "\"", result); ASSERT_TRUE(status.ok()); ASSERT_EQ(result.size(), 3UL); From 5b73f1c2d9b3060827b440e1d343164f767ef6c7 Mon Sep 17 00:00:00 2001 From: "yudong.cai" Date: Wed, 6 Nov 2019 12:04:28 +0800 Subject: [PATCH 005/103] #207 update changelog --- CHANGELOG.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ffe0e3fc09..88b2a69e64 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,8 +1,18 @@ # Changelog Please mark all change in change log and use the ticket from JIRA. +# Milvus 0.5.3 (TODO) -# Milvus 0.5.2 (TODO) +## Bug + +## Feature + +## Improvement +- \#207 - Add more unittest for config set/get + +## Task + +# Milvus 0.5.2 (2019-11-05) ## Bug - \#194 - Search faild: message="Table file doesn't exist" @@ -10,7 +20,7 @@ Please mark all change in change log and use the ticket from JIRA. ## Feature ## Improvement -- \#190 - Update default config:use_blas_threshold to 1100 and server version printout to 0.52 +- \#190 - Update default config:use_blas_threshold to 1100 and server version printout to 0.5.2 ## Task From 5a9f7997abe11fe8ea2d2282b9ff6e415565568d Mon Sep 17 00:00:00 2001 From: "yudong.cai" Date: Wed, 6 Nov 2019 13:12:04 +0800 Subject: [PATCH 006/103] #207 fix unittest build error --- core/unittest/CMakeLists.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/core/unittest/CMakeLists.txt b/core/unittest/CMakeLists.txt index 62b5bdf256..361735ad2c 100644 --- a/core/unittest/CMakeLists.txt +++ b/core/unittest/CMakeLists.txt @@ -80,6 +80,7 @@ set(helper_files ${MILVUS_ENGINE_SRC}/utils/CommonUtil.cpp ${MILVUS_ENGINE_SRC}/utils/TimeRecorder.cpp ${MILVUS_ENGINE_SRC}/utils/Status.cpp + ${MILVUS_ENGINE_SRC}/utils/StringHelpFunctions.cpp ${MILVUS_ENGINE_SRC}/utils/ValidationUtil.cpp ${MILVUS_ENGINE_SRC}/external/easyloggingpp/easylogging++.cc ) From 2deba2dcb99c36e7ae00711a84760cd9a841f1b2 Mon Sep 17 00:00:00 2001 From: "yudong.cai" Date: Wed, 6 Nov 2019 16:22:38 +0800 Subject: [PATCH 007/103] #208 optimize unittest to support run single test more easily --- CHANGELOG.md | 1 + core/src/server/Config.cpp | 14 ++++++++++---- core/src/server/Config.h | 3 ++- core/unittest/db/CMakeLists.txt | 12 ++++++++++-- core/unittest/db/utils.cpp | 4 ++-- core/unittest/metrics/CMakeLists.txt | 6 +++++- core/unittest/scheduler/CMakeLists.txt | 12 +++++++++++- .../scheduler/{task_test.cpp => test_task.cpp} | 0 core/unittest/server/CMakeLists.txt | 7 ++++++- core/unittest/wrapper/CMakeLists.txt | 6 +++++- core/unittest/wrapper/utils.cpp | 4 ++-- 11 files changed, 54 insertions(+), 15 deletions(-) rename core/unittest/scheduler/{task_test.cpp => test_task.cpp} (100%) diff --git a/CHANGELOG.md b/CHANGELOG.md index 88b2a69e64..d49922b259 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ Please mark all change in change log and use the ticket from JIRA. ## Improvement - \#207 - Add more unittest for config set/get +- \#208 - optimize unittest to support run single test more easily ## Task diff --git a/core/src/server/Config.cpp b/core/src/server/Config.cpp index 20bcb937e7..ccc518cac3 100644 --- a/core/src/server/Config.cpp +++ b/core/src/server/Config.cpp @@ -795,11 +795,16 @@ Config::GetConfigStr(const std::string& parent_key, const std::string& child_key } std::string -Config::GetConfigSequenceStr(const std::string& parent_key, const std::string& child_key, const std::string& delim) { +Config::GetConfigSequenceStr(const std::string& parent_key, const std::string& child_key, const std::string& delim, + const std::string& default_value) { std::string value; if (!GetConfigValueInMem(parent_key, child_key, value).ok()) { std::vector sequence = GetConfigNode(parent_key).GetSequence(child_key); - server::StringHelpFunctions::MergeStringWithDelimeter(sequence, delim, value); + if (sequence.empty()) { + value = default_value; + } else { + server::StringHelpFunctions::MergeStringWithDelimeter(sequence, delim, value); + } SetConfigValueInMem(parent_key, child_key, value); } return value; @@ -1028,8 +1033,9 @@ Config::GetResourceConfigMode(std::string& value) { Status Config::GetResourceConfigSearchResources(std::vector& value) { - std::string str = GetConfigSequenceStr(CONFIG_RESOURCE, CONFIG_RESOURCE_SEARCH_RESOURCES, - CONFIG_RESOURCE_SEARCH_RESOURCES_DELIMITER); + std::string str = + GetConfigSequenceStr(CONFIG_RESOURCE, CONFIG_RESOURCE_SEARCH_RESOURCES, + CONFIG_RESOURCE_SEARCH_RESOURCES_DELIMITER, CONFIG_RESOURCE_SEARCH_RESOURCES_DEFAULT); server::StringHelpFunctions::SplitStringByDelimeter(str, CONFIG_RESOURCE_SEARCH_RESOURCES_DELIMITER, value); return CheckResourceConfigSearchResources(value); } diff --git a/core/src/server/Config.h b/core/src/server/Config.h index df43f04077..e466b70a3b 100644 --- a/core/src/server/Config.h +++ b/core/src/server/Config.h @@ -186,7 +186,8 @@ class Config { std::string GetConfigStr(const std::string& parent_key, const std::string& child_key, const std::string& default_value = ""); std::string - GetConfigSequenceStr(const std::string& parent_key, const std::string& child_key, const std::string& delim); + GetConfigSequenceStr(const std::string& parent_key, const std::string& child_key, const std::string& delim = ",", + const std::string& default_value = ""); public: /* server config */ diff --git a/core/unittest/db/CMakeLists.txt b/core/unittest/db/CMakeLists.txt index 3954dd8656..92ddc910af 100644 --- a/core/unittest/db/CMakeLists.txt +++ b/core/unittest/db/CMakeLists.txt @@ -17,8 +17,16 @@ # under the License. #------------------------------------------------------------------------------- - -aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR} test_files) +set(test_files + ${CMAKE_CURRENT_SOURCE_DIR}/test_db.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_db_mysql.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_engine.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_mem.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_meta.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_meta_mysql.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_misc.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_search.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/utils.cpp) cuda_add_executable(test_db ${common_files} diff --git a/core/unittest/db/utils.cpp b/core/unittest/db/utils.cpp index 16e195079c..9dce779737 100644 --- a/core/unittest/db/utils.cpp +++ b/core/unittest/db/utils.cpp @@ -60,8 +60,8 @@ static const char " port: 8080 # port prometheus used to fetch metrics\n" "\n" "cache_config:\n" - " cpu_mem_capacity: 16 # GB, CPU memory used for cache\n" - " cpu_mem_threshold: 0.85 # percentage of data kept when cache cleanup triggered\n" + " cpu_cache_capacity: 16 # GB, CPU memory used for cache\n" + " cpu_cache_threshold: 0.85 # percentage of data kept when cache cleanup triggered\n" " cache_insert_data: false # whether load inserted data into cache\n" "\n" "engine_config:\n" diff --git a/core/unittest/metrics/CMakeLists.txt b/core/unittest/metrics/CMakeLists.txt index eba8146baa..ad9d4e3943 100644 --- a/core/unittest/metrics/CMakeLists.txt +++ b/core/unittest/metrics/CMakeLists.txt @@ -17,7 +17,11 @@ # under the License. #------------------------------------------------------------------------------- -aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR} test_files) +set(test_files + ${CMAKE_CURRENT_SOURCE_DIR}/test_metricbase.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_metrics.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_prometheus.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/utils.cpp) add_executable(test_metrics ${common_files} diff --git a/core/unittest/scheduler/CMakeLists.txt b/core/unittest/scheduler/CMakeLists.txt index 087f93f017..6a48d40191 100644 --- a/core/unittest/scheduler/CMakeLists.txt +++ b/core/unittest/scheduler/CMakeLists.txt @@ -17,7 +17,17 @@ # under the License. #------------------------------------------------------------------------------- -aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR} test_files) +set(test_files + ${CMAKE_CURRENT_SOURCE_DIR}/test_algorithm.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_event.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_node.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_normal.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_resource.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_resource_factory.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_resource_mgr.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_scheduler.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_task.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_tasktable.cpp) cuda_add_executable(test_scheduler ${common_files} diff --git a/core/unittest/scheduler/task_test.cpp b/core/unittest/scheduler/test_task.cpp similarity index 100% rename from core/unittest/scheduler/task_test.cpp rename to core/unittest/scheduler/test_task.cpp diff --git a/core/unittest/server/CMakeLists.txt b/core/unittest/server/CMakeLists.txt index 1f89de8d3f..742b1fad5a 100644 --- a/core/unittest/server/CMakeLists.txt +++ b/core/unittest/server/CMakeLists.txt @@ -17,7 +17,12 @@ # under the License. #------------------------------------------------------------------------------- -aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR} test_files) +set(test_files + ${CMAKE_CURRENT_SOURCE_DIR}/test_cache.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_config.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_rpc.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_util.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/utils.cpp) include_directories("${CUDA_TOOLKIT_ROOT_DIR}/include") link_directories("${CUDA_TOOLKIT_ROOT_DIR}/lib64") diff --git a/core/unittest/wrapper/CMakeLists.txt b/core/unittest/wrapper/CMakeLists.txt index ef145a9f50..3f2be064c2 100644 --- a/core/unittest/wrapper/CMakeLists.txt +++ b/core/unittest/wrapper/CMakeLists.txt @@ -17,7 +17,11 @@ # under the License. #------------------------------------------------------------------------------- -aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR} test_files) +set(test_files + ${CMAKE_CURRENT_SOURCE_DIR}/test_hybrid_index.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_knowhere.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/test_wrapper.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/utils.cpp) set(wrapper_files ${MILVUS_ENGINE_SRC}/wrapper/DataTransfer.cpp diff --git a/core/unittest/wrapper/utils.cpp b/core/unittest/wrapper/utils.cpp index b397a35d7c..e17077aece 100644 --- a/core/unittest/wrapper/utils.cpp +++ b/core/unittest/wrapper/utils.cpp @@ -50,8 +50,8 @@ static const char " port: 8080 # port prometheus used to fetch metrics\n" "\n" "cache_config:\n" - " cpu_mem_capacity: 16 # GB, CPU memory used for cache\n" - " cpu_mem_threshold: 0.85 # percentage of data kept when cache cleanup triggered\n" + " cpu_cache_capacity: 16 # GB, CPU memory used for cache\n" + " cpu_cache_threshold: 0.85 # percentage of data kept when cache cleanup triggered\n" " cache_insert_data: false # whether load inserted data into cache\n" "\n" "engine_config:\n" From bf87a834dd13ef570df5fec6f38245a36ba9100d Mon Sep 17 00:00:00 2001 From: quicksilver Date: Wed, 6 Nov 2019 17:14:31 +0800 Subject: [PATCH 008/103] add travis CI --- .travis.yaml | 11 +++ ci/scripts/build.sh | 123 ++++++++++++++++++++++++++++++++ ci/travis/install_dependency.sh | 35 +++++++++ ci/travis/travis_build.sh | 24 +++++++ ci/travis/travis_env_common.sh | 10 +++ 5 files changed, 203 insertions(+) create mode 100644 .travis.yaml create mode 100755 ci/scripts/build.sh create mode 100755 ci/travis/install_dependency.sh create mode 100644 ci/travis/travis_build.sh create mode 100644 ci/travis/travis_env_common.sh diff --git a/.travis.yaml b/.travis.yaml new file mode 100644 index 0000000000..7b008a38e4 --- /dev/null +++ b/.travis.yaml @@ -0,0 +1,11 @@ +sudo: required +dist: bionic +addons: + apt: + update: true + packages: + - cmake-3.14 +install: + - source $TRAVIS_BUILD_DIR/ci/travis/install_dependency.sh +script: + - $TRAVIS_BUILD_DIR/ci/travis/travis_build.sh diff --git a/ci/scripts/build.sh b/ci/scripts/build.sh new file mode 100755 index 0000000000..fb6f94f670 --- /dev/null +++ b/ci/scripts/build.sh @@ -0,0 +1,123 @@ +#!/bin/bash + +set -ex + +SOURCE="${BASH_SOURCE[0]}" +while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink + DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" + SOURCE="$(readlink "$SOURCE")" + [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located +done +SCRIPTS_DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" + +MILVUS_CORE_DIR="${SCRIPTS_DIR}/../../core" +CORE_BUILD_DIR="${MILVUS_CORE_DIR}/cmake_build" +BUILD_TYPE="Debug" +BUILD_UNITTEST="OFF" +INSTALL_PREFIX="/opt/milvus" +BUILD_COVERAGE="OFF" +USE_JFROG_CACHE="OFF" +RUN_CPPLINT="OFF" +CUDA_COMPILER=/usr/local/cuda/bin/nvcc + +while getopts "o:t:b:ulcjh" arg +do + case $arg in + o) + INSTALL_PREFIX=$OPTARG + ;; + t) + BUILD_TYPE=$OPTARG # BUILD_TYPE + ;; + b) + CORE_BUILD_DIR=$OPTARG # CORE_BUILD_DIR + ;; + u) + echo "Build and run unittest cases" ; + BUILD_UNITTEST="ON"; + ;; + l) + RUN_CPPLINT="ON" + ;; + c) + BUILD_COVERAGE="ON" + ;; + j) + USE_JFROG_CACHE="ON" + ;; + h) # help + echo " + +parameter: +-o: install prefix(default: /opt/milvus) +-t: build type(default: Debug) +-b: core code build directory +-u: building unit test options(default: OFF) +-l: run cpplint, clang-format and clang-tidy(default: OFF) +-c: code coverage(default: OFF) +-j: use jfrog cache build directory(default: OFF) +-h: help + +usage: +./build.sh -o \${INSTALL_PREFIX} -t \${BUILD_TYPE} -b \${CORE_BUILD_DIR} [-u] [-l] [-c] [-j] [-h] + " + exit 0 + ;; + ?) + echo "ERROR! unknown argument" + exit 1 + ;; + esac +done + +if [[ ! -d ${CORE_BUILD_DIR} ]]; then + mkdir ${CORE_BUILD_DIR} +fi + +pushd ${CORE_BUILD_DIR} + +# remove make cache since build.sh -l use default variables +# force update the variables each time +make rebuild_cache + +CMAKE_CMD="cmake \ +-DCMAKE_INSTALL_PREFIX=${INSTALL_PREFIX} +-DCMAKE_BUILD_TYPE=${BUILD_TYPE} \ +-DCMAKE_CUDA_COMPILER=${CUDA_COMPILER} \ +-DBUILD_UNIT_TEST=${BUILD_UNITTEST} \ +-DBUILD_COVERAGE=${BUILD_COVERAGE} \ +-DUSE_JFROG_CACHE=${USE_JFROG_CACHE} \ +${MILVUS_CORE_DIR}" +echo ${CMAKE_CMD} +${CMAKE_CMD} + +if [[ ${RUN_CPPLINT} == "ON" ]]; then + # cpplint check + make lint + if [ $? -ne 0 ]; then + echo "ERROR! cpplint check failed" + exit 1 + fi + echo "cpplint check passed!" + + # clang-format check + make check-clang-format + if [ $? -ne 0 ]; then + echo "ERROR! clang-format check failed" + exit 1 + fi + echo "clang-format check passed!" + +# # clang-tidy check +# make check-clang-tidy +# if [ $? -ne 0 ]; then +# echo "ERROR! clang-tidy check failed" +# rm -f CMakeCache.txt +# exit 1 +# fi +# echo "clang-tidy check passed!" +else + # compile and build + make -j8 || exit 1 + make install || exit 1 +fi diff --git a/ci/travis/install_dependency.sh b/ci/travis/install_dependency.sh new file mode 100755 index 0000000000..bc4a972b8f --- /dev/null +++ b/ci/travis/install_dependency.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash + +set -ex + +wget -P /tmp https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB + +sudo apt-key add /tmp/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB + +echo "deb https://apt.repos.intel.com/mkl all main" | \ + sudo tee /etc/apt/sources.list.d/intel-mkl.list + +sudo apt-get update -qq + +sudo apt-get install -y -q --no-install-recommends \ + flex \ + bison \ + gfortran \ + lsb-core \ + libtool \ + automake \ + pkg-config \ + libboost-filesystem-dev \ + libboost-system-dev \ + libboost-regex-dev \ + intel-mkl-gnu-2019.4-243 \ + intel-mkl-core-2019.4-243 \ + libmysqlclient-dev \ + clang-format-6.0 \ + clang-tidy-6.0 \ + lcov + +sudo ln -s /usr/lib/x86_64-linux-gnu/libmysqlclient.so \ + /usr/lib/x86_64-linux-gnu/libmysqlclient_r.so + +export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/opt/intel/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64 \ No newline at end of file diff --git a/ci/travis/travis_build.sh b/ci/travis/travis_build.sh new file mode 100644 index 0000000000..3cde1d5a4d --- /dev/null +++ b/ci/travis/travis_build.sh @@ -0,0 +1,24 @@ +#!/usr/bin/env bash + +set -ex + +source $TRAVIS_BUILD_DIR/ci/travis/travis_env_common.sh + +only_library_mode=no + +while true; do + case "$1" in + --only-library) + only_library_mode=yes + shift ;; + *) break ;; + esac +done + +BUILD_COMMON_FLAGS="-t ${MILVUS_BUILD_TYPE} -o ${MILVUS_INSTALL_PREFIX} -b ${MILVUS_BUILD_DIR}" + +if [ $only_library_mode == "yes" ]; then + ${TRAVIS_BUILD_DIR}/ci/scripts/build.sh ${BUILD_COMMON_FLAGS} +else + ${TRAVIS_BUILD_DIR}/ci/scripts/build.sh ${BUILD_COMMON_FLAGS} -u -c +fi \ No newline at end of file diff --git a/ci/travis/travis_env_common.sh b/ci/travis/travis_env_common.sh new file mode 100644 index 0000000000..ac63d2950b --- /dev/null +++ b/ci/travis/travis_env_common.sh @@ -0,0 +1,10 @@ +export MILVUS_CORE_DIR=${TRAVIS_BUILD_DIR}/core +export MILVUS_BUILD_DIR=${TRAVIS_BUILD_DIR}/core/cmake_build +export MILVUS_INSTALL_PREFIX=/opt/milvus +export MILVUS_TRAVIS_COVERAGE=${MILVUS_TRAVIS_COVERAGE:=0} + +if ["$MILVUS_TRAVIS_COVERAGE" == "1"]; then + export MILVUS_CPP_COVERAGE_FILE=${TRAVIS_BUILD_DIR}/output_new.info +fi + +export MILVUS_BUILD_TYPE=${MILVUS_BUILD_TYPE:=Release} From 191d63fc4b7f4e091c65a5b4b7ddc02a54a6d193 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Wed, 6 Nov 2019 17:23:49 +0800 Subject: [PATCH 009/103] rename travis CI file --- .travis.yaml => .travis.yml | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename .travis.yaml => .travis.yml (100%) diff --git a/.travis.yaml b/.travis.yml similarity index 100% rename from .travis.yaml rename to .travis.yml From 2f571df6deac0513ea61c7c7c9a810ee94a199ab Mon Sep 17 00:00:00 2001 From: quicksilver Date: Wed, 6 Nov 2019 17:27:10 +0800 Subject: [PATCH 010/103] disable install cmake-3.14 in travis.yml --- .travis.yml | 2 -- 1 file changed, 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 7b008a38e4..9abb055c52 100644 --- a/.travis.yml +++ b/.travis.yml @@ -3,8 +3,6 @@ dist: bionic addons: apt: update: true - packages: - - cmake-3.14 install: - source $TRAVIS_BUILD_DIR/ci/travis/install_dependency.sh script: From dc94aa22cbb69d56d3717fb65949a1372331462f Mon Sep 17 00:00:00 2001 From: quicksilver Date: Wed, 6 Nov 2019 17:44:56 +0800 Subject: [PATCH 011/103] setting language in travis.yml --- .travis.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.travis.yml b/.travis.yml index 9abb055c52..ec3b1c41fa 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,3 +1,4 @@ +language: cpp sudo: required dist: bionic addons: From 1551adf908eba47c8a05f6ac31695a4e82e225d7 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Wed, 6 Nov 2019 17:47:01 +0800 Subject: [PATCH 012/103] update ci/travis/travis_build.sh --- ci/travis/travis_build.sh | 0 1 file changed, 0 insertions(+), 0 deletions(-) mode change 100644 => 100755 ci/travis/travis_build.sh diff --git a/ci/travis/travis_build.sh b/ci/travis/travis_build.sh old mode 100644 new mode 100755 From be384f22f9e6d61fc113d1fb925705ce84d2ebcb Mon Sep 17 00:00:00 2001 From: quicksilver Date: Wed, 6 Nov 2019 17:55:30 +0800 Subject: [PATCH 013/103] update ci/scripts/build.sh --- ci/scripts/build.sh | 39 ++++----------------------------- ci/travis/install_dependency.sh | 4 ++-- 2 files changed, 6 insertions(+), 37 deletions(-) diff --git a/ci/scripts/build.sh b/ci/scripts/build.sh index fb6f94f670..6276359fae 100755 --- a/ci/scripts/build.sh +++ b/ci/scripts/build.sh @@ -1,6 +1,6 @@ #!/bin/bash -set -ex +set -e SOURCE="${BASH_SOURCE[0]}" while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink @@ -76,10 +76,6 @@ fi pushd ${CORE_BUILD_DIR} -# remove make cache since build.sh -l use default variables -# force update the variables each time -make rebuild_cache - CMAKE_CMD="cmake \ -DCMAKE_INSTALL_PREFIX=${INSTALL_PREFIX} -DCMAKE_BUILD_TYPE=${BUILD_TYPE} \ @@ -91,33 +87,6 @@ ${MILVUS_CORE_DIR}" echo ${CMAKE_CMD} ${CMAKE_CMD} -if [[ ${RUN_CPPLINT} == "ON" ]]; then - # cpplint check - make lint - if [ $? -ne 0 ]; then - echo "ERROR! cpplint check failed" - exit 1 - fi - echo "cpplint check passed!" - - # clang-format check - make check-clang-format - if [ $? -ne 0 ]; then - echo "ERROR! clang-format check failed" - exit 1 - fi - echo "clang-format check passed!" - -# # clang-tidy check -# make check-clang-tidy -# if [ $? -ne 0 ]; then -# echo "ERROR! clang-tidy check failed" -# rm -f CMakeCache.txt -# exit 1 -# fi -# echo "clang-tidy check passed!" -else - # compile and build - make -j8 || exit 1 - make install || exit 1 -fi +# compile and build +make -j8 || exit 1 +make install || exit 1 diff --git a/ci/travis/install_dependency.sh b/ci/travis/install_dependency.sh index bc4a972b8f..cec4e9c30f 100755 --- a/ci/travis/install_dependency.sh +++ b/ci/travis/install_dependency.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash -set -ex +set -e wget -P /tmp https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB @@ -32,4 +32,4 @@ sudo apt-get install -y -q --no-install-recommends \ sudo ln -s /usr/lib/x86_64-linux-gnu/libmysqlclient.so \ /usr/lib/x86_64-linux-gnu/libmysqlclient_r.so -export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/opt/intel/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64 \ No newline at end of file +export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/opt/intel/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64 From 6809839ffc4d8d0dc59f78f208296bf4d72522a2 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Wed, 6 Nov 2019 18:02:33 +0800 Subject: [PATCH 014/103] Install cmake-3.14 on Travis CI --- ci/travis/install_dependency.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/ci/travis/install_dependency.sh b/ci/travis/install_dependency.sh index cec4e9c30f..df4752b2ef 100755 --- a/ci/travis/install_dependency.sh +++ b/ci/travis/install_dependency.sh @@ -2,6 +2,9 @@ set -e +wget -qO- "https://cmake.org/files/v3.14/cmake-3.14.3-Linux-x86_64.tar.gz" | + sudo tar --strip-components=1 -xz -C /usr/local + wget -P /tmp https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB sudo apt-key add /tmp/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB From 84c47daf866a4e64e68d599e33a32d84c8403fed Mon Sep 17 00:00:00 2001 From: quicksilver Date: Wed, 6 Nov 2019 18:13:53 +0800 Subject: [PATCH 015/103] add before_install stage in .travis.yml --- .travis.yml | 2 ++ ci/travis/install_dependency.sh | 3 --- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index ec3b1c41fa..2f9462e261 100644 --- a/.travis.yml +++ b/.travis.yml @@ -4,6 +4,8 @@ dist: bionic addons: apt: update: true +before_install: + - wget -qO- "https://cmake.org/files/v3.14/cmake-3.14.3-Linux-x86_64.tar.gz" | sudo tar --strip-components=1 -xz -C /usr/local install: - source $TRAVIS_BUILD_DIR/ci/travis/install_dependency.sh script: diff --git a/ci/travis/install_dependency.sh b/ci/travis/install_dependency.sh index df4752b2ef..cec4e9c30f 100755 --- a/ci/travis/install_dependency.sh +++ b/ci/travis/install_dependency.sh @@ -2,9 +2,6 @@ set -e -wget -qO- "https://cmake.org/files/v3.14/cmake-3.14.3-Linux-x86_64.tar.gz" | - sudo tar --strip-components=1 -xz -C /usr/local - wget -P /tmp https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB sudo apt-key add /tmp/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB From 81dc258a07ba9e3db341fd8f9b2029d7f1216c45 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Thu, 7 Nov 2019 10:03:55 +0800 Subject: [PATCH 016/103] add before_install stage in .travis.yml --- .travis.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 2f9462e261..760d2d8ace 100644 --- a/.travis.yml +++ b/.travis.yml @@ -5,7 +5,8 @@ addons: apt: update: true before_install: - - wget -qO- "https://cmake.org/files/v3.14/cmake-3.14.3-Linux-x86_64.tar.gz" | sudo tar --strip-components=1 -xz -C /usr/local + - wget -O cmake.sh https://cmake.org/files/v3.14/cmake-3.14.0-Linux-x86_64.sh + - sudo sh cmake.sh --skip-license --exclude-subdir --prefix=/usr/local install: - source $TRAVIS_BUILD_DIR/ci/travis/install_dependency.sh script: From e7994a7c00629181ca133385ff1f10fe01dcce0a Mon Sep 17 00:00:00 2001 From: quicksilver Date: Thu, 7 Nov 2019 10:16:15 +0800 Subject: [PATCH 017/103] update cmake version --- .travis.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.travis.yml b/.travis.yml index 760d2d8ace..1f98f4ab83 100644 --- a/.travis.yml +++ b/.travis.yml @@ -7,6 +7,7 @@ addons: before_install: - wget -O cmake.sh https://cmake.org/files/v3.14/cmake-3.14.0-Linux-x86_64.sh - sudo sh cmake.sh --skip-license --exclude-subdir --prefix=/usr/local + - export PATH="/usr/local:$PATH" install: - source $TRAVIS_BUILD_DIR/ci/travis/install_dependency.sh script: From 35264a1f85da02f44ba22dd01362f2d7db54869e Mon Sep 17 00:00:00 2001 From: quicksilver Date: Thu, 7 Nov 2019 10:22:25 +0800 Subject: [PATCH 018/103] update cmake version --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 1f98f4ab83..df573e74f7 100644 --- a/.travis.yml +++ b/.travis.yml @@ -7,7 +7,7 @@ addons: before_install: - wget -O cmake.sh https://cmake.org/files/v3.14/cmake-3.14.0-Linux-x86_64.sh - sudo sh cmake.sh --skip-license --exclude-subdir --prefix=/usr/local - - export PATH="/usr/local:$PATH" + - export PATH="$PATH:/usr/local" install: - source $TRAVIS_BUILD_DIR/ci/travis/install_dependency.sh script: From 32b29fe7b893b397c978ccc649348115087a8eb7 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Thu, 7 Nov 2019 10:29:00 +0800 Subject: [PATCH 019/103] update cmake version --- .travis.yml | 4 ---- core/CMakeLists.txt | 4 ++-- core/src/index/CMakeLists.txt | 2 +- 3 files changed, 3 insertions(+), 7 deletions(-) diff --git a/.travis.yml b/.travis.yml index df573e74f7..ec3b1c41fa 100644 --- a/.travis.yml +++ b/.travis.yml @@ -4,10 +4,6 @@ dist: bionic addons: apt: update: true -before_install: - - wget -O cmake.sh https://cmake.org/files/v3.14/cmake-3.14.0-Linux-x86_64.sh - - sudo sh cmake.sh --skip-license --exclude-subdir --prefix=/usr/local - - export PATH="$PATH:/usr/local" install: - source $TRAVIS_BUILD_DIR/ci/travis/install_dependency.sh script: diff --git a/core/CMakeLists.txt b/core/CMakeLists.txt index a59e80a6e8..800993e1ce 100644 --- a/core/CMakeLists.txt +++ b/core/CMakeLists.txt @@ -18,7 +18,7 @@ #------------------------------------------------------------------------------- -cmake_minimum_required(VERSION 3.14) +cmake_minimum_required(VERSION 3.12) message(STATUS "Building using CMake version: ${CMAKE_VERSION}") set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_CURRENT_SOURCE_DIR}/cmake") @@ -273,4 +273,4 @@ if (${CLANG_TIDY_FOUND}) --source_dir ${CMAKE_CURRENT_SOURCE_DIR}/src ${MILVUS_LINT_QUIET}) -endif () \ No newline at end of file +endif () diff --git a/core/src/index/CMakeLists.txt b/core/src/index/CMakeLists.txt index fcb291eda2..560a73ea16 100644 --- a/core/src/index/CMakeLists.txt +++ b/core/src/index/CMakeLists.txt @@ -18,7 +18,7 @@ #------------------------------------------------------------------------------- -cmake_minimum_required(VERSION 3.14) +cmake_minimum_required(VERSION 3.12) message(STATUS "------------------------------KNOWHERE-----------------------------------") message(STATUS "Building using CMake version: ${CMAKE_VERSION}") From 6e8f397a69c07cc62932d63664e1edd9cee7d9b5 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Thu, 7 Nov 2019 10:58:00 +0800 Subject: [PATCH 020/103] cpu version for default setting --- ci/scripts/build.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/ci/scripts/build.sh b/ci/scripts/build.sh index 6276359fae..253ee5893d 100755 --- a/ci/scripts/build.sh +++ b/ci/scripts/build.sh @@ -18,9 +18,10 @@ INSTALL_PREFIX="/opt/milvus" BUILD_COVERAGE="OFF" USE_JFROG_CACHE="OFF" RUN_CPPLINT="OFF" +CPU_VERSION="ON" CUDA_COMPILER=/usr/local/cuda/bin/nvcc -while getopts "o:t:b:ulcjh" arg +while getopts "o:t:b:gulcjh" arg do case $arg in o) @@ -32,6 +33,9 @@ do b) CORE_BUILD_DIR=$OPTARG # CORE_BUILD_DIR ;; + g) + CPU_VERSION="OFF"; + ;; u) echo "Build and run unittest cases" ; BUILD_UNITTEST="ON"; @@ -52,6 +56,7 @@ parameter: -o: install prefix(default: /opt/milvus) -t: build type(default: Debug) -b: core code build directory +-g: gpu version -u: building unit test options(default: OFF) -l: run cpplint, clang-format and clang-tidy(default: OFF) -c: code coverage(default: OFF) @@ -80,6 +85,7 @@ CMAKE_CMD="cmake \ -DCMAKE_INSTALL_PREFIX=${INSTALL_PREFIX} -DCMAKE_BUILD_TYPE=${BUILD_TYPE} \ -DCMAKE_CUDA_COMPILER=${CUDA_COMPILER} \ +-DMILVUS_CPU_VERSION=${CPU_VERSION} \ -DBUILD_UNIT_TEST=${BUILD_UNITTEST} \ -DBUILD_COVERAGE=${BUILD_COVERAGE} \ -DUSE_JFROG_CACHE=${USE_JFROG_CACHE} \ From 2d86057603078d405e4d3daeb0736742c5b92f25 Mon Sep 17 00:00:00 2001 From: zhenwu Date: Thu, 7 Nov 2019 13:55:10 +0800 Subject: [PATCH 021/103] Update test case: version to 0.5.3 --- tests/milvus-java-test/pom.xml | 4 ++-- tests/milvus_python_test/test_mix.py | 2 +- tests/milvus_python_test/test_ping.py | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/milvus-java-test/pom.xml b/tests/milvus-java-test/pom.xml index 4da715e292..7006bf159f 100644 --- a/tests/milvus-java-test/pom.xml +++ b/tests/milvus-java-test/pom.xml @@ -99,7 +99,7 @@ io.milvus milvus-sdk-java - 0.2.0-SNAPSHOT + 0.2.2 @@ -134,4 +134,4 @@ - \ No newline at end of file + diff --git a/tests/milvus_python_test/test_mix.py b/tests/milvus_python_test/test_mix.py index f099db5c31..fa6d57e408 100644 --- a/tests/milvus_python_test/test_mix.py +++ b/tests/milvus_python_test/test_mix.py @@ -180,4 +180,4 @@ def check_result(result, id): if len(result) >= 5: return id in [result[0].id, result[1].id, result[2].id, result[3].id, result[4].id] else: - return id in (i.id for i in result) \ No newline at end of file + return id in (i.id for i in result) diff --git a/tests/milvus_python_test/test_ping.py b/tests/milvus_python_test/test_ping.py index d63ab93f11..3f79f45298 100644 --- a/tests/milvus_python_test/test_ping.py +++ b/tests/milvus_python_test/test_ping.py @@ -1,7 +1,7 @@ import logging import pytest -__version__ = '0.5.1' +__version__ = '0.5.3' class TestPing: From 5d9826e3de33421d0c8e49c7cd605fdb56b53572 Mon Sep 17 00:00:00 2001 From: zhenwu Date: Thu, 7 Nov 2019 14:00:49 +0800 Subject: [PATCH 022/103] Update mix case level --- tests/milvus_python_test/test_mix.py | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/milvus_python_test/test_mix.py b/tests/milvus_python_test/test_mix.py index fa6d57e408..84afb52975 100644 --- a/tests/milvus_python_test/test_mix.py +++ b/tests/milvus_python_test/test_mix.py @@ -26,6 +26,7 @@ index_params = {'index_type': IndexType.IVFLAT, 'nlist': 16384} class TestMixBase: # TODO: enable + @pytest.mark.level(2) def test_search_during_createIndex(self, args): loops = 100000 table = gen_unique_str() From 24c18e5ddde40bfffea7694b134d12c71b14e7e2 Mon Sep 17 00:00:00 2001 From: jielinxu Date: Thu, 7 Nov 2019 14:13:52 +0800 Subject: [PATCH 023/103] Update README --- README.md | 5 +- README_CN.md | 173 ++++++++------------------------------------------- 2 files changed, 29 insertions(+), 149 deletions(-) diff --git a/README.md b/README.md index 311d0d6843..5504515c8a 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@ For more detailed introduction of Milvus and its architecture, see [Milvus overv Milvus provides stable [Python](https://github.com/milvus-io/pymilvus), [Java](https://github.com/milvus-io/milvus-sdk-java) and C++ APIs. -Keep up-to-date with newest releases and latest updates by reading Milvus [release notes](https://www.milvus.io/docs/en/release/v0.5.0/). +Keep up-to-date with newest releases and latest updates by reading Milvus [release notes](https://www.milvus.io/docs/en/release/v0.5.1/). ## Get started @@ -56,7 +56,8 @@ To connect with other users and contributors, welcome to join our [Slack channel We greatly appreciate the help of the following people. -- [akihoni](https://github.com/akihoni) found a broken link and a small typo in the README file. +- [akihoni](https://github.com/akihoni) provided the CN version of README, and found a broken link in the doc. +- [goodhamgupta](https://github.com/goodhamgupta) fixed a filename typo in the bootcamp doc. ## Resources diff --git a/README_CN.md b/README_CN.md index 5dad64af9b..88fc5c672c 100644 --- a/README_CN.md +++ b/README_CN.md @@ -1,155 +1,33 @@ ![Milvuslogo](https://raw.githubusercontent.com/milvus-io/docs/master/assets/milvus_logo.png) +[![Slack](https://img.shields.io/badge/Join-Slack-orange)](https://join.slack.com/t/milvusio/shared_invite/enQtNzY1OTQ0NDI3NjMzLWNmYmM1NmNjOTQ5MGI5NDhhYmRhMGU5M2NhNzhhMDMzY2MzNDdlYjM5ODQ5MmE3ODFlYzU3YjJkNmVlNDQ2ZTk) ![LICENSE](https://img.shields.io/badge/license-Apache--2.0-brightgreen) ![Language](https://img.shields.io/badge/language-C%2B%2B-blue) [![codebeat badge](https://codebeat.co/badges/e030a4f6-b126-4475-a938-4723d54ec3a7?style=plastic)](https://codebeat.co/projects/github-com-jinhai-cn-milvus-master) - -![Release](https://img.shields.io/badge/release-v0.5.0-orange) -![Release_date](https://img.shields.io/badge/release_date-October-yellowgreen) - -- [Slack 频道](https://join.slack.com/t/milvusio/shared_invite/enQtNzY1OTQ0NDI3NjMzLWNmYmM1NmNjOTQ5MGI5NDhhYmRhMGU5M2NhNzhhMDMzY2MzNDdlYjM5ODQ5MmE3ODFlYzU3YjJkNmVlNDQ2ZTk) -- [Twitter](https://twitter.com/milvusio) -- [Facebook](https://www.facebook.com/io.milvus.5) -- [博客](https://www.milvus.io/blog/) -- [CSDN](https://zilliz.blog.csdn.net/) -- [中文官网](https://www.milvus.io/zh-CN/) - -# 欢迎来到 Milvus +![Release](https://img.shields.io/badge/release-v0.5.1-yellowgreen) +![Release_date](https://img.shields.io/badge/release%20date-November-yellowgreen) ## Milvus 是什么 Milvus 是一款开源的、针对海量特征向量的相似性搜索引擎。基于异构众核计算框架设计，成本更低，性能更好。在有限的计算资源下，十亿向量搜索仅毫秒响应。 -Milvus 提供稳定的 Python、Java 以及 C++ 的 API 接口。 +若要了解 Milvus 详细介绍和整体架构，请访问 [Milvus 简介](https://www.milvus.io/docs/zh-CN/aboutmilvus/overview/)。 -通过 [版本发布说明](https://milvus.io/docs/zh-CN/release/v0.5.0/) 获取最新发行版本的 Milvus。 +Milvus 提供稳定的 [Python](https://github.com/milvus-io/pymilvus)、[Java](https://github.com/milvus-io/milvus-sdk-java) 以及 C++ 的 API 接口。 -- 异构众核 - - Milvus 基于异构众核计算框架设计，成本更低，性能更好。 - -- 多元化索引 - - Milvus 支持多种索引方式，使用量化索引、基于树的索引和图索引等算法。 - -- 资源智能管理 - - Milvus 根据实际数据规模和可利用资源，智能调节优化查询计算和索引构建过程。 - -- 水平扩容 - - Milvus 支持在线 / 离线扩容，仅需执行简单命令，便可弹性伸缩计算节点和存储节点。 - -- 高可用性 - - Milvus 集成了 Kubernetes 框架，能有效避免单点障碍情况的发生。 - -- 简单易用 - - Milvus 安装简单，使用方便，并可使您专注于特征向量。 - -- 可视化监控 - - 您可以使用基于 Prometheus 的图形化监控，以便实时跟踪系统性能。 - -## 整体架构 - -![Milvus_arch](https://github.com/milvus-io/docs/blob/master/assets/milvus_arch.png) +通过 [版本发布说明](https://milvus.io/docs/zh-CN/release/v0.5.1/) 获取最新版本的功能和更新。 ## 开始使用 Milvus -### 硬件要求 +请参阅 [Milvus 安装指南](https://www.milvus.io/docs/zh-CN/userguide/install_milvus/) 使用 Docker 容器安装 Milvus。若要基于源码编译，请访问 [源码安装](install.md)。 -| 硬件设备 | 推荐配置 | -| -------- | ------------------------------------- | -| CPU | Intel CPU Haswell 及以上 | -| GPU | NVIDIA Pascal 系列及以上 | -| 内存 | 8 GB 或以上（取决于具体向量数据规模） | -| 硬盘 | SATA 3.0 SSD 及以上 | - -### 使用 Docker - -您可以方便地使用 Docker 安装 Milvus。具体请查看 [Milvus 安装指南](https://milvus.io/docs/zh-CN/userguide/install_milvus/)。 - -### 从源代码编译 - -#### 软件要求 - -- Ubuntu 18.04 及以上 -- CMake 3.14 及以上 -- CUDA 10.0 及以上 -- NVIDIA driver 418 及以上 - -#### 编译 - -##### 第一步 安装依赖项 - -```shell -$ cd [Milvus sourcecode path]/core -$ ./ubuntu_build_deps.sh -``` - -##### 第二步 编译 - -```shell -$ cd [Milvus sourcecode path]/core -$ ./build.sh -t Debug -or -$ ./build.sh -t Release -``` - -当您成功编译后，所有 Milvus 必需组件将安装在`[Milvus root path]/core/milvus`路径下。 - -##### 启动 Milvus 服务 - -```shell -$ cd [Milvus root path]/core/milvus -``` - -在 `LD_LIBRARY_PATH` 中添加 `lib/` 目录： - -```shell -$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/milvus/lib -``` - -启动 Milvus 服务： - -```shell -$ cd scripts -$ ./start_server.sh -``` - -若要停止 Milvus 服务，请使用如下命令： - -```shell -$ ./stop_server.sh -``` - -若需要修改 Milvus 配置文件 `conf/server_config.yaml` 和`conf/log_config.conf`，请查看 [Milvus 配置](https://milvus.io/docs/zh-CN/reference/milvus_config/)。 +若要更改 Milvus 设置，请参阅 [Milvus 配置](https://www.milvus.io/docs/zh-CN/reference/milvus_config/)。 ### 开始您的第一个 Milvus 程序 -#### 运行 Python 示例代码 +您可以尝试用 [Python](https://www.milvus.io/docs/en/userguide/example_code/) 或 [Java example code](https://github.com/milvus-io/milvus-sdk-java/tree/master/examples) 运行 Milvus 示例代码。 -请确保系统的 Python 版本为 [Python 3.5](https://www.python.org/downloads/) 或以上。 - -安装 Milvus Python SDK。 - -```shell -# Install Milvus Python SDK -$ pip install pymilvus==0.2.3 -``` - -创建 `example.py` 文件，并向文件中加入 [Python 示例代码](https://github.com/milvus-io/pymilvus/blob/master/examples/advanced_example.py)。 - -运行示例代码 - -```shell -# Run Milvus Python example -$ python3 example.py -``` - -#### 运行 C++ 示例代码 +若要使用 C++ 示例代码，请使用以下命令： ```shell # Run Milvus C++ example @@ -157,39 +35,40 @@ $ python3 example.py $ ./sdk_simple ``` -#### 运行 Java 示例代码 +## 路线图 -请确保系统的 Java 版本为 Java 8 或以上。 - -请从[此处](https://github.com/milvus-io/milvus-sdk-java/tree/master/examples)获取 Java 示例代码。 +请阅读我们的[路线图](https://milvus.io/docs/zh-CN/roadmap/)以了解更多即将开发的新功能。 ## 贡献者指南 -我们由衷欢迎您推送贡献。关于贡献流程的详细信息，请参阅 [贡献者指南](https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md)。本项目遵循 Milvus [行为准则](https://github.com/milvus-io/milvus/blob/master/CODE_OF_CONDUCT.md)。如果您希望参与本项目，请遵守该准则的内容。 +我们由衷欢迎您推送贡献。关于贡献流程的详细信息，请参阅[贡献者指南](https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md)。本项目遵循 Milvus [行为准则](https://github.com/milvus-io/milvus/blob/master/CODE_OF_CONDUCT.md)。如果您希望参与本项目，请遵守该准则的内容。 -我们使用 [GitHub issues](https://github.com/milvus-io/milvus/issues/new/choose) 追踪问题和补丁。若您希望提出问题或进行讨论，请加入我们的社区。 +我们使用 [GitHub issues](https://github.com/milvus-io/milvus/issues) 追踪问题和补丁。若您希望提出问题或进行讨论，请加入我们的社区。 ## 加入 Milvus 社区 -欢迎加入我们的 [Slack 频道](https://join.slack.com/t/milvusio/shared_invite/enQtNzY1OTQ0NDI3NjMzLWNmYmM1NmNjOTQ5MGI5NDhhYmRhMGU5M2NhNzhhMDMzY2MzNDdlYjM5ODQ5MmE3ODFlYzU3YjJkNmVlNDQ2ZTk) 以便与其他用户和贡献者进行交流。 +欢迎加入我们的 [Slack 频道](https://join.slack.com/t/milvusio/shared_invite/enQtNzY1OTQ0NDI3NjMzLWNmYmM1NmNjOTQ5MGI5NDhhYmRhMGU5M2NhNzhhMDMzY2MzNDdlYjM5ODQ5MmE3ODFlYzU3YjJkNmVlNDQ2ZTk)以便与其他用户和贡献者进行交流。 -## Milvus 路线图 +## 贡献者 -请阅读我们的[路线图](https://milvus.io/docs/zh-CN/roadmap/)以获得更多即将开发的新功能。 +以下是 Milvus 贡献者名单，在此我们深表感谢： + +- [akihoni](https://github.com/akihoni) 提供了中文版 README，并发现了 README 中的无效链接。 +- [goodhamgupta](https://github.com/goodhamgupta) 发现并修正了在线训练营文档中的文件名拼写错误。 ## 相关链接 -[Milvus 官方网站](https://www.milvus.io/) +- [Milvus.io](https://www.milvus.io) -[Milvus 文档](https://www.milvus.io/docs/en/userguide/install_milvus/) +- [Milvus 在线训练营](https://github.com/milvus-io/bootcamp) -[Milvus 在线训练营](https://github.com/milvus-io/bootcamp) +- [Milvus Medium](https://medium.com/@milvusio) -[Milvus 博客](https://www.milvus.io/blog/) +- [Milvus CSDN](https://zilliz.blog.csdn.net/) -[Milvus CSDN](https://zilliz.blog.csdn.net/) +- [Milvus Twitter](https://twitter.com/milvusio) -[Milvus 路线图](https://milvus.io/docs/en/roadmap/) +- [Milvus Facebook](https://www.facebook.com/io.milvus.5) ## 许可协议 From 07dfe09aedfb5ed9323915c5eaf6dc8fc2e305c2 Mon Sep 17 00:00:00 2001 From: fishpenguin Date: Thu, 7 Nov 2019 15:23:28 +0800 Subject: [PATCH 024/103] #204 - improve grpc performance in search --- CHANGELOG.md | 1 + core/src/grpc/README.md | 4 + core/src/grpc/gen-milvus/milvus.grpc.pb.cc | 44 +- core/src/grpc/gen-milvus/milvus.grpc.pb.h | 540 ++++----- core/src/grpc/gen-milvus/milvus.pb.cc | 1077 +++++------------ core/src/grpc/gen-milvus/milvus.pb.h | 611 +++------- core/src/grpc/milvus.proto | 24 +- .../examples/grpcsimple/src/ClientTest.cpp | 35 +- core/src/sdk/grpc/ClientProxy.cpp | 37 +- core/src/sdk/grpc/ClientProxy.h | 2 +- core/src/sdk/grpc/GrpcClient.cpp | 10 +- core/src/sdk/grpc/GrpcClient.h | 2 +- core/src/sdk/include/MilvusApi.h | 15 +- core/src/sdk/interface/ConnectionImpl.cpp | 5 +- core/src/sdk/interface/ConnectionImpl.h | 2 +- .../server/grpc_impl/GrpcRequestHandler.cpp | 4 +- .../src/server/grpc_impl/GrpcRequestHandler.h | 4 +- core/src/server/grpc_impl/GrpcRequestTask.cpp | 30 +- core/src/server/grpc_impl/GrpcRequestTask.h | 6 +- core/src/server/grpc_impl/GrpcServer.cpp | 2 +- core/unittest/server/test_rpc.cpp | 22 +- 21 files changed, 878 insertions(+), 1599 deletions(-) create mode 100644 core/src/grpc/README.md diff --git a/CHANGELOG.md b/CHANGELOG.md index d49922b259..aad647ed7b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ Please mark all change in change log and use the ticket from JIRA. ## Feature ## Improvement +- \#204 - improve grpc performance in search - \#207 - Add more unittest for config set/get - \#208 - optimize unittest to support run single test more easily diff --git a/core/src/grpc/README.md b/core/src/grpc/README.md new file mode 100644 index 0000000000..d3056fc094 --- /dev/null +++ b/core/src/grpc/README.md @@ -0,0 +1,4 @@ +We manually add two APIs in "milvus.pd.h": +add_vector_data +add_row_id_array +If proto files need be generated again, remember to re-add above APIs. \ No newline at end of file diff --git a/core/src/grpc/gen-milvus/milvus.grpc.pb.cc b/core/src/grpc/gen-milvus/milvus.grpc.pb.cc index 82a1b99162..9fea388455 100644 --- a/core/src/grpc/gen-milvus/milvus.grpc.pb.cc +++ b/core/src/grpc/gen-milvus/milvus.grpc.pb.cc @@ -201,60 +201,60 @@ void MilvusService::Stub::experimental_async::Insert(::grpc::ClientContext* cont return ::grpc_impl::internal::ClientAsyncResponseReaderFactory< ::milvus::grpc::VectorIds>::Create(channel_.get(), cq, rpcmethod_Insert_, context, request, false); } -::grpc::Status MilvusService::Stub::Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::milvus::grpc::TopKQueryResultList* response) { +::grpc::Status MilvusService::Stub::Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::milvus::grpc::TopKQueryResult* response) { return ::grpc::internal::BlockingUnaryCall(channel_.get(), rpcmethod_Search_, context, request, response); } -void MilvusService::Stub::experimental_async::Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response, std::function f) { +void MilvusService::Stub::experimental_async::Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResult* response, std::function f) { ::grpc_impl::internal::CallbackUnaryCall(stub_->channel_.get(), stub_->rpcmethod_Search_, context, request, response, std::move(f)); } -void MilvusService::Stub::experimental_async::Search(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResultList* response, std::function f) { +void MilvusService::Stub::experimental_async::Search(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResult* response, std::function f) { ::grpc_impl::internal::CallbackUnaryCall(stub_->channel_.get(), stub_->rpcmethod_Search_, context, request, response, std::move(f)); } -void MilvusService::Stub::experimental_async::Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response, ::grpc::experimental::ClientUnaryReactor* reactor) { +void MilvusService::Stub::experimental_async::Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResult* response, ::grpc::experimental::ClientUnaryReactor* reactor) { ::grpc_impl::internal::ClientCallbackUnaryFactory::Create(stub_->channel_.get(), stub_->rpcmethod_Search_, context, request, response, reactor); } -void MilvusService::Stub::experimental_async::Search(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResultList* response, ::grpc::experimental::ClientUnaryReactor* reactor) { +void MilvusService::Stub::experimental_async::Search(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResult* response, ::grpc::experimental::ClientUnaryReactor* reactor) { ::grpc_impl::internal::ClientCallbackUnaryFactory::Create(stub_->channel_.get(), stub_->rpcmethod_Search_, context, request, response, reactor); } -::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>* MilvusService::Stub::AsyncSearchRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) { - return ::grpc_impl::internal::ClientAsyncResponseReaderFactory< ::milvus::grpc::TopKQueryResultList>::Create(channel_.get(), cq, rpcmethod_Search_, context, request, true); +::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>* MilvusService::Stub::AsyncSearchRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) { + return ::grpc_impl::internal::ClientAsyncResponseReaderFactory< ::milvus::grpc::TopKQueryResult>::Create(channel_.get(), cq, rpcmethod_Search_, context, request, true); } -::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>* MilvusService::Stub::PrepareAsyncSearchRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) { - return ::grpc_impl::internal::ClientAsyncResponseReaderFactory< ::milvus::grpc::TopKQueryResultList>::Create(channel_.get(), cq, rpcmethod_Search_, context, request, false); +::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>* MilvusService::Stub::PrepareAsyncSearchRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) { + return ::grpc_impl::internal::ClientAsyncResponseReaderFactory< ::milvus::grpc::TopKQueryResult>::Create(channel_.get(), cq, rpcmethod_Search_, context, request, false); } -::grpc::Status MilvusService::Stub::SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::milvus::grpc::TopKQueryResultList* response) { +::grpc::Status MilvusService::Stub::SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::milvus::grpc::TopKQueryResult* response) { return ::grpc::internal::BlockingUnaryCall(channel_.get(), rpcmethod_SearchInFiles_, context, request, response); } -void MilvusService::Stub::experimental_async::SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response, std::function f) { +void MilvusService::Stub::experimental_async::SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResult* response, std::function f) { ::grpc_impl::internal::CallbackUnaryCall(stub_->channel_.get(), stub_->rpcmethod_SearchInFiles_, context, request, response, std::move(f)); } -void MilvusService::Stub::experimental_async::SearchInFiles(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResultList* response, std::function f) { +void MilvusService::Stub::experimental_async::SearchInFiles(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResult* response, std::function f) { ::grpc_impl::internal::CallbackUnaryCall(stub_->channel_.get(), stub_->rpcmethod_SearchInFiles_, context, request, response, std::move(f)); } -void MilvusService::Stub::experimental_async::SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response, ::grpc::experimental::ClientUnaryReactor* reactor) { +void MilvusService::Stub::experimental_async::SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResult* response, ::grpc::experimental::ClientUnaryReactor* reactor) { ::grpc_impl::internal::ClientCallbackUnaryFactory::Create(stub_->channel_.get(), stub_->rpcmethod_SearchInFiles_, context, request, response, reactor); } -void MilvusService::Stub::experimental_async::SearchInFiles(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResultList* response, ::grpc::experimental::ClientUnaryReactor* reactor) { +void MilvusService::Stub::experimental_async::SearchInFiles(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResult* response, ::grpc::experimental::ClientUnaryReactor* reactor) { ::grpc_impl::internal::ClientCallbackUnaryFactory::Create(stub_->channel_.get(), stub_->rpcmethod_SearchInFiles_, context, request, response, reactor); } -::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>* MilvusService::Stub::AsyncSearchInFilesRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) { - return ::grpc_impl::internal::ClientAsyncResponseReaderFactory< ::milvus::grpc::TopKQueryResultList>::Create(channel_.get(), cq, rpcmethod_SearchInFiles_, context, request, true); +::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>* MilvusService::Stub::AsyncSearchInFilesRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) { + return ::grpc_impl::internal::ClientAsyncResponseReaderFactory< ::milvus::grpc::TopKQueryResult>::Create(channel_.get(), cq, rpcmethod_SearchInFiles_, context, request, true); } -::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>* MilvusService::Stub::PrepareAsyncSearchInFilesRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) { - return ::grpc_impl::internal::ClientAsyncResponseReaderFactory< ::milvus::grpc::TopKQueryResultList>::Create(channel_.get(), cq, rpcmethod_SearchInFiles_, context, request, false); +::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>* MilvusService::Stub::PrepareAsyncSearchInFilesRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) { + return ::grpc_impl::internal::ClientAsyncResponseReaderFactory< ::milvus::grpc::TopKQueryResult>::Create(channel_.get(), cq, rpcmethod_SearchInFiles_, context, request, false); } ::grpc::Status MilvusService::Stub::DescribeTable(::grpc::ClientContext* context, const ::milvus::grpc::TableName& request, ::milvus::grpc::TableSchema* response) { @@ -510,12 +510,12 @@ MilvusService::Service::Service() { AddMethod(new ::grpc::internal::RpcServiceMethod( MilvusService_method_names[5], ::grpc::internal::RpcMethod::NORMAL_RPC, - new ::grpc::internal::RpcMethodHandler< MilvusService::Service, ::milvus::grpc::SearchParam, ::milvus::grpc::TopKQueryResultList>( + new ::grpc::internal::RpcMethodHandler< MilvusService::Service, ::milvus::grpc::SearchParam, ::milvus::grpc::TopKQueryResult>( std::mem_fn(&MilvusService::Service::Search), this))); AddMethod(new ::grpc::internal::RpcServiceMethod( MilvusService_method_names[6], ::grpc::internal::RpcMethod::NORMAL_RPC, - new ::grpc::internal::RpcMethodHandler< MilvusService::Service, ::milvus::grpc::SearchInFilesParam, ::milvus::grpc::TopKQueryResultList>( + new ::grpc::internal::RpcMethodHandler< MilvusService::Service, ::milvus::grpc::SearchInFilesParam, ::milvus::grpc::TopKQueryResult>( std::mem_fn(&MilvusService::Service::SearchInFiles), this))); AddMethod(new ::grpc::internal::RpcServiceMethod( MilvusService_method_names[7], @@ -597,14 +597,14 @@ MilvusService::Service::~Service() { return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } -::grpc::Status MilvusService::Service::Search(::grpc::ServerContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response) { +::grpc::Status MilvusService::Service::Search(::grpc::ServerContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResult* response) { (void) context; (void) request; (void) response; return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } -::grpc::Status MilvusService::Service::SearchInFiles(::grpc::ServerContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response) { +::grpc::Status MilvusService::Service::SearchInFiles(::grpc::ServerContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResult* response) { (void) context; (void) request; (void) response; diff --git a/core/src/grpc/gen-milvus/milvus.grpc.pb.h b/core/src/grpc/gen-milvus/milvus.grpc.pb.h index 8ea2d13c80..4fa38c7abb 100644 --- a/core/src/grpc/gen-milvus/milvus.grpc.pb.h +++ b/core/src/grpc/gen-milvus/milvus.grpc.pb.h @@ -130,12 +130,12 @@ class MilvusService final { // @param topk, how many similarity vectors will be searched. // // @return query result array. - virtual ::grpc::Status Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::milvus::grpc::TopKQueryResultList* response) = 0; - std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResultList>> AsyncSearch(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) { - return std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResultList>>(AsyncSearchRaw(context, request, cq)); + virtual ::grpc::Status Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::milvus::grpc::TopKQueryResult* response) = 0; + std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResult>> AsyncSearch(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) { + return std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResult>>(AsyncSearchRaw(context, request, cq)); } - std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResultList>> PrepareAsyncSearch(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) { - return std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResultList>>(PrepareAsyncSearchRaw(context, request, cq)); + std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResult>> PrepareAsyncSearch(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) { + return std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResult>>(PrepareAsyncSearchRaw(context, request, cq)); } // * // @brief Internal use query interface @@ -148,12 +148,12 @@ class MilvusService final { // @param topk, how many similarity vectors will be searched. // // @return query result array. - virtual ::grpc::Status SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::milvus::grpc::TopKQueryResultList* response) = 0; - std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResultList>> AsyncSearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) { - return std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResultList>>(AsyncSearchInFilesRaw(context, request, cq)); + virtual ::grpc::Status SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::milvus::grpc::TopKQueryResult* response) = 0; + std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResult>> AsyncSearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) { + return std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResult>>(AsyncSearchInFilesRaw(context, request, cq)); } - std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResultList>> PrepareAsyncSearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) { - return std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResultList>>(PrepareAsyncSearchInFilesRaw(context, request, cq)); + std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResult>> PrepareAsyncSearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) { + return std::unique_ptr< ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResult>>(PrepareAsyncSearchInFilesRaw(context, request, cq)); } // * // @brief Get table schema @@ -335,10 +335,10 @@ class MilvusService final { // @param topk, how many similarity vectors will be searched. // // @return query result array. - virtual void Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response, std::function) = 0; - virtual void Search(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResultList* response, std::function) = 0; - virtual void Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response, ::grpc::experimental::ClientUnaryReactor* reactor) = 0; - virtual void Search(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResultList* response, ::grpc::experimental::ClientUnaryReactor* reactor) = 0; + virtual void Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResult* response, std::function) = 0; + virtual void Search(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResult* response, std::function) = 0; + virtual void Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResult* response, ::grpc::experimental::ClientUnaryReactor* reactor) = 0; + virtual void Search(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResult* response, ::grpc::experimental::ClientUnaryReactor* reactor) = 0; // * // @brief Internal use query interface // @@ -350,10 +350,10 @@ class MilvusService final { // @param topk, how many similarity vectors will be searched. // // @return query result array. - virtual void SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response, std::function) = 0; - virtual void SearchInFiles(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResultList* response, std::function) = 0; - virtual void SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response, ::grpc::experimental::ClientUnaryReactor* reactor) = 0; - virtual void SearchInFiles(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResultList* response, ::grpc::experimental::ClientUnaryReactor* reactor) = 0; + virtual void SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResult* response, std::function) = 0; + virtual void SearchInFiles(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResult* response, std::function) = 0; + virtual void SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResult* response, ::grpc::experimental::ClientUnaryReactor* reactor) = 0; + virtual void SearchInFiles(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResult* response, ::grpc::experimental::ClientUnaryReactor* reactor) = 0; // * // @brief Get table schema // @@ -452,10 +452,10 @@ class MilvusService final { virtual ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::Status>* PrepareAsyncCreateIndexRaw(::grpc::ClientContext* context, const ::milvus::grpc::IndexParam& request, ::grpc::CompletionQueue* cq) = 0; virtual ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::VectorIds>* AsyncInsertRaw(::grpc::ClientContext* context, const ::milvus::grpc::InsertParam& request, ::grpc::CompletionQueue* cq) = 0; virtual ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::VectorIds>* PrepareAsyncInsertRaw(::grpc::ClientContext* context, const ::milvus::grpc::InsertParam& request, ::grpc::CompletionQueue* cq) = 0; - virtual ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResultList>* AsyncSearchRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) = 0; - virtual ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResultList>* PrepareAsyncSearchRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) = 0; - virtual ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResultList>* AsyncSearchInFilesRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) = 0; - virtual ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResultList>* PrepareAsyncSearchInFilesRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) = 0; + virtual ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResult>* AsyncSearchRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) = 0; + virtual ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResult>* PrepareAsyncSearchRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) = 0; + virtual ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResult>* AsyncSearchInFilesRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) = 0; + virtual ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TopKQueryResult>* PrepareAsyncSearchInFilesRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) = 0; virtual ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TableSchema>* AsyncDescribeTableRaw(::grpc::ClientContext* context, const ::milvus::grpc::TableName& request, ::grpc::CompletionQueue* cq) = 0; virtual ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TableSchema>* PrepareAsyncDescribeTableRaw(::grpc::ClientContext* context, const ::milvus::grpc::TableName& request, ::grpc::CompletionQueue* cq) = 0; virtual ::grpc::ClientAsyncResponseReaderInterface< ::milvus::grpc::TableRowCount>* AsyncCountTableRaw(::grpc::ClientContext* context, const ::milvus::grpc::TableName& request, ::grpc::CompletionQueue* cq) = 0; @@ -511,19 +511,19 @@ class MilvusService final { std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::VectorIds>> PrepareAsyncInsert(::grpc::ClientContext* context, const ::milvus::grpc::InsertParam& request, ::grpc::CompletionQueue* cq) { return std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::VectorIds>>(PrepareAsyncInsertRaw(context, request, cq)); } - ::grpc::Status Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::milvus::grpc::TopKQueryResultList* response) override; - std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>> AsyncSearch(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) { - return std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>>(AsyncSearchRaw(context, request, cq)); + ::grpc::Status Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::milvus::grpc::TopKQueryResult* response) override; + std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>> AsyncSearch(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) { + return std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>>(AsyncSearchRaw(context, request, cq)); } - std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>> PrepareAsyncSearch(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) { - return std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>>(PrepareAsyncSearchRaw(context, request, cq)); + std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>> PrepareAsyncSearch(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) { + return std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>>(PrepareAsyncSearchRaw(context, request, cq)); } - ::grpc::Status SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::milvus::grpc::TopKQueryResultList* response) override; - std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>> AsyncSearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) { - return std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>>(AsyncSearchInFilesRaw(context, request, cq)); + ::grpc::Status SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::milvus::grpc::TopKQueryResult* response) override; + std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>> AsyncSearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) { + return std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>>(AsyncSearchInFilesRaw(context, request, cq)); } - std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>> PrepareAsyncSearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) { - return std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>>(PrepareAsyncSearchInFilesRaw(context, request, cq)); + std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>> PrepareAsyncSearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) { + return std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>>(PrepareAsyncSearchInFilesRaw(context, request, cq)); } ::grpc::Status DescribeTable(::grpc::ClientContext* context, const ::milvus::grpc::TableName& request, ::milvus::grpc::TableSchema* response) override; std::unique_ptr< ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TableSchema>> AsyncDescribeTable(::grpc::ClientContext* context, const ::milvus::grpc::TableName& request, ::grpc::CompletionQueue* cq) { @@ -604,14 +604,14 @@ class MilvusService final { void Insert(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::VectorIds* response, std::function) override; void Insert(::grpc::ClientContext* context, const ::milvus::grpc::InsertParam* request, ::milvus::grpc::VectorIds* response, ::grpc::experimental::ClientUnaryReactor* reactor) override; void Insert(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::VectorIds* response, ::grpc::experimental::ClientUnaryReactor* reactor) override; - void Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response, std::function) override; - void Search(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResultList* response, std::function) override; - void Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response, ::grpc::experimental::ClientUnaryReactor* reactor) override; - void Search(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResultList* response, ::grpc::experimental::ClientUnaryReactor* reactor) override; - void SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response, std::function) override; - void SearchInFiles(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResultList* response, std::function) override; - void SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response, ::grpc::experimental::ClientUnaryReactor* reactor) override; - void SearchInFiles(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResultList* response, ::grpc::experimental::ClientUnaryReactor* reactor) override; + void Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResult* response, std::function) override; + void Search(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResult* response, std::function) override; + void Search(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResult* response, ::grpc::experimental::ClientUnaryReactor* reactor) override; + void Search(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResult* response, ::grpc::experimental::ClientUnaryReactor* reactor) override; + void SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResult* response, std::function) override; + void SearchInFiles(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResult* response, std::function) override; + void SearchInFiles(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResult* response, ::grpc::experimental::ClientUnaryReactor* reactor) override; + void SearchInFiles(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TopKQueryResult* response, ::grpc::experimental::ClientUnaryReactor* reactor) override; void DescribeTable(::grpc::ClientContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableSchema* response, std::function) override; void DescribeTable(::grpc::ClientContext* context, const ::grpc::ByteBuffer* request, ::milvus::grpc::TableSchema* response, std::function) override; void DescribeTable(::grpc::ClientContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableSchema* response, ::grpc::experimental::ClientUnaryReactor* reactor) override; @@ -665,10 +665,10 @@ class MilvusService final { ::grpc::ClientAsyncResponseReader< ::milvus::grpc::Status>* PrepareAsyncCreateIndexRaw(::grpc::ClientContext* context, const ::milvus::grpc::IndexParam& request, ::grpc::CompletionQueue* cq) override; ::grpc::ClientAsyncResponseReader< ::milvus::grpc::VectorIds>* AsyncInsertRaw(::grpc::ClientContext* context, const ::milvus::grpc::InsertParam& request, ::grpc::CompletionQueue* cq) override; ::grpc::ClientAsyncResponseReader< ::milvus::grpc::VectorIds>* PrepareAsyncInsertRaw(::grpc::ClientContext* context, const ::milvus::grpc::InsertParam& request, ::grpc::CompletionQueue* cq) override; - ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>* AsyncSearchRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) override; - ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>* PrepareAsyncSearchRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) override; - ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>* AsyncSearchInFilesRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) override; - ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResultList>* PrepareAsyncSearchInFilesRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) override; + ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>* AsyncSearchRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) override; + ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>* PrepareAsyncSearchRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchParam& request, ::grpc::CompletionQueue* cq) override; + ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>* AsyncSearchInFilesRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) override; + ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TopKQueryResult>* PrepareAsyncSearchInFilesRaw(::grpc::ClientContext* context, const ::milvus::grpc::SearchInFilesParam& request, ::grpc::CompletionQueue* cq) override; ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TableSchema>* AsyncDescribeTableRaw(::grpc::ClientContext* context, const ::milvus::grpc::TableName& request, ::grpc::CompletionQueue* cq) override; ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TableSchema>* PrepareAsyncDescribeTableRaw(::grpc::ClientContext* context, const ::milvus::grpc::TableName& request, ::grpc::CompletionQueue* cq) override; ::grpc::ClientAsyncResponseReader< ::milvus::grpc::TableRowCount>* AsyncCountTableRaw(::grpc::ClientContext* context, const ::milvus::grpc::TableName& request, ::grpc::CompletionQueue* cq) override; @@ -760,7 +760,7 @@ class MilvusService final { // @param topk, how many similarity vectors will be searched. // // @return query result array. - virtual ::grpc::Status Search(::grpc::ServerContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response); + virtual ::grpc::Status Search(::grpc::ServerContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResult* response); // * // @brief Internal use query interface // @@ -772,7 +772,7 @@ class MilvusService final { // @param topk, how many similarity vectors will be searched. // // @return query result array. - virtual ::grpc::Status SearchInFiles(::grpc::ServerContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response); + virtual ::grpc::Status SearchInFiles(::grpc::ServerContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResult* response); // * // @brief Get table schema // @@ -838,7 +838,7 @@ class MilvusService final { template class WithAsyncMethod_CreateTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_CreateTable() { ::grpc::Service::MarkMethodAsync(0); @@ -847,7 +847,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CreateTable(::grpc::ServerContext* context, const ::milvus::grpc::TableSchema* request, ::milvus::grpc::Status* response) override { + ::grpc::Status CreateTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableSchema* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -858,7 +858,7 @@ class MilvusService final { template class WithAsyncMethod_HasTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_HasTable() { ::grpc::Service::MarkMethodAsync(1); @@ -867,7 +867,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status HasTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::BoolReply* response) override { + ::grpc::Status HasTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::BoolReply* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -878,7 +878,7 @@ class MilvusService final { template class WithAsyncMethod_DropTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_DropTable() { ::grpc::Service::MarkMethodAsync(2); @@ -887,7 +887,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DropTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DropTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -898,7 +898,7 @@ class MilvusService final { template class WithAsyncMethod_CreateIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_CreateIndex() { ::grpc::Service::MarkMethodAsync(3); @@ -907,7 +907,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CreateIndex(::grpc::ServerContext* context, const ::milvus::grpc::IndexParam* request, ::milvus::grpc::Status* response) override { + ::grpc::Status CreateIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::IndexParam* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -918,7 +918,7 @@ class MilvusService final { template class WithAsyncMethod_Insert : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_Insert() { ::grpc::Service::MarkMethodAsync(4); @@ -927,7 +927,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Insert(::grpc::ServerContext* context, const ::milvus::grpc::InsertParam* request, ::milvus::grpc::VectorIds* response) override { + ::grpc::Status Insert(::grpc::ServerContext* /*context*/, const ::milvus::grpc::InsertParam* /*request*/, ::milvus::grpc::VectorIds* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -938,7 +938,7 @@ class MilvusService final { template class WithAsyncMethod_Search : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_Search() { ::grpc::Service::MarkMethodAsync(5); @@ -947,18 +947,18 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Search(::grpc::ServerContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response) override { + ::grpc::Status Search(::grpc::ServerContext* /*context*/, const ::milvus::grpc::SearchParam* /*request*/, ::milvus::grpc::TopKQueryResult* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - void RequestSearch(::grpc::ServerContext* context, ::milvus::grpc::SearchParam* request, ::grpc::ServerAsyncResponseWriter< ::milvus::grpc::TopKQueryResultList>* response, ::grpc::CompletionQueue* new_call_cq, ::grpc::ServerCompletionQueue* notification_cq, void *tag) { + void RequestSearch(::grpc::ServerContext* context, ::milvus::grpc::SearchParam* request, ::grpc::ServerAsyncResponseWriter< ::milvus::grpc::TopKQueryResult>* response, ::grpc::CompletionQueue* new_call_cq, ::grpc::ServerCompletionQueue* notification_cq, void *tag) { ::grpc::Service::RequestAsyncUnary(5, context, request, response, new_call_cq, notification_cq, tag); } }; template class WithAsyncMethod_SearchInFiles : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_SearchInFiles() { ::grpc::Service::MarkMethodAsync(6); @@ -967,18 +967,18 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status SearchInFiles(::grpc::ServerContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response) override { + ::grpc::Status SearchInFiles(::grpc::ServerContext* /*context*/, const ::milvus::grpc::SearchInFilesParam* /*request*/, ::milvus::grpc::TopKQueryResult* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - void RequestSearchInFiles(::grpc::ServerContext* context, ::milvus::grpc::SearchInFilesParam* request, ::grpc::ServerAsyncResponseWriter< ::milvus::grpc::TopKQueryResultList>* response, ::grpc::CompletionQueue* new_call_cq, ::grpc::ServerCompletionQueue* notification_cq, void *tag) { + void RequestSearchInFiles(::grpc::ServerContext* context, ::milvus::grpc::SearchInFilesParam* request, ::grpc::ServerAsyncResponseWriter< ::milvus::grpc::TopKQueryResult>* response, ::grpc::CompletionQueue* new_call_cq, ::grpc::ServerCompletionQueue* notification_cq, void *tag) { ::grpc::Service::RequestAsyncUnary(6, context, request, response, new_call_cq, notification_cq, tag); } }; template class WithAsyncMethod_DescribeTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_DescribeTable() { ::grpc::Service::MarkMethodAsync(7); @@ -987,7 +987,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DescribeTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableSchema* response) override { + ::grpc::Status DescribeTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::TableSchema* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -998,7 +998,7 @@ class MilvusService final { template class WithAsyncMethod_CountTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_CountTable() { ::grpc::Service::MarkMethodAsync(8); @@ -1007,7 +1007,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CountTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableRowCount* response) override { + ::grpc::Status CountTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::TableRowCount* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1018,7 +1018,7 @@ class MilvusService final { template class WithAsyncMethod_ShowTables : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_ShowTables() { ::grpc::Service::MarkMethodAsync(9); @@ -1027,7 +1027,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status ShowTables(::grpc::ServerContext* context, const ::milvus::grpc::Command* request, ::milvus::grpc::TableNameList* response) override { + ::grpc::Status ShowTables(::grpc::ServerContext* /*context*/, const ::milvus::grpc::Command* /*request*/, ::milvus::grpc::TableNameList* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1038,7 +1038,7 @@ class MilvusService final { template class WithAsyncMethod_Cmd : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_Cmd() { ::grpc::Service::MarkMethodAsync(10); @@ -1047,7 +1047,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Cmd(::grpc::ServerContext* context, const ::milvus::grpc::Command* request, ::milvus::grpc::StringReply* response) override { + ::grpc::Status Cmd(::grpc::ServerContext* /*context*/, const ::milvus::grpc::Command* /*request*/, ::milvus::grpc::StringReply* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1058,7 +1058,7 @@ class MilvusService final { template class WithAsyncMethod_DeleteByRange : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_DeleteByRange() { ::grpc::Service::MarkMethodAsync(11); @@ -1067,7 +1067,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DeleteByRange(::grpc::ServerContext* context, const ::milvus::grpc::DeleteByRangeParam* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DeleteByRange(::grpc::ServerContext* /*context*/, const ::milvus::grpc::DeleteByRangeParam* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1078,7 +1078,7 @@ class MilvusService final { template class WithAsyncMethod_PreloadTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_PreloadTable() { ::grpc::Service::MarkMethodAsync(12); @@ -1087,7 +1087,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status PreloadTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status PreloadTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1098,7 +1098,7 @@ class MilvusService final { template class WithAsyncMethod_DescribeIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_DescribeIndex() { ::grpc::Service::MarkMethodAsync(13); @@ -1107,7 +1107,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DescribeIndex(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::IndexParam* response) override { + ::grpc::Status DescribeIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::IndexParam* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1118,7 +1118,7 @@ class MilvusService final { template class WithAsyncMethod_DropIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithAsyncMethod_DropIndex() { ::grpc::Service::MarkMethodAsync(14); @@ -1127,7 +1127,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DropIndex(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DropIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1139,7 +1139,7 @@ class MilvusService final { template class ExperimentalWithCallbackMethod_CreateTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_CreateTable() { ::grpc::Service::experimental().MarkMethodCallback(0, @@ -1161,16 +1161,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CreateTable(::grpc::ServerContext* context, const ::milvus::grpc::TableSchema* request, ::milvus::grpc::Status* response) override { + ::grpc::Status CreateTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableSchema* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void CreateTable(::grpc::ServerContext* context, const ::milvus::grpc::TableSchema* request, ::milvus::grpc::Status* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void CreateTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableSchema* /*request*/, ::milvus::grpc::Status* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithCallbackMethod_HasTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_HasTable() { ::grpc::Service::experimental().MarkMethodCallback(1, @@ -1192,16 +1192,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status HasTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::BoolReply* response) override { + ::grpc::Status HasTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::BoolReply* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void HasTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::BoolReply* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void HasTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::BoolReply* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithCallbackMethod_DropTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_DropTable() { ::grpc::Service::experimental().MarkMethodCallback(2, @@ -1223,16 +1223,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DropTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DropTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void DropTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void DropTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithCallbackMethod_CreateIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_CreateIndex() { ::grpc::Service::experimental().MarkMethodCallback(3, @@ -1254,16 +1254,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CreateIndex(::grpc::ServerContext* context, const ::milvus::grpc::IndexParam* request, ::milvus::grpc::Status* response) override { + ::grpc::Status CreateIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::IndexParam* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void CreateIndex(::grpc::ServerContext* context, const ::milvus::grpc::IndexParam* request, ::milvus::grpc::Status* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void CreateIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::IndexParam* /*request*/, ::milvus::grpc::Status* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithCallbackMethod_Insert : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_Insert() { ::grpc::Service::experimental().MarkMethodCallback(4, @@ -1285,30 +1285,30 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Insert(::grpc::ServerContext* context, const ::milvus::grpc::InsertParam* request, ::milvus::grpc::VectorIds* response) override { + ::grpc::Status Insert(::grpc::ServerContext* /*context*/, const ::milvus::grpc::InsertParam* /*request*/, ::milvus::grpc::VectorIds* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void Insert(::grpc::ServerContext* context, const ::milvus::grpc::InsertParam* request, ::milvus::grpc::VectorIds* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void Insert(::grpc::ServerContext* /*context*/, const ::milvus::grpc::InsertParam* /*request*/, ::milvus::grpc::VectorIds* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithCallbackMethod_Search : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_Search() { ::grpc::Service::experimental().MarkMethodCallback(5, - new ::grpc_impl::internal::CallbackUnaryHandler< ::milvus::grpc::SearchParam, ::milvus::grpc::TopKQueryResultList>( + new ::grpc_impl::internal::CallbackUnaryHandler< ::milvus::grpc::SearchParam, ::milvus::grpc::TopKQueryResult>( [this](::grpc::ServerContext* context, const ::milvus::grpc::SearchParam* request, - ::milvus::grpc::TopKQueryResultList* response, + ::milvus::grpc::TopKQueryResult* response, ::grpc::experimental::ServerCallbackRpcController* controller) { return this->Search(context, request, response, controller); })); } void SetMessageAllocatorFor_Search( - ::grpc::experimental::MessageAllocator< ::milvus::grpc::SearchParam, ::milvus::grpc::TopKQueryResultList>* allocator) { - static_cast<::grpc_impl::internal::CallbackUnaryHandler< ::milvus::grpc::SearchParam, ::milvus::grpc::TopKQueryResultList>*>( + ::grpc::experimental::MessageAllocator< ::milvus::grpc::SearchParam, ::milvus::grpc::TopKQueryResult>* allocator) { + static_cast<::grpc_impl::internal::CallbackUnaryHandler< ::milvus::grpc::SearchParam, ::milvus::grpc::TopKQueryResult>*>( ::grpc::Service::experimental().GetHandler(5)) ->SetMessageAllocator(allocator); } @@ -1316,30 +1316,30 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Search(::grpc::ServerContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response) override { + ::grpc::Status Search(::grpc::ServerContext* /*context*/, const ::milvus::grpc::SearchParam* /*request*/, ::milvus::grpc::TopKQueryResult* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void Search(::grpc::ServerContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void Search(::grpc::ServerContext* /*context*/, const ::milvus::grpc::SearchParam* /*request*/, ::milvus::grpc::TopKQueryResult* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithCallbackMethod_SearchInFiles : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_SearchInFiles() { ::grpc::Service::experimental().MarkMethodCallback(6, - new ::grpc_impl::internal::CallbackUnaryHandler< ::milvus::grpc::SearchInFilesParam, ::milvus::grpc::TopKQueryResultList>( + new ::grpc_impl::internal::CallbackUnaryHandler< ::milvus::grpc::SearchInFilesParam, ::milvus::grpc::TopKQueryResult>( [this](::grpc::ServerContext* context, const ::milvus::grpc::SearchInFilesParam* request, - ::milvus::grpc::TopKQueryResultList* response, + ::milvus::grpc::TopKQueryResult* response, ::grpc::experimental::ServerCallbackRpcController* controller) { return this->SearchInFiles(context, request, response, controller); })); } void SetMessageAllocatorFor_SearchInFiles( - ::grpc::experimental::MessageAllocator< ::milvus::grpc::SearchInFilesParam, ::milvus::grpc::TopKQueryResultList>* allocator) { - static_cast<::grpc_impl::internal::CallbackUnaryHandler< ::milvus::grpc::SearchInFilesParam, ::milvus::grpc::TopKQueryResultList>*>( + ::grpc::experimental::MessageAllocator< ::milvus::grpc::SearchInFilesParam, ::milvus::grpc::TopKQueryResult>* allocator) { + static_cast<::grpc_impl::internal::CallbackUnaryHandler< ::milvus::grpc::SearchInFilesParam, ::milvus::grpc::TopKQueryResult>*>( ::grpc::Service::experimental().GetHandler(6)) ->SetMessageAllocator(allocator); } @@ -1347,16 +1347,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status SearchInFiles(::grpc::ServerContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response) override { + ::grpc::Status SearchInFiles(::grpc::ServerContext* /*context*/, const ::milvus::grpc::SearchInFilesParam* /*request*/, ::milvus::grpc::TopKQueryResult* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void SearchInFiles(::grpc::ServerContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void SearchInFiles(::grpc::ServerContext* /*context*/, const ::milvus::grpc::SearchInFilesParam* /*request*/, ::milvus::grpc::TopKQueryResult* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithCallbackMethod_DescribeTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_DescribeTable() { ::grpc::Service::experimental().MarkMethodCallback(7, @@ -1378,16 +1378,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DescribeTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableSchema* response) override { + ::grpc::Status DescribeTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::TableSchema* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void DescribeTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableSchema* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void DescribeTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::TableSchema* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithCallbackMethod_CountTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_CountTable() { ::grpc::Service::experimental().MarkMethodCallback(8, @@ -1409,16 +1409,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CountTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableRowCount* response) override { + ::grpc::Status CountTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::TableRowCount* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void CountTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableRowCount* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void CountTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::TableRowCount* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithCallbackMethod_ShowTables : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_ShowTables() { ::grpc::Service::experimental().MarkMethodCallback(9, @@ -1440,16 +1440,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status ShowTables(::grpc::ServerContext* context, const ::milvus::grpc::Command* request, ::milvus::grpc::TableNameList* response) override { + ::grpc::Status ShowTables(::grpc::ServerContext* /*context*/, const ::milvus::grpc::Command* /*request*/, ::milvus::grpc::TableNameList* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void ShowTables(::grpc::ServerContext* context, const ::milvus::grpc::Command* request, ::milvus::grpc::TableNameList* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void ShowTables(::grpc::ServerContext* /*context*/, const ::milvus::grpc::Command* /*request*/, ::milvus::grpc::TableNameList* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithCallbackMethod_Cmd : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_Cmd() { ::grpc::Service::experimental().MarkMethodCallback(10, @@ -1471,16 +1471,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Cmd(::grpc::ServerContext* context, const ::milvus::grpc::Command* request, ::milvus::grpc::StringReply* response) override { + ::grpc::Status Cmd(::grpc::ServerContext* /*context*/, const ::milvus::grpc::Command* /*request*/, ::milvus::grpc::StringReply* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void Cmd(::grpc::ServerContext* context, const ::milvus::grpc::Command* request, ::milvus::grpc::StringReply* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void Cmd(::grpc::ServerContext* /*context*/, const ::milvus::grpc::Command* /*request*/, ::milvus::grpc::StringReply* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithCallbackMethod_DeleteByRange : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_DeleteByRange() { ::grpc::Service::experimental().MarkMethodCallback(11, @@ -1502,16 +1502,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DeleteByRange(::grpc::ServerContext* context, const ::milvus::grpc::DeleteByRangeParam* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DeleteByRange(::grpc::ServerContext* /*context*/, const ::milvus::grpc::DeleteByRangeParam* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void DeleteByRange(::grpc::ServerContext* context, const ::milvus::grpc::DeleteByRangeParam* request, ::milvus::grpc::Status* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void DeleteByRange(::grpc::ServerContext* /*context*/, const ::milvus::grpc::DeleteByRangeParam* /*request*/, ::milvus::grpc::Status* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithCallbackMethod_PreloadTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_PreloadTable() { ::grpc::Service::experimental().MarkMethodCallback(12, @@ -1533,16 +1533,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status PreloadTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status PreloadTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void PreloadTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void PreloadTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithCallbackMethod_DescribeIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_DescribeIndex() { ::grpc::Service::experimental().MarkMethodCallback(13, @@ -1564,16 +1564,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DescribeIndex(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::IndexParam* response) override { + ::grpc::Status DescribeIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::IndexParam* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void DescribeIndex(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::IndexParam* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void DescribeIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::IndexParam* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithCallbackMethod_DropIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithCallbackMethod_DropIndex() { ::grpc::Service::experimental().MarkMethodCallback(14, @@ -1595,17 +1595,17 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DropIndex(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DropIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void DropIndex(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void DropIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; typedef ExperimentalWithCallbackMethod_CreateTable > > > > > > > > > > > > > > ExperimentalCallbackService; template class WithGenericMethod_CreateTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_CreateTable() { ::grpc::Service::MarkMethodGeneric(0); @@ -1614,7 +1614,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CreateTable(::grpc::ServerContext* context, const ::milvus::grpc::TableSchema* request, ::milvus::grpc::Status* response) override { + ::grpc::Status CreateTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableSchema* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1622,7 +1622,7 @@ class MilvusService final { template class WithGenericMethod_HasTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_HasTable() { ::grpc::Service::MarkMethodGeneric(1); @@ -1631,7 +1631,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status HasTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::BoolReply* response) override { + ::grpc::Status HasTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::BoolReply* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1639,7 +1639,7 @@ class MilvusService final { template class WithGenericMethod_DropTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_DropTable() { ::grpc::Service::MarkMethodGeneric(2); @@ -1648,7 +1648,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DropTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DropTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1656,7 +1656,7 @@ class MilvusService final { template class WithGenericMethod_CreateIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_CreateIndex() { ::grpc::Service::MarkMethodGeneric(3); @@ -1665,7 +1665,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CreateIndex(::grpc::ServerContext* context, const ::milvus::grpc::IndexParam* request, ::milvus::grpc::Status* response) override { + ::grpc::Status CreateIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::IndexParam* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1673,7 +1673,7 @@ class MilvusService final { template class WithGenericMethod_Insert : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_Insert() { ::grpc::Service::MarkMethodGeneric(4); @@ -1682,7 +1682,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Insert(::grpc::ServerContext* context, const ::milvus::grpc::InsertParam* request, ::milvus::grpc::VectorIds* response) override { + ::grpc::Status Insert(::grpc::ServerContext* /*context*/, const ::milvus::grpc::InsertParam* /*request*/, ::milvus::grpc::VectorIds* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1690,7 +1690,7 @@ class MilvusService final { template class WithGenericMethod_Search : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_Search() { ::grpc::Service::MarkMethodGeneric(5); @@ -1699,7 +1699,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Search(::grpc::ServerContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response) override { + ::grpc::Status Search(::grpc::ServerContext* /*context*/, const ::milvus::grpc::SearchParam* /*request*/, ::milvus::grpc::TopKQueryResult* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1707,7 +1707,7 @@ class MilvusService final { template class WithGenericMethod_SearchInFiles : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_SearchInFiles() { ::grpc::Service::MarkMethodGeneric(6); @@ -1716,7 +1716,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status SearchInFiles(::grpc::ServerContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response) override { + ::grpc::Status SearchInFiles(::grpc::ServerContext* /*context*/, const ::milvus::grpc::SearchInFilesParam* /*request*/, ::milvus::grpc::TopKQueryResult* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1724,7 +1724,7 @@ class MilvusService final { template class WithGenericMethod_DescribeTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_DescribeTable() { ::grpc::Service::MarkMethodGeneric(7); @@ -1733,7 +1733,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DescribeTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableSchema* response) override { + ::grpc::Status DescribeTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::TableSchema* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1741,7 +1741,7 @@ class MilvusService final { template class WithGenericMethod_CountTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_CountTable() { ::grpc::Service::MarkMethodGeneric(8); @@ -1750,7 +1750,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CountTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableRowCount* response) override { + ::grpc::Status CountTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::TableRowCount* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1758,7 +1758,7 @@ class MilvusService final { template class WithGenericMethod_ShowTables : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_ShowTables() { ::grpc::Service::MarkMethodGeneric(9); @@ -1767,7 +1767,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status ShowTables(::grpc::ServerContext* context, const ::milvus::grpc::Command* request, ::milvus::grpc::TableNameList* response) override { + ::grpc::Status ShowTables(::grpc::ServerContext* /*context*/, const ::milvus::grpc::Command* /*request*/, ::milvus::grpc::TableNameList* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1775,7 +1775,7 @@ class MilvusService final { template class WithGenericMethod_Cmd : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_Cmd() { ::grpc::Service::MarkMethodGeneric(10); @@ -1784,7 +1784,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Cmd(::grpc::ServerContext* context, const ::milvus::grpc::Command* request, ::milvus::grpc::StringReply* response) override { + ::grpc::Status Cmd(::grpc::ServerContext* /*context*/, const ::milvus::grpc::Command* /*request*/, ::milvus::grpc::StringReply* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1792,7 +1792,7 @@ class MilvusService final { template class WithGenericMethod_DeleteByRange : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_DeleteByRange() { ::grpc::Service::MarkMethodGeneric(11); @@ -1801,7 +1801,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DeleteByRange(::grpc::ServerContext* context, const ::milvus::grpc::DeleteByRangeParam* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DeleteByRange(::grpc::ServerContext* /*context*/, const ::milvus::grpc::DeleteByRangeParam* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1809,7 +1809,7 @@ class MilvusService final { template class WithGenericMethod_PreloadTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_PreloadTable() { ::grpc::Service::MarkMethodGeneric(12); @@ -1818,7 +1818,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status PreloadTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status PreloadTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1826,7 +1826,7 @@ class MilvusService final { template class WithGenericMethod_DescribeIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_DescribeIndex() { ::grpc::Service::MarkMethodGeneric(13); @@ -1835,7 +1835,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DescribeIndex(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::IndexParam* response) override { + ::grpc::Status DescribeIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::IndexParam* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1843,7 +1843,7 @@ class MilvusService final { template class WithGenericMethod_DropIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithGenericMethod_DropIndex() { ::grpc::Service::MarkMethodGeneric(14); @@ -1852,7 +1852,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DropIndex(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DropIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1860,7 +1860,7 @@ class MilvusService final { template class WithRawMethod_CreateTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_CreateTable() { ::grpc::Service::MarkMethodRaw(0); @@ -1869,7 +1869,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CreateTable(::grpc::ServerContext* context, const ::milvus::grpc::TableSchema* request, ::milvus::grpc::Status* response) override { + ::grpc::Status CreateTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableSchema* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1880,7 +1880,7 @@ class MilvusService final { template class WithRawMethod_HasTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_HasTable() { ::grpc::Service::MarkMethodRaw(1); @@ -1889,7 +1889,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status HasTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::BoolReply* response) override { + ::grpc::Status HasTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::BoolReply* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1900,7 +1900,7 @@ class MilvusService final { template class WithRawMethod_DropTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_DropTable() { ::grpc::Service::MarkMethodRaw(2); @@ -1909,7 +1909,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DropTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DropTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1920,7 +1920,7 @@ class MilvusService final { template class WithRawMethod_CreateIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_CreateIndex() { ::grpc::Service::MarkMethodRaw(3); @@ -1929,7 +1929,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CreateIndex(::grpc::ServerContext* context, const ::milvus::grpc::IndexParam* request, ::milvus::grpc::Status* response) override { + ::grpc::Status CreateIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::IndexParam* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1940,7 +1940,7 @@ class MilvusService final { template class WithRawMethod_Insert : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_Insert() { ::grpc::Service::MarkMethodRaw(4); @@ -1949,7 +1949,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Insert(::grpc::ServerContext* context, const ::milvus::grpc::InsertParam* request, ::milvus::grpc::VectorIds* response) override { + ::grpc::Status Insert(::grpc::ServerContext* /*context*/, const ::milvus::grpc::InsertParam* /*request*/, ::milvus::grpc::VectorIds* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1960,7 +1960,7 @@ class MilvusService final { template class WithRawMethod_Search : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_Search() { ::grpc::Service::MarkMethodRaw(5); @@ -1969,7 +1969,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Search(::grpc::ServerContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response) override { + ::grpc::Status Search(::grpc::ServerContext* /*context*/, const ::milvus::grpc::SearchParam* /*request*/, ::milvus::grpc::TopKQueryResult* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -1980,7 +1980,7 @@ class MilvusService final { template class WithRawMethod_SearchInFiles : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_SearchInFiles() { ::grpc::Service::MarkMethodRaw(6); @@ -1989,7 +1989,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status SearchInFiles(::grpc::ServerContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response) override { + ::grpc::Status SearchInFiles(::grpc::ServerContext* /*context*/, const ::milvus::grpc::SearchInFilesParam* /*request*/, ::milvus::grpc::TopKQueryResult* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2000,7 +2000,7 @@ class MilvusService final { template class WithRawMethod_DescribeTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_DescribeTable() { ::grpc::Service::MarkMethodRaw(7); @@ -2009,7 +2009,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DescribeTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableSchema* response) override { + ::grpc::Status DescribeTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::TableSchema* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2020,7 +2020,7 @@ class MilvusService final { template class WithRawMethod_CountTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_CountTable() { ::grpc::Service::MarkMethodRaw(8); @@ -2029,7 +2029,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CountTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableRowCount* response) override { + ::grpc::Status CountTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::TableRowCount* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2040,7 +2040,7 @@ class MilvusService final { template class WithRawMethod_ShowTables : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_ShowTables() { ::grpc::Service::MarkMethodRaw(9); @@ -2049,7 +2049,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status ShowTables(::grpc::ServerContext* context, const ::milvus::grpc::Command* request, ::milvus::grpc::TableNameList* response) override { + ::grpc::Status ShowTables(::grpc::ServerContext* /*context*/, const ::milvus::grpc::Command* /*request*/, ::milvus::grpc::TableNameList* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2060,7 +2060,7 @@ class MilvusService final { template class WithRawMethod_Cmd : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_Cmd() { ::grpc::Service::MarkMethodRaw(10); @@ -2069,7 +2069,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Cmd(::grpc::ServerContext* context, const ::milvus::grpc::Command* request, ::milvus::grpc::StringReply* response) override { + ::grpc::Status Cmd(::grpc::ServerContext* /*context*/, const ::milvus::grpc::Command* /*request*/, ::milvus::grpc::StringReply* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2080,7 +2080,7 @@ class MilvusService final { template class WithRawMethod_DeleteByRange : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_DeleteByRange() { ::grpc::Service::MarkMethodRaw(11); @@ -2089,7 +2089,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DeleteByRange(::grpc::ServerContext* context, const ::milvus::grpc::DeleteByRangeParam* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DeleteByRange(::grpc::ServerContext* /*context*/, const ::milvus::grpc::DeleteByRangeParam* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2100,7 +2100,7 @@ class MilvusService final { template class WithRawMethod_PreloadTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_PreloadTable() { ::grpc::Service::MarkMethodRaw(12); @@ -2109,7 +2109,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status PreloadTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status PreloadTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2120,7 +2120,7 @@ class MilvusService final { template class WithRawMethod_DescribeIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_DescribeIndex() { ::grpc::Service::MarkMethodRaw(13); @@ -2129,7 +2129,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DescribeIndex(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::IndexParam* response) override { + ::grpc::Status DescribeIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::IndexParam* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2140,7 +2140,7 @@ class MilvusService final { template class WithRawMethod_DropIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithRawMethod_DropIndex() { ::grpc::Service::MarkMethodRaw(14); @@ -2149,7 +2149,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DropIndex(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DropIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2160,7 +2160,7 @@ class MilvusService final { template class ExperimentalWithRawCallbackMethod_CreateTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_CreateTable() { ::grpc::Service::experimental().MarkMethodRawCallback(0, @@ -2176,16 +2176,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CreateTable(::grpc::ServerContext* context, const ::milvus::grpc::TableSchema* request, ::milvus::grpc::Status* response) override { + ::grpc::Status CreateTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableSchema* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void CreateTable(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void CreateTable(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithRawCallbackMethod_HasTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_HasTable() { ::grpc::Service::experimental().MarkMethodRawCallback(1, @@ -2201,16 +2201,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status HasTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::BoolReply* response) override { + ::grpc::Status HasTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::BoolReply* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void HasTable(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void HasTable(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithRawCallbackMethod_DropTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_DropTable() { ::grpc::Service::experimental().MarkMethodRawCallback(2, @@ -2226,16 +2226,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DropTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DropTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void DropTable(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void DropTable(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithRawCallbackMethod_CreateIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_CreateIndex() { ::grpc::Service::experimental().MarkMethodRawCallback(3, @@ -2251,16 +2251,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CreateIndex(::grpc::ServerContext* context, const ::milvus::grpc::IndexParam* request, ::milvus::grpc::Status* response) override { + ::grpc::Status CreateIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::IndexParam* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void CreateIndex(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void CreateIndex(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithRawCallbackMethod_Insert : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_Insert() { ::grpc::Service::experimental().MarkMethodRawCallback(4, @@ -2276,16 +2276,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Insert(::grpc::ServerContext* context, const ::milvus::grpc::InsertParam* request, ::milvus::grpc::VectorIds* response) override { + ::grpc::Status Insert(::grpc::ServerContext* /*context*/, const ::milvus::grpc::InsertParam* /*request*/, ::milvus::grpc::VectorIds* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void Insert(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void Insert(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithRawCallbackMethod_Search : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_Search() { ::grpc::Service::experimental().MarkMethodRawCallback(5, @@ -2301,16 +2301,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Search(::grpc::ServerContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response) override { + ::grpc::Status Search(::grpc::ServerContext* /*context*/, const ::milvus::grpc::SearchParam* /*request*/, ::milvus::grpc::TopKQueryResult* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void Search(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void Search(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithRawCallbackMethod_SearchInFiles : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_SearchInFiles() { ::grpc::Service::experimental().MarkMethodRawCallback(6, @@ -2326,16 +2326,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status SearchInFiles(::grpc::ServerContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response) override { + ::grpc::Status SearchInFiles(::grpc::ServerContext* /*context*/, const ::milvus::grpc::SearchInFilesParam* /*request*/, ::milvus::grpc::TopKQueryResult* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void SearchInFiles(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void SearchInFiles(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithRawCallbackMethod_DescribeTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_DescribeTable() { ::grpc::Service::experimental().MarkMethodRawCallback(7, @@ -2351,16 +2351,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DescribeTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableSchema* response) override { + ::grpc::Status DescribeTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::TableSchema* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void DescribeTable(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void DescribeTable(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithRawCallbackMethod_CountTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_CountTable() { ::grpc::Service::experimental().MarkMethodRawCallback(8, @@ -2376,16 +2376,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status CountTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableRowCount* response) override { + ::grpc::Status CountTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::TableRowCount* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void CountTable(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void CountTable(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithRawCallbackMethod_ShowTables : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_ShowTables() { ::grpc::Service::experimental().MarkMethodRawCallback(9, @@ -2401,16 +2401,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status ShowTables(::grpc::ServerContext* context, const ::milvus::grpc::Command* request, ::milvus::grpc::TableNameList* response) override { + ::grpc::Status ShowTables(::grpc::ServerContext* /*context*/, const ::milvus::grpc::Command* /*request*/, ::milvus::grpc::TableNameList* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void ShowTables(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void ShowTables(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithRawCallbackMethod_Cmd : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_Cmd() { ::grpc::Service::experimental().MarkMethodRawCallback(10, @@ -2426,16 +2426,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status Cmd(::grpc::ServerContext* context, const ::milvus::grpc::Command* request, ::milvus::grpc::StringReply* response) override { + ::grpc::Status Cmd(::grpc::ServerContext* /*context*/, const ::milvus::grpc::Command* /*request*/, ::milvus::grpc::StringReply* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void Cmd(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void Cmd(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithRawCallbackMethod_DeleteByRange : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_DeleteByRange() { ::grpc::Service::experimental().MarkMethodRawCallback(11, @@ -2451,16 +2451,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DeleteByRange(::grpc::ServerContext* context, const ::milvus::grpc::DeleteByRangeParam* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DeleteByRange(::grpc::ServerContext* /*context*/, const ::milvus::grpc::DeleteByRangeParam* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void DeleteByRange(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void DeleteByRange(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithRawCallbackMethod_PreloadTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_PreloadTable() { ::grpc::Service::experimental().MarkMethodRawCallback(12, @@ -2476,16 +2476,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status PreloadTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status PreloadTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void PreloadTable(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void PreloadTable(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithRawCallbackMethod_DescribeIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_DescribeIndex() { ::grpc::Service::experimental().MarkMethodRawCallback(13, @@ -2501,16 +2501,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DescribeIndex(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::IndexParam* response) override { + ::grpc::Status DescribeIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::IndexParam* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void DescribeIndex(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void DescribeIndex(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class ExperimentalWithRawCallbackMethod_DropIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: ExperimentalWithRawCallbackMethod_DropIndex() { ::grpc::Service::experimental().MarkMethodRawCallback(14, @@ -2526,16 +2526,16 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable synchronous version of this method - ::grpc::Status DropIndex(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DropIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } - virtual void DropIndex(::grpc::ServerContext* context, const ::grpc::ByteBuffer* request, ::grpc::ByteBuffer* response, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } + virtual void DropIndex(::grpc::ServerContext* /*context*/, const ::grpc::ByteBuffer* /*request*/, ::grpc::ByteBuffer* /*response*/, ::grpc::experimental::ServerCallbackRpcController* controller) { controller->Finish(::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, "")); } }; template class WithStreamedUnaryMethod_CreateTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_CreateTable() { ::grpc::Service::MarkMethodStreamed(0, @@ -2545,7 +2545,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status CreateTable(::grpc::ServerContext* context, const ::milvus::grpc::TableSchema* request, ::milvus::grpc::Status* response) override { + ::grpc::Status CreateTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableSchema* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2555,7 +2555,7 @@ class MilvusService final { template class WithStreamedUnaryMethod_HasTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_HasTable() { ::grpc::Service::MarkMethodStreamed(1, @@ -2565,7 +2565,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status HasTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::BoolReply* response) override { + ::grpc::Status HasTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::BoolReply* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2575,7 +2575,7 @@ class MilvusService final { template class WithStreamedUnaryMethod_DropTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_DropTable() { ::grpc::Service::MarkMethodStreamed(2, @@ -2585,7 +2585,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status DropTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DropTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2595,7 +2595,7 @@ class MilvusService final { template class WithStreamedUnaryMethod_CreateIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_CreateIndex() { ::grpc::Service::MarkMethodStreamed(3, @@ -2605,7 +2605,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status CreateIndex(::grpc::ServerContext* context, const ::milvus::grpc::IndexParam* request, ::milvus::grpc::Status* response) override { + ::grpc::Status CreateIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::IndexParam* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2615,7 +2615,7 @@ class MilvusService final { template class WithStreamedUnaryMethod_Insert : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_Insert() { ::grpc::Service::MarkMethodStreamed(4, @@ -2625,7 +2625,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status Insert(::grpc::ServerContext* context, const ::milvus::grpc::InsertParam* request, ::milvus::grpc::VectorIds* response) override { + ::grpc::Status Insert(::grpc::ServerContext* /*context*/, const ::milvus::grpc::InsertParam* /*request*/, ::milvus::grpc::VectorIds* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2635,47 +2635,47 @@ class MilvusService final { template class WithStreamedUnaryMethod_Search : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_Search() { ::grpc::Service::MarkMethodStreamed(5, - new ::grpc::internal::StreamedUnaryHandler< ::milvus::grpc::SearchParam, ::milvus::grpc::TopKQueryResultList>(std::bind(&WithStreamedUnaryMethod_Search::StreamedSearch, this, std::placeholders::_1, std::placeholders::_2))); + new ::grpc::internal::StreamedUnaryHandler< ::milvus::grpc::SearchParam, ::milvus::grpc::TopKQueryResult>(std::bind(&WithStreamedUnaryMethod_Search::StreamedSearch, this, std::placeholders::_1, std::placeholders::_2))); } ~WithStreamedUnaryMethod_Search() override { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status Search(::grpc::ServerContext* context, const ::milvus::grpc::SearchParam* request, ::milvus::grpc::TopKQueryResultList* response) override { + ::grpc::Status Search(::grpc::ServerContext* /*context*/, const ::milvus::grpc::SearchParam* /*request*/, ::milvus::grpc::TopKQueryResult* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } // replace default version of method with streamed unary - virtual ::grpc::Status StreamedSearch(::grpc::ServerContext* context, ::grpc::ServerUnaryStreamer< ::milvus::grpc::SearchParam,::milvus::grpc::TopKQueryResultList>* server_unary_streamer) = 0; + virtual ::grpc::Status StreamedSearch(::grpc::ServerContext* context, ::grpc::ServerUnaryStreamer< ::milvus::grpc::SearchParam,::milvus::grpc::TopKQueryResult>* server_unary_streamer) = 0; }; template class WithStreamedUnaryMethod_SearchInFiles : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_SearchInFiles() { ::grpc::Service::MarkMethodStreamed(6, - new ::grpc::internal::StreamedUnaryHandler< ::milvus::grpc::SearchInFilesParam, ::milvus::grpc::TopKQueryResultList>(std::bind(&WithStreamedUnaryMethod_SearchInFiles::StreamedSearchInFiles, this, std::placeholders::_1, std::placeholders::_2))); + new ::grpc::internal::StreamedUnaryHandler< ::milvus::grpc::SearchInFilesParam, ::milvus::grpc::TopKQueryResult>(std::bind(&WithStreamedUnaryMethod_SearchInFiles::StreamedSearchInFiles, this, std::placeholders::_1, std::placeholders::_2))); } ~WithStreamedUnaryMethod_SearchInFiles() override { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status SearchInFiles(::grpc::ServerContext* context, const ::milvus::grpc::SearchInFilesParam* request, ::milvus::grpc::TopKQueryResultList* response) override { + ::grpc::Status SearchInFiles(::grpc::ServerContext* /*context*/, const ::milvus::grpc::SearchInFilesParam* /*request*/, ::milvus::grpc::TopKQueryResult* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } // replace default version of method with streamed unary - virtual ::grpc::Status StreamedSearchInFiles(::grpc::ServerContext* context, ::grpc::ServerUnaryStreamer< ::milvus::grpc::SearchInFilesParam,::milvus::grpc::TopKQueryResultList>* server_unary_streamer) = 0; + virtual ::grpc::Status StreamedSearchInFiles(::grpc::ServerContext* context, ::grpc::ServerUnaryStreamer< ::milvus::grpc::SearchInFilesParam,::milvus::grpc::TopKQueryResult>* server_unary_streamer) = 0; }; template class WithStreamedUnaryMethod_DescribeTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_DescribeTable() { ::grpc::Service::MarkMethodStreamed(7, @@ -2685,7 +2685,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status DescribeTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableSchema* response) override { + ::grpc::Status DescribeTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::TableSchema* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2695,7 +2695,7 @@ class MilvusService final { template class WithStreamedUnaryMethod_CountTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_CountTable() { ::grpc::Service::MarkMethodStreamed(8, @@ -2705,7 +2705,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status CountTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::TableRowCount* response) override { + ::grpc::Status CountTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::TableRowCount* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2715,7 +2715,7 @@ class MilvusService final { template class WithStreamedUnaryMethod_ShowTables : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_ShowTables() { ::grpc::Service::MarkMethodStreamed(9, @@ -2725,7 +2725,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status ShowTables(::grpc::ServerContext* context, const ::milvus::grpc::Command* request, ::milvus::grpc::TableNameList* response) override { + ::grpc::Status ShowTables(::grpc::ServerContext* /*context*/, const ::milvus::grpc::Command* /*request*/, ::milvus::grpc::TableNameList* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2735,7 +2735,7 @@ class MilvusService final { template class WithStreamedUnaryMethod_Cmd : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_Cmd() { ::grpc::Service::MarkMethodStreamed(10, @@ -2745,7 +2745,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status Cmd(::grpc::ServerContext* context, const ::milvus::grpc::Command* request, ::milvus::grpc::StringReply* response) override { + ::grpc::Status Cmd(::grpc::ServerContext* /*context*/, const ::milvus::grpc::Command* /*request*/, ::milvus::grpc::StringReply* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2755,7 +2755,7 @@ class MilvusService final { template class WithStreamedUnaryMethod_DeleteByRange : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_DeleteByRange() { ::grpc::Service::MarkMethodStreamed(11, @@ -2765,7 +2765,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status DeleteByRange(::grpc::ServerContext* context, const ::milvus::grpc::DeleteByRangeParam* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DeleteByRange(::grpc::ServerContext* /*context*/, const ::milvus::grpc::DeleteByRangeParam* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2775,7 +2775,7 @@ class MilvusService final { template class WithStreamedUnaryMethod_PreloadTable : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_PreloadTable() { ::grpc::Service::MarkMethodStreamed(12, @@ -2785,7 +2785,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status PreloadTable(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status PreloadTable(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2795,7 +2795,7 @@ class MilvusService final { template class WithStreamedUnaryMethod_DescribeIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_DescribeIndex() { ::grpc::Service::MarkMethodStreamed(13, @@ -2805,7 +2805,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status DescribeIndex(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::IndexParam* response) override { + ::grpc::Status DescribeIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::IndexParam* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } @@ -2815,7 +2815,7 @@ class MilvusService final { template class WithStreamedUnaryMethod_DropIndex : public BaseClass { private: - void BaseClassMustBeDerivedFromService(const Service *service) {} + void BaseClassMustBeDerivedFromService(const Service* /*service*/) {} public: WithStreamedUnaryMethod_DropIndex() { ::grpc::Service::MarkMethodStreamed(14, @@ -2825,7 +2825,7 @@ class MilvusService final { BaseClassMustBeDerivedFromService(this); } // disable regular version of this method - ::grpc::Status DropIndex(::grpc::ServerContext* context, const ::milvus::grpc::TableName* request, ::milvus::grpc::Status* response) override { + ::grpc::Status DropIndex(::grpc::ServerContext* /*context*/, const ::milvus::grpc::TableName* /*request*/, ::milvus::grpc::Status* /*response*/) override { abort(); return ::grpc::Status(::grpc::StatusCode::UNIMPLEMENTED, ""); } diff --git a/core/src/grpc/gen-milvus/milvus.pb.cc b/core/src/grpc/gen-milvus/milvus.pb.cc index fe416a4773..6cdee4a2e6 100644 --- a/core/src/grpc/gen-milvus/milvus.pb.cc +++ b/core/src/grpc/gen-milvus/milvus.pb.cc @@ -16,12 +16,10 @@ // @@protoc_insertion_point(includes) #include extern PROTOBUF_INTERNAL_EXPORT_milvus_2eproto ::PROTOBUF_NAMESPACE_ID::internal::SCCInfo<0> scc_info_Index_milvus_2eproto; -extern PROTOBUF_INTERNAL_EXPORT_milvus_2eproto ::PROTOBUF_NAMESPACE_ID::internal::SCCInfo<0> scc_info_QueryResult_milvus_2eproto; extern PROTOBUF_INTERNAL_EXPORT_milvus_2eproto ::PROTOBUF_NAMESPACE_ID::internal::SCCInfo<0> scc_info_Range_milvus_2eproto; extern PROTOBUF_INTERNAL_EXPORT_milvus_2eproto ::PROTOBUF_NAMESPACE_ID::internal::SCCInfo<0> scc_info_RowRecord_milvus_2eproto; extern PROTOBUF_INTERNAL_EXPORT_milvus_2eproto ::PROTOBUF_NAMESPACE_ID::internal::SCCInfo<2> scc_info_SearchParam_milvus_2eproto; extern PROTOBUF_INTERNAL_EXPORT_status_2eproto ::PROTOBUF_NAMESPACE_ID::internal::SCCInfo<0> scc_info_Status_status_2eproto; -extern PROTOBUF_INTERNAL_EXPORT_milvus_2eproto ::PROTOBUF_NAMESPACE_ID::internal::SCCInfo<1> scc_info_TopKQueryResult_milvus_2eproto; namespace milvus { namespace grpc { class TableNameDefaultTypeInternal { @@ -60,18 +58,10 @@ class SearchInFilesParamDefaultTypeInternal { public: ::PROTOBUF_NAMESPACE_ID::internal::ExplicitlyConstructed _instance; } _SearchInFilesParam_default_instance_; -class QueryResultDefaultTypeInternal { - public: - ::PROTOBUF_NAMESPACE_ID::internal::ExplicitlyConstructed _instance; -} _QueryResult_default_instance_; class TopKQueryResultDefaultTypeInternal { public: ::PROTOBUF_NAMESPACE_ID::internal::ExplicitlyConstructed _instance; } _TopKQueryResult_default_instance_; -class TopKQueryResultListDefaultTypeInternal { - public: - ::PROTOBUF_NAMESPACE_ID::internal::ExplicitlyConstructed _instance; -} _TopKQueryResultList_default_instance_; class StringReplyDefaultTypeInternal { public: ::PROTOBUF_NAMESPACE_ID::internal::ExplicitlyConstructed _instance; @@ -191,20 +181,6 @@ static void InitDefaultsscc_info_InsertParam_milvus_2eproto() { {{ATOMIC_VAR_INIT(::PROTOBUF_NAMESPACE_ID::internal::SCCInfoBase::kUninitialized), 1, InitDefaultsscc_info_InsertParam_milvus_2eproto}, { &scc_info_RowRecord_milvus_2eproto.base,}}; -static void InitDefaultsscc_info_QueryResult_milvus_2eproto() { - GOOGLE_PROTOBUF_VERIFY_VERSION; - - { - void* ptr = &::milvus::grpc::_QueryResult_default_instance_; - new (ptr) ::milvus::grpc::QueryResult(); - ::PROTOBUF_NAMESPACE_ID::internal::OnShutdownDestroyMessage(ptr); - } - ::milvus::grpc::QueryResult::InitAsDefaultInstance(); -} - -::PROTOBUF_NAMESPACE_ID::internal::SCCInfo<0> scc_info_QueryResult_milvus_2eproto = - {{ATOMIC_VAR_INIT(::PROTOBUF_NAMESPACE_ID::internal::SCCInfoBase::kUninitialized), 0, InitDefaultsscc_info_QueryResult_milvus_2eproto}, {}}; - static void InitDefaultsscc_info_Range_milvus_2eproto() { GOOGLE_PROTOBUF_VERIFY_VERSION; @@ -351,23 +327,7 @@ static void InitDefaultsscc_info_TopKQueryResult_milvus_2eproto() { ::PROTOBUF_NAMESPACE_ID::internal::SCCInfo<1> scc_info_TopKQueryResult_milvus_2eproto = {{ATOMIC_VAR_INIT(::PROTOBUF_NAMESPACE_ID::internal::SCCInfoBase::kUninitialized), 1, InitDefaultsscc_info_TopKQueryResult_milvus_2eproto}, { - &scc_info_QueryResult_milvus_2eproto.base,}}; - -static void InitDefaultsscc_info_TopKQueryResultList_milvus_2eproto() { - GOOGLE_PROTOBUF_VERIFY_VERSION; - - { - void* ptr = &::milvus::grpc::_TopKQueryResultList_default_instance_; - new (ptr) ::milvus::grpc::TopKQueryResultList(); - ::PROTOBUF_NAMESPACE_ID::internal::OnShutdownDestroyMessage(ptr); - } - ::milvus::grpc::TopKQueryResultList::InitAsDefaultInstance(); -} - -::PROTOBUF_NAMESPACE_ID::internal::SCCInfo<2> scc_info_TopKQueryResultList_milvus_2eproto = - {{ATOMIC_VAR_INIT(::PROTOBUF_NAMESPACE_ID::internal::SCCInfoBase::kUninitialized), 2, InitDefaultsscc_info_TopKQueryResultList_milvus_2eproto}, { - &scc_info_Status_status_2eproto.base, - &scc_info_TopKQueryResult_milvus_2eproto.base,}}; + &scc_info_Status_status_2eproto.base,}}; static void InitDefaultsscc_info_VectorIds_milvus_2eproto() { GOOGLE_PROTOBUF_VERIFY_VERSION; @@ -384,7 +344,7 @@ static void InitDefaultsscc_info_VectorIds_milvus_2eproto() { {{ATOMIC_VAR_INIT(::PROTOBUF_NAMESPACE_ID::internal::SCCInfoBase::kUninitialized), 1, InitDefaultsscc_info_VectorIds_milvus_2eproto}, { &scc_info_Status_status_2eproto.base,}}; -static ::PROTOBUF_NAMESPACE_ID::Metadata file_level_metadata_milvus_2eproto[19]; +static ::PROTOBUF_NAMESPACE_ID::Metadata file_level_metadata_milvus_2eproto[17]; static constexpr ::PROTOBUF_NAMESPACE_ID::EnumDescriptor const** file_level_enum_descriptors_milvus_2eproto = nullptr; static constexpr ::PROTOBUF_NAMESPACE_ID::ServiceDescriptor const** file_level_service_descriptors_milvus_2eproto = nullptr; @@ -458,25 +418,15 @@ const ::PROTOBUF_NAMESPACE_ID::uint32 TableStruct_milvus_2eproto::offsets[] PROT PROTOBUF_FIELD_OFFSET(::milvus::grpc::SearchInFilesParam, file_id_array_), PROTOBUF_FIELD_OFFSET(::milvus::grpc::SearchInFilesParam, search_param_), ~0u, // no _has_bits_ - PROTOBUF_FIELD_OFFSET(::milvus::grpc::QueryResult, _internal_metadata_), - ~0u, // no _extensions_ - ~0u, // no _oneof_case_ - ~0u, // no _weak_field_map_ - PROTOBUF_FIELD_OFFSET(::milvus::grpc::QueryResult, id_), - PROTOBUF_FIELD_OFFSET(::milvus::grpc::QueryResult, distance_), - ~0u, // no _has_bits_ PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, _internal_metadata_), ~0u, // no _extensions_ ~0u, // no _oneof_case_ ~0u, // no _weak_field_map_ - PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, query_result_arrays_), - ~0u, // no _has_bits_ - PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResultList, _internal_metadata_), - ~0u, // no _extensions_ - ~0u, // no _oneof_case_ - ~0u, // no _weak_field_map_ - PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResultList, status_), - PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResultList, topk_query_result_), + PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, status_), + PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, nq_), + PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, topk_), + PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, ids_binary_), + PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, distances_binary_), ~0u, // no _has_bits_ PROTOBUF_FIELD_OFFSET(::milvus::grpc::StringReply, _internal_metadata_), ~0u, // no _extensions_ @@ -537,16 +487,14 @@ static const ::PROTOBUF_NAMESPACE_ID::internal::MigrationSchema schemas[] PROTOB { 44, -1, sizeof(::milvus::grpc::VectorIds)}, { 51, -1, sizeof(::milvus::grpc::SearchParam)}, { 61, -1, sizeof(::milvus::grpc::SearchInFilesParam)}, - { 68, -1, sizeof(::milvus::grpc::QueryResult)}, - { 75, -1, sizeof(::milvus::grpc::TopKQueryResult)}, - { 81, -1, sizeof(::milvus::grpc::TopKQueryResultList)}, - { 88, -1, sizeof(::milvus::grpc::StringReply)}, - { 95, -1, sizeof(::milvus::grpc::BoolReply)}, - { 102, -1, sizeof(::milvus::grpc::TableRowCount)}, - { 109, -1, sizeof(::milvus::grpc::Command)}, - { 115, -1, sizeof(::milvus::grpc::Index)}, - { 122, -1, sizeof(::milvus::grpc::IndexParam)}, - { 130, -1, sizeof(::milvus::grpc::DeleteByRangeParam)}, + { 68, -1, sizeof(::milvus::grpc::TopKQueryResult)}, + { 78, -1, sizeof(::milvus::grpc::StringReply)}, + { 85, -1, sizeof(::milvus::grpc::BoolReply)}, + { 92, -1, sizeof(::milvus::grpc::TableRowCount)}, + { 99, -1, sizeof(::milvus::grpc::Command)}, + { 105, -1, sizeof(::milvus::grpc::Index)}, + { 112, -1, sizeof(::milvus::grpc::IndexParam)}, + { 120, -1, sizeof(::milvus::grpc::DeleteByRangeParam)}, }; static ::PROTOBUF_NAMESPACE_ID::Message const * const file_default_instances[] = { @@ -559,9 +507,7 @@ static ::PROTOBUF_NAMESPACE_ID::Message const * const file_default_instances[] = reinterpret_cast(&::milvus::grpc::_VectorIds_default_instance_), reinterpret_cast(&::milvus::grpc::_SearchParam_default_instance_), reinterpret_cast(&::milvus::grpc::_SearchInFilesParam_default_instance_), - reinterpret_cast(&::milvus::grpc::_QueryResult_default_instance_), reinterpret_cast(&::milvus::grpc::_TopKQueryResult_default_instance_), - reinterpret_cast(&::milvus::grpc::_TopKQueryResultList_default_instance_), reinterpret_cast(&::milvus::grpc::_StringReply_default_instance_), reinterpret_cast(&::milvus::grpc::_BoolReply_default_instance_), reinterpret_cast(&::milvus::grpc::_TableRowCount_default_instance_), @@ -592,62 +538,58 @@ const char descriptor_table_protodef_milvus_2eproto[] PROTOBUF_SECTION_VARIABLE( "c.Range\022\014\n\004topk\030\004 \001(\003\022\016\n\006nprobe\030\005 \001(\003\"[\n" "\022SearchInFilesParam\022\025\n\rfile_id_array\030\001 \003" "(\t\022.\n\014search_param\030\002 \001(\0132\030.milvus.grpc.S" - "earchParam\"+\n\013QueryResult\022\n\n\002id\030\001 \001(\003\022\020\n" - "\010distance\030\002 \001(\001\"H\n\017TopKQueryResult\0225\n\023qu" - "ery_result_arrays\030\001 \003(\0132\030.milvus.grpc.Qu" - "eryResult\"s\n\023TopKQueryResultList\022#\n\006stat" - "us\030\001 \001(\0132\023.milvus.grpc.Status\0227\n\021topk_qu" - "ery_result\030\002 \003(\0132\034.milvus.grpc.TopKQuery" - "Result\"H\n\013StringReply\022#\n\006status\030\001 \001(\0132\023." - "milvus.grpc.Status\022\024\n\014string_reply\030\002 \001(\t" - "\"D\n\tBoolReply\022#\n\006status\030\001 \001(\0132\023.milvus.g" - "rpc.Status\022\022\n\nbool_reply\030\002 \001(\010\"M\n\rTableR" - "owCount\022#\n\006status\030\001 \001(\0132\023.milvus.grpc.St" - "atus\022\027\n\017table_row_count\030\002 \001(\003\"\026\n\007Command" - "\022\013\n\003cmd\030\001 \001(\t\"*\n\005Index\022\022\n\nindex_type\030\001 \001" - "(\005\022\r\n\005nlist\030\002 \001(\005\"h\n\nIndexParam\022#\n\006statu" - "s\030\001 \001(\0132\023.milvus.grpc.Status\022\022\n\ntable_na" - "me\030\002 \001(\t\022!\n\005index\030\003 \001(\0132\022.milvus.grpc.In" - "dex\"K\n\022DeleteByRangeParam\022!\n\005range\030\001 \001(\013" - "2\022.milvus.grpc.Range\022\022\n\ntable_name\030\002 \001(\t" - "2\360\007\n\rMilvusService\022>\n\013CreateTable\022\030.milv" - "us.grpc.TableSchema\032\023.milvus.grpc.Status" - "\"\000\022<\n\010HasTable\022\026.milvus.grpc.TableName\032\026" - ".milvus.grpc.BoolReply\"\000\022:\n\tDropTable\022\026." - "milvus.grpc.TableName\032\023.milvus.grpc.Stat" - "us\"\000\022=\n\013CreateIndex\022\027.milvus.grpc.IndexP" - "aram\032\023.milvus.grpc.Status\"\000\022<\n\006Insert\022\030." - "milvus.grpc.InsertParam\032\026.milvus.grpc.Ve" - "ctorIds\"\000\022F\n\006Search\022\030.milvus.grpc.Search" - "Param\032 .milvus.grpc.TopKQueryResultList\"" - "\000\022T\n\rSearchInFiles\022\037.milvus.grpc.SearchI" - "nFilesParam\032 .milvus.grpc.TopKQueryResul" - "tList\"\000\022C\n\rDescribeTable\022\026.milvus.grpc.T" - "ableName\032\030.milvus.grpc.TableSchema\"\000\022B\n\n" - "CountTable\022\026.milvus.grpc.TableName\032\032.mil" - "vus.grpc.TableRowCount\"\000\022@\n\nShowTables\022\024" - ".milvus.grpc.Command\032\032.milvus.grpc.Table" - "NameList\"\000\0227\n\003Cmd\022\024.milvus.grpc.Command\032" - "\030.milvus.grpc.StringReply\"\000\022G\n\rDeleteByR" - "ange\022\037.milvus.grpc.DeleteByRangeParam\032\023." - "milvus.grpc.Status\"\000\022=\n\014PreloadTable\022\026.m" - "ilvus.grpc.TableName\032\023.milvus.grpc.Statu" - "s\"\000\022B\n\rDescribeIndex\022\026.milvus.grpc.Table" - "Name\032\027.milvus.grpc.IndexParam\"\000\022:\n\tDropI" - "ndex\022\026.milvus.grpc.TableName\032\023.milvus.gr" - "pc.Status\"\000b\006proto3" + "earchParam\"~\n\017TopKQueryResult\022#\n\006status\030" + "\001 \001(\0132\023.milvus.grpc.Status\022\n\n\002nq\030\002 \001(\003\022\014" + "\n\004topk\030\003 \001(\003\022\022\n\nids_binary\030\004 \001(\014\022\030\n\020dist" + "ances_binary\030\005 \001(\014\"H\n\013StringReply\022#\n\006sta" + "tus\030\001 \001(\0132\023.milvus.grpc.Status\022\024\n\014string" + "_reply\030\002 \001(\t\"D\n\tBoolReply\022#\n\006status\030\001 \001(" + "\0132\023.milvus.grpc.Status\022\022\n\nbool_reply\030\002 \001" + "(\010\"M\n\rTableRowCount\022#\n\006status\030\001 \001(\0132\023.mi" + "lvus.grpc.Status\022\027\n\017table_row_count\030\002 \001(" + "\003\"\026\n\007Command\022\013\n\003cmd\030\001 \001(\t\"*\n\005Index\022\022\n\nin" + "dex_type\030\001 \001(\005\022\r\n\005nlist\030\002 \001(\005\"h\n\nIndexPa" + "ram\022#\n\006status\030\001 \001(\0132\023.milvus.grpc.Status" + "\022\022\n\ntable_name\030\002 \001(\t\022!\n\005index\030\003 \001(\0132\022.mi" + "lvus.grpc.Index\"K\n\022DeleteByRangeParam\022!\n" + "\005range\030\001 \001(\0132\022.milvus.grpc.Range\022\022\n\ntabl" + "e_name\030\002 \001(\t2\350\007\n\rMilvusService\022>\n\013Create" + "Table\022\030.milvus.grpc.TableSchema\032\023.milvus" + ".grpc.Status\"\000\022<\n\010HasTable\022\026.milvus.grpc" + ".TableName\032\026.milvus.grpc.BoolReply\"\000\022:\n\t" + "DropTable\022\026.milvus.grpc.TableName\032\023.milv" + "us.grpc.Status\"\000\022=\n\013CreateIndex\022\027.milvus" + ".grpc.IndexParam\032\023.milvus.grpc.Status\"\000\022" + "<\n\006Insert\022\030.milvus.grpc.InsertParam\032\026.mi" + "lvus.grpc.VectorIds\"\000\022B\n\006Search\022\030.milvus" + ".grpc.SearchParam\032\034.milvus.grpc.TopKQuer" + "yResult\"\000\022P\n\rSearchInFiles\022\037.milvus.grpc" + ".SearchInFilesParam\032\034.milvus.grpc.TopKQu" + "eryResult\"\000\022C\n\rDescribeTable\022\026.milvus.gr" + "pc.TableName\032\030.milvus.grpc.TableSchema\"\000" + "\022B\n\nCountTable\022\026.milvus.grpc.TableName\032\032" + ".milvus.grpc.TableRowCount\"\000\022@\n\nShowTabl" + "es\022\024.milvus.grpc.Command\032\032.milvus.grpc.T" + "ableNameList\"\000\0227\n\003Cmd\022\024.milvus.grpc.Comm" + "and\032\030.milvus.grpc.StringReply\"\000\022G\n\rDelet" + "eByRange\022\037.milvus.grpc.DeleteByRangePara" + "m\032\023.milvus.grpc.Status\"\000\022=\n\014PreloadTable" + "\022\026.milvus.grpc.TableName\032\023.milvus.grpc.S" + "tatus\"\000\022B\n\rDescribeIndex\022\026.milvus.grpc.T" + "ableName\032\027.milvus.grpc.IndexParam\"\000\022:\n\tD" + "ropIndex\022\026.milvus.grpc.TableName\032\023.milvu" + "s.grpc.Status\"\000b\006proto3" ; static const ::PROTOBUF_NAMESPACE_ID::internal::DescriptorTable*const descriptor_table_milvus_2eproto_deps[1] = { &::descriptor_table_status_2eproto, }; -static ::PROTOBUF_NAMESPACE_ID::internal::SCCInfoBase*const descriptor_table_milvus_2eproto_sccs[19] = { +static ::PROTOBUF_NAMESPACE_ID::internal::SCCInfoBase*const descriptor_table_milvus_2eproto_sccs[17] = { &scc_info_BoolReply_milvus_2eproto.base, &scc_info_Command_milvus_2eproto.base, &scc_info_DeleteByRangeParam_milvus_2eproto.base, &scc_info_Index_milvus_2eproto.base, &scc_info_IndexParam_milvus_2eproto.base, &scc_info_InsertParam_milvus_2eproto.base, - &scc_info_QueryResult_milvus_2eproto.base, &scc_info_Range_milvus_2eproto.base, &scc_info_RowRecord_milvus_2eproto.base, &scc_info_SearchInFilesParam_milvus_2eproto.base, @@ -658,16 +600,15 @@ static ::PROTOBUF_NAMESPACE_ID::internal::SCCInfoBase*const descriptor_table_mil &scc_info_TableRowCount_milvus_2eproto.base, &scc_info_TableSchema_milvus_2eproto.base, &scc_info_TopKQueryResult_milvus_2eproto.base, - &scc_info_TopKQueryResultList_milvus_2eproto.base, &scc_info_VectorIds_milvus_2eproto.base, }; static ::PROTOBUF_NAMESPACE_ID::internal::once_flag descriptor_table_milvus_2eproto_once; static bool descriptor_table_milvus_2eproto_initialized = false; const ::PROTOBUF_NAMESPACE_ID::internal::DescriptorTable descriptor_table_milvus_2eproto = { - &descriptor_table_milvus_2eproto_initialized, descriptor_table_protodef_milvus_2eproto, "milvus.proto", 2539, - &descriptor_table_milvus_2eproto_once, descriptor_table_milvus_2eproto_sccs, descriptor_table_milvus_2eproto_deps, 19, 1, + &descriptor_table_milvus_2eproto_initialized, descriptor_table_protodef_milvus_2eproto, "milvus.proto", 2423, + &descriptor_table_milvus_2eproto_once, descriptor_table_milvus_2eproto_sccs, descriptor_table_milvus_2eproto_deps, 17, 1, schemas, file_default_instances, TableStruct_milvus_2eproto::offsets, - file_level_metadata_milvus_2eproto, 19, file_level_enum_descriptors_milvus_2eproto, file_level_service_descriptors_milvus_2eproto, + file_level_metadata_milvus_2eproto, 17, file_level_enum_descriptors_milvus_2eproto, file_level_service_descriptors_milvus_2eproto, }; // Force running AddDescriptors() at dynamic initialization time. @@ -3876,308 +3817,27 @@ void SearchInFilesParam::InternalSwap(SearchInFilesParam* other) { } -// =================================================================== - -void QueryResult::InitAsDefaultInstance() { -} -class QueryResult::_Internal { - public: -}; - -QueryResult::QueryResult() - : ::PROTOBUF_NAMESPACE_ID::Message(), _internal_metadata_(nullptr) { - SharedCtor(); - // @@protoc_insertion_point(constructor:milvus.grpc.QueryResult) -} -QueryResult::QueryResult(const QueryResult& from) - : ::PROTOBUF_NAMESPACE_ID::Message(), - _internal_metadata_(nullptr) { - _internal_metadata_.MergeFrom(from._internal_metadata_); - ::memcpy(&id_, &from.id_, - static_cast(reinterpret_cast(&distance_) - - reinterpret_cast(&id_)) + sizeof(distance_)); - // @@protoc_insertion_point(copy_constructor:milvus.grpc.QueryResult) -} - -void QueryResult::SharedCtor() { - ::memset(&id_, 0, static_cast( - reinterpret_cast(&distance_) - - reinterpret_cast(&id_)) + sizeof(distance_)); -} - -QueryResult::~QueryResult() { - // @@protoc_insertion_point(destructor:milvus.grpc.QueryResult) - SharedDtor(); -} - -void QueryResult::SharedDtor() { -} - -void QueryResult::SetCachedSize(int size) const { - _cached_size_.Set(size); -} -const QueryResult& QueryResult::default_instance() { - ::PROTOBUF_NAMESPACE_ID::internal::InitSCC(&::scc_info_QueryResult_milvus_2eproto.base); - return *internal_default_instance(); -} - - -void QueryResult::Clear() { -// @@protoc_insertion_point(message_clear_start:milvus.grpc.QueryResult) - ::PROTOBUF_NAMESPACE_ID::uint32 cached_has_bits = 0; - // Prevent compiler warnings about cached_has_bits being unused - (void) cached_has_bits; - - ::memset(&id_, 0, static_cast( - reinterpret_cast(&distance_) - - reinterpret_cast(&id_)) + sizeof(distance_)); - _internal_metadata_.Clear(); -} - -#if GOOGLE_PROTOBUF_ENABLE_EXPERIMENTAL_PARSER -const char* QueryResult::_InternalParse(const char* ptr, ::PROTOBUF_NAMESPACE_ID::internal::ParseContext* ctx) { -#define CHK_(x) if (PROTOBUF_PREDICT_FALSE(!(x))) goto failure - while (!ctx->Done(&ptr)) { - ::PROTOBUF_NAMESPACE_ID::uint32 tag; - ptr = ::PROTOBUF_NAMESPACE_ID::internal::ReadTag(ptr, &tag); - CHK_(ptr); - switch (tag >> 3) { - // int64 id = 1; - case 1: - if (PROTOBUF_PREDICT_TRUE(static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 8)) { - id_ = ::PROTOBUF_NAMESPACE_ID::internal::ReadVarint(&ptr); - CHK_(ptr); - } else goto handle_unusual; - continue; - // double distance = 2; - case 2: - if (PROTOBUF_PREDICT_TRUE(static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 17)) { - distance_ = ::PROTOBUF_NAMESPACE_ID::internal::UnalignedLoad(ptr); - ptr += sizeof(double); - } else goto handle_unusual; - continue; - default: { - handle_unusual: - if ((tag & 7) == 4 || tag == 0) { - ctx->SetLastTag(tag); - goto success; - } - ptr = UnknownFieldParse(tag, &_internal_metadata_, ptr, ctx); - CHK_(ptr != nullptr); - continue; - } - } // switch - } // while -success: - return ptr; -failure: - ptr = nullptr; - goto success; -#undef CHK_ -} -#else // GOOGLE_PROTOBUF_ENABLE_EXPERIMENTAL_PARSER -bool QueryResult::MergePartialFromCodedStream( - ::PROTOBUF_NAMESPACE_ID::io::CodedInputStream* input) { -#define DO_(EXPRESSION) if (!PROTOBUF_PREDICT_TRUE(EXPRESSION)) goto failure - ::PROTOBUF_NAMESPACE_ID::uint32 tag; - // @@protoc_insertion_point(parse_start:milvus.grpc.QueryResult) - for (;;) { - ::std::pair<::PROTOBUF_NAMESPACE_ID::uint32, bool> p = input->ReadTagWithCutoffNoLastTag(127u); - tag = p.first; - if (!p.second) goto handle_unusual; - switch (::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::GetTagFieldNumber(tag)) { - // int64 id = 1; - case 1: { - if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (8 & 0xFF)) { - - DO_((::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadPrimitive< - ::PROTOBUF_NAMESPACE_ID::int64, ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::TYPE_INT64>( - input, &id_))); - } else { - goto handle_unusual; - } - break; - } - - // double distance = 2; - case 2: { - if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (17 & 0xFF)) { - - DO_((::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadPrimitive< - double, ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::TYPE_DOUBLE>( - input, &distance_))); - } else { - goto handle_unusual; - } - break; - } - - default: { - handle_unusual: - if (tag == 0) { - goto success; - } - DO_(::PROTOBUF_NAMESPACE_ID::internal::WireFormat::SkipField( - input, tag, _internal_metadata_.mutable_unknown_fields())); - break; - } - } - } -success: - // @@protoc_insertion_point(parse_success:milvus.grpc.QueryResult) - return true; -failure: - // @@protoc_insertion_point(parse_failure:milvus.grpc.QueryResult) - return false; -#undef DO_ -} -#endif // GOOGLE_PROTOBUF_ENABLE_EXPERIMENTAL_PARSER - -void QueryResult::SerializeWithCachedSizes( - ::PROTOBUF_NAMESPACE_ID::io::CodedOutputStream* output) const { - // @@protoc_insertion_point(serialize_start:milvus.grpc.QueryResult) - ::PROTOBUF_NAMESPACE_ID::uint32 cached_has_bits = 0; - (void) cached_has_bits; - - // int64 id = 1; - if (this->id() != 0) { - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteInt64(1, this->id(), output); - } - - // double distance = 2; - if (!(this->distance() <= 0 && this->distance() >= 0)) { - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteDouble(2, this->distance(), output); - } - - if (_internal_metadata_.have_unknown_fields()) { - ::PROTOBUF_NAMESPACE_ID::internal::WireFormat::SerializeUnknownFields( - _internal_metadata_.unknown_fields(), output); - } - // @@protoc_insertion_point(serialize_end:milvus.grpc.QueryResult) -} - -::PROTOBUF_NAMESPACE_ID::uint8* QueryResult::InternalSerializeWithCachedSizesToArray( - ::PROTOBUF_NAMESPACE_ID::uint8* target) const { - // @@protoc_insertion_point(serialize_to_array_start:milvus.grpc.QueryResult) - ::PROTOBUF_NAMESPACE_ID::uint32 cached_has_bits = 0; - (void) cached_has_bits; - - // int64 id = 1; - if (this->id() != 0) { - target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteInt64ToArray(1, this->id(), target); - } - - // double distance = 2; - if (!(this->distance() <= 0 && this->distance() >= 0)) { - target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteDoubleToArray(2, this->distance(), target); - } - - if (_internal_metadata_.have_unknown_fields()) { - target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormat::SerializeUnknownFieldsToArray( - _internal_metadata_.unknown_fields(), target); - } - // @@protoc_insertion_point(serialize_to_array_end:milvus.grpc.QueryResult) - return target; -} - -size_t QueryResult::ByteSizeLong() const { -// @@protoc_insertion_point(message_byte_size_start:milvus.grpc.QueryResult) - size_t total_size = 0; - - if (_internal_metadata_.have_unknown_fields()) { - total_size += - ::PROTOBUF_NAMESPACE_ID::internal::WireFormat::ComputeUnknownFieldsSize( - _internal_metadata_.unknown_fields()); - } - ::PROTOBUF_NAMESPACE_ID::uint32 cached_has_bits = 0; - // Prevent compiler warnings about cached_has_bits being unused - (void) cached_has_bits; - - // int64 id = 1; - if (this->id() != 0) { - total_size += 1 + - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::Int64Size( - this->id()); - } - - // double distance = 2; - if (!(this->distance() <= 0 && this->distance() >= 0)) { - total_size += 1 + 8; - } - - int cached_size = ::PROTOBUF_NAMESPACE_ID::internal::ToCachedSize(total_size); - SetCachedSize(cached_size); - return total_size; -} - -void QueryResult::MergeFrom(const ::PROTOBUF_NAMESPACE_ID::Message& from) { -// @@protoc_insertion_point(generalized_merge_from_start:milvus.grpc.QueryResult) - GOOGLE_DCHECK_NE(&from, this); - const QueryResult* source = - ::PROTOBUF_NAMESPACE_ID::DynamicCastToGenerated( - &from); - if (source == nullptr) { - // @@protoc_insertion_point(generalized_merge_from_cast_fail:milvus.grpc.QueryResult) - ::PROTOBUF_NAMESPACE_ID::internal::ReflectionOps::Merge(from, this); - } else { - // @@protoc_insertion_point(generalized_merge_from_cast_success:milvus.grpc.QueryResult) - MergeFrom(*source); - } -} - -void QueryResult::MergeFrom(const QueryResult& from) { -// @@protoc_insertion_point(class_specific_merge_from_start:milvus.grpc.QueryResult) - GOOGLE_DCHECK_NE(&from, this); - _internal_metadata_.MergeFrom(from._internal_metadata_); - ::PROTOBUF_NAMESPACE_ID::uint32 cached_has_bits = 0; - (void) cached_has_bits; - - if (from.id() != 0) { - set_id(from.id()); - } - if (!(from.distance() <= 0 && from.distance() >= 0)) { - set_distance(from.distance()); - } -} - -void QueryResult::CopyFrom(const ::PROTOBUF_NAMESPACE_ID::Message& from) { -// @@protoc_insertion_point(generalized_copy_from_start:milvus.grpc.QueryResult) - if (&from == this) return; - Clear(); - MergeFrom(from); -} - -void QueryResult::CopyFrom(const QueryResult& from) { -// @@protoc_insertion_point(class_specific_copy_from_start:milvus.grpc.QueryResult) - if (&from == this) return; - Clear(); - MergeFrom(from); -} - -bool QueryResult::IsInitialized() const { - return true; -} - -void QueryResult::InternalSwap(QueryResult* other) { - using std::swap; - _internal_metadata_.Swap(&other->_internal_metadata_); - swap(id_, other->id_); - swap(distance_, other->distance_); -} - -::PROTOBUF_NAMESPACE_ID::Metadata QueryResult::GetMetadata() const { - return GetMetadataStatic(); -} - - // =================================================================== void TopKQueryResult::InitAsDefaultInstance() { + ::milvus::grpc::_TopKQueryResult_default_instance_._instance.get_mutable()->status_ = const_cast< ::milvus::grpc::Status*>( + ::milvus::grpc::Status::internal_default_instance()); } class TopKQueryResult::_Internal { public: + static const ::milvus::grpc::Status& status(const TopKQueryResult* msg); }; +const ::milvus::grpc::Status& +TopKQueryResult::_Internal::status(const TopKQueryResult* msg) { + return *msg->status_; +} +void TopKQueryResult::clear_status() { + if (GetArenaNoVirtual() == nullptr && status_ != nullptr) { + delete status_; + } + status_ = nullptr; +} TopKQueryResult::TopKQueryResult() : ::PROTOBUF_NAMESPACE_ID::Message(), _internal_metadata_(nullptr) { SharedCtor(); @@ -4185,14 +3845,34 @@ TopKQueryResult::TopKQueryResult() } TopKQueryResult::TopKQueryResult(const TopKQueryResult& from) : ::PROTOBUF_NAMESPACE_ID::Message(), - _internal_metadata_(nullptr), - query_result_arrays_(from.query_result_arrays_) { + _internal_metadata_(nullptr) { _internal_metadata_.MergeFrom(from._internal_metadata_); + ids_binary_.UnsafeSetDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); + if (!from.ids_binary().empty()) { + ids_binary_.AssignWithDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), from.ids_binary_); + } + distances_binary_.UnsafeSetDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); + if (!from.distances_binary().empty()) { + distances_binary_.AssignWithDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), from.distances_binary_); + } + if (from.has_status()) { + status_ = new ::milvus::grpc::Status(*from.status_); + } else { + status_ = nullptr; + } + ::memcpy(&nq_, &from.nq_, + static_cast(reinterpret_cast(&topk_) - + reinterpret_cast(&nq_)) + sizeof(topk_)); // @@protoc_insertion_point(copy_constructor:milvus.grpc.TopKQueryResult) } void TopKQueryResult::SharedCtor() { ::PROTOBUF_NAMESPACE_ID::internal::InitSCC(&scc_info_TopKQueryResult_milvus_2eproto.base); + ids_binary_.UnsafeSetDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); + distances_binary_.UnsafeSetDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); + ::memset(&status_, 0, static_cast( + reinterpret_cast(&topk_) - + reinterpret_cast(&status_)) + sizeof(topk_)); } TopKQueryResult::~TopKQueryResult() { @@ -4201,6 +3881,9 @@ TopKQueryResult::~TopKQueryResult() { } void TopKQueryResult::SharedDtor() { + ids_binary_.DestroyNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); + distances_binary_.DestroyNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); + if (this != internal_default_instance()) delete status_; } void TopKQueryResult::SetCachedSize(int size) const { @@ -4218,7 +3901,15 @@ void TopKQueryResult::Clear() { // Prevent compiler warnings about cached_has_bits being unused (void) cached_has_bits; - query_result_arrays_.Clear(); + ids_binary_.ClearToEmptyNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); + distances_binary_.ClearToEmptyNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); + if (GetArenaNoVirtual() == nullptr && status_ != nullptr) { + delete status_; + } + status_ = nullptr; + ::memset(&nq_, 0, static_cast( + reinterpret_cast(&topk_) - + reinterpret_cast(&nq_)) + sizeof(topk_)); _internal_metadata_.Clear(); } @@ -4230,16 +3921,39 @@ const char* TopKQueryResult::_InternalParse(const char* ptr, ::PROTOBUF_NAMESPAC ptr = ::PROTOBUF_NAMESPACE_ID::internal::ReadTag(ptr, &tag); CHK_(ptr); switch (tag >> 3) { - // repeated .milvus.grpc.QueryResult query_result_arrays = 1; + // .milvus.grpc.Status status = 1; case 1: if (PROTOBUF_PREDICT_TRUE(static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 10)) { - ptr -= 1; - do { - ptr += 1; - ptr = ctx->ParseMessage(add_query_result_arrays(), ptr); - CHK_(ptr); - if (!ctx->DataAvailable(ptr)) break; - } while (::PROTOBUF_NAMESPACE_ID::internal::UnalignedLoad<::PROTOBUF_NAMESPACE_ID::uint8>(ptr) == 10); + ptr = ctx->ParseMessage(mutable_status(), ptr); + CHK_(ptr); + } else goto handle_unusual; + continue; + // int64 nq = 2; + case 2: + if (PROTOBUF_PREDICT_TRUE(static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 16)) { + nq_ = ::PROTOBUF_NAMESPACE_ID::internal::ReadVarint(&ptr); + CHK_(ptr); + } else goto handle_unusual; + continue; + // int64 topk = 3; + case 3: + if (PROTOBUF_PREDICT_TRUE(static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 24)) { + topk_ = ::PROTOBUF_NAMESPACE_ID::internal::ReadVarint(&ptr); + CHK_(ptr); + } else goto handle_unusual; + continue; + // bytes ids_binary = 4; + case 4: + if (PROTOBUF_PREDICT_TRUE(static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 34)) { + ptr = ::PROTOBUF_NAMESPACE_ID::internal::InlineGreedyStringParser(mutable_ids_binary(), ptr, ctx); + CHK_(ptr); + } else goto handle_unusual; + continue; + // bytes distances_binary = 5; + case 5: + if (PROTOBUF_PREDICT_TRUE(static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 42)) { + ptr = ::PROTOBUF_NAMESPACE_ID::internal::InlineGreedyStringParser(mutable_distances_binary(), ptr, ctx); + CHK_(ptr); } else goto handle_unusual; continue; default: { @@ -4272,11 +3986,59 @@ bool TopKQueryResult::MergePartialFromCodedStream( tag = p.first; if (!p.second) goto handle_unusual; switch (::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::GetTagFieldNumber(tag)) { - // repeated .milvus.grpc.QueryResult query_result_arrays = 1; + // .milvus.grpc.Status status = 1; case 1: { if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (10 & 0xFF)) { DO_(::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadMessage( - input, add_query_result_arrays())); + input, mutable_status())); + } else { + goto handle_unusual; + } + break; + } + + // int64 nq = 2; + case 2: { + if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (16 & 0xFF)) { + + DO_((::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadPrimitive< + ::PROTOBUF_NAMESPACE_ID::int64, ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::TYPE_INT64>( + input, &nq_))); + } else { + goto handle_unusual; + } + break; + } + + // int64 topk = 3; + case 3: { + if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (24 & 0xFF)) { + + DO_((::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadPrimitive< + ::PROTOBUF_NAMESPACE_ID::int64, ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::TYPE_INT64>( + input, &topk_))); + } else { + goto handle_unusual; + } + break; + } + + // bytes ids_binary = 4; + case 4: { + if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (34 & 0xFF)) { + DO_(::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadBytes( + input, this->mutable_ids_binary())); + } else { + goto handle_unusual; + } + break; + } + + // bytes distances_binary = 5; + case 5: { + if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (42 & 0xFF)) { + DO_(::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadBytes( + input, this->mutable_distances_binary())); } else { goto handle_unusual; } @@ -4310,13 +4072,32 @@ void TopKQueryResult::SerializeWithCachedSizes( ::PROTOBUF_NAMESPACE_ID::uint32 cached_has_bits = 0; (void) cached_has_bits; - // repeated .milvus.grpc.QueryResult query_result_arrays = 1; - for (unsigned int i = 0, - n = static_cast(this->query_result_arrays_size()); i < n; i++) { + // .milvus.grpc.Status status = 1; + if (this->has_status()) { ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteMessageMaybeToArray( - 1, - this->query_result_arrays(static_cast(i)), - output); + 1, _Internal::status(this), output); + } + + // int64 nq = 2; + if (this->nq() != 0) { + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteInt64(2, this->nq(), output); + } + + // int64 topk = 3; + if (this->topk() != 0) { + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteInt64(3, this->topk(), output); + } + + // bytes ids_binary = 4; + if (this->ids_binary().size() > 0) { + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteBytesMaybeAliased( + 4, this->ids_binary(), output); + } + + // bytes distances_binary = 5; + if (this->distances_binary().size() > 0) { + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteBytesMaybeAliased( + 5, this->distances_binary(), output); } if (_internal_metadata_.have_unknown_fields()) { @@ -4332,12 +4113,35 @@ void TopKQueryResult::SerializeWithCachedSizes( ::PROTOBUF_NAMESPACE_ID::uint32 cached_has_bits = 0; (void) cached_has_bits; - // repeated .milvus.grpc.QueryResult query_result_arrays = 1; - for (unsigned int i = 0, - n = static_cast(this->query_result_arrays_size()); i < n; i++) { + // .milvus.grpc.Status status = 1; + if (this->has_status()) { target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite:: InternalWriteMessageToArray( - 1, this->query_result_arrays(static_cast(i)), target); + 1, _Internal::status(this), target); + } + + // int64 nq = 2; + if (this->nq() != 0) { + target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteInt64ToArray(2, this->nq(), target); + } + + // int64 topk = 3; + if (this->topk() != 0) { + target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteInt64ToArray(3, this->topk(), target); + } + + // bytes ids_binary = 4; + if (this->ids_binary().size() > 0) { + target = + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteBytesToArray( + 4, this->ids_binary(), target); + } + + // bytes distances_binary = 5; + if (this->distances_binary().size() > 0) { + target = + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteBytesToArray( + 5, this->distances_binary(), target); } if (_internal_metadata_.have_unknown_fields()) { @@ -4361,15 +4165,39 @@ size_t TopKQueryResult::ByteSizeLong() const { // Prevent compiler warnings about cached_has_bits being unused (void) cached_has_bits; - // repeated .milvus.grpc.QueryResult query_result_arrays = 1; - { - unsigned int count = static_cast(this->query_result_arrays_size()); - total_size += 1UL * count; - for (unsigned int i = 0; i < count; i++) { - total_size += - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::MessageSize( - this->query_result_arrays(static_cast(i))); - } + // bytes ids_binary = 4; + if (this->ids_binary().size() > 0) { + total_size += 1 + + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::BytesSize( + this->ids_binary()); + } + + // bytes distances_binary = 5; + if (this->distances_binary().size() > 0) { + total_size += 1 + + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::BytesSize( + this->distances_binary()); + } + + // .milvus.grpc.Status status = 1; + if (this->has_status()) { + total_size += 1 + + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::MessageSize( + *status_); + } + + // int64 nq = 2; + if (this->nq() != 0) { + total_size += 1 + + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::Int64Size( + this->nq()); + } + + // int64 topk = 3; + if (this->topk() != 0) { + total_size += 1 + + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::Int64Size( + this->topk()); } int cached_size = ::PROTOBUF_NAMESPACE_ID::internal::ToCachedSize(total_size); @@ -4399,7 +4227,23 @@ void TopKQueryResult::MergeFrom(const TopKQueryResult& from) { ::PROTOBUF_NAMESPACE_ID::uint32 cached_has_bits = 0; (void) cached_has_bits; - query_result_arrays_.MergeFrom(from.query_result_arrays_); + if (from.ids_binary().size() > 0) { + + ids_binary_.AssignWithDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), from.ids_binary_); + } + if (from.distances_binary().size() > 0) { + + distances_binary_.AssignWithDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), from.distances_binary_); + } + if (from.has_status()) { + mutable_status()->::milvus::grpc::Status::MergeFrom(from.status()); + } + if (from.nq() != 0) { + set_nq(from.nq()); + } + if (from.topk() != 0) { + set_topk(from.topk()); + } } void TopKQueryResult::CopyFrom(const ::PROTOBUF_NAMESPACE_ID::Message& from) { @@ -4423,7 +4267,13 @@ bool TopKQueryResult::IsInitialized() const { void TopKQueryResult::InternalSwap(TopKQueryResult* other) { using std::swap; _internal_metadata_.Swap(&other->_internal_metadata_); - CastToBase(&query_result_arrays_)->InternalSwap(CastToBase(&other->query_result_arrays_)); + ids_binary_.Swap(&other->ids_binary_, &::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); + distances_binary_.Swap(&other->distances_binary_, &::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), + GetArenaNoVirtual()); + swap(status_, other->status_); + swap(nq_, other->nq_); + swap(topk_, other->topk_); } ::PROTOBUF_NAMESPACE_ID::Metadata TopKQueryResult::GetMetadata() const { @@ -4431,333 +4281,6 @@ void TopKQueryResult::InternalSwap(TopKQueryResult* other) { } -// =================================================================== - -void TopKQueryResultList::InitAsDefaultInstance() { - ::milvus::grpc::_TopKQueryResultList_default_instance_._instance.get_mutable()->status_ = const_cast< ::milvus::grpc::Status*>( - ::milvus::grpc::Status::internal_default_instance()); -} -class TopKQueryResultList::_Internal { - public: - static const ::milvus::grpc::Status& status(const TopKQueryResultList* msg); -}; - -const ::milvus::grpc::Status& -TopKQueryResultList::_Internal::status(const TopKQueryResultList* msg) { - return *msg->status_; -} -void TopKQueryResultList::clear_status() { - if (GetArenaNoVirtual() == nullptr && status_ != nullptr) { - delete status_; - } - status_ = nullptr; -} -TopKQueryResultList::TopKQueryResultList() - : ::PROTOBUF_NAMESPACE_ID::Message(), _internal_metadata_(nullptr) { - SharedCtor(); - // @@protoc_insertion_point(constructor:milvus.grpc.TopKQueryResultList) -} -TopKQueryResultList::TopKQueryResultList(const TopKQueryResultList& from) - : ::PROTOBUF_NAMESPACE_ID::Message(), - _internal_metadata_(nullptr), - topk_query_result_(from.topk_query_result_) { - _internal_metadata_.MergeFrom(from._internal_metadata_); - if (from.has_status()) { - status_ = new ::milvus::grpc::Status(*from.status_); - } else { - status_ = nullptr; - } - // @@protoc_insertion_point(copy_constructor:milvus.grpc.TopKQueryResultList) -} - -void TopKQueryResultList::SharedCtor() { - ::PROTOBUF_NAMESPACE_ID::internal::InitSCC(&scc_info_TopKQueryResultList_milvus_2eproto.base); - status_ = nullptr; -} - -TopKQueryResultList::~TopKQueryResultList() { - // @@protoc_insertion_point(destructor:milvus.grpc.TopKQueryResultList) - SharedDtor(); -} - -void TopKQueryResultList::SharedDtor() { - if (this != internal_default_instance()) delete status_; -} - -void TopKQueryResultList::SetCachedSize(int size) const { - _cached_size_.Set(size); -} -const TopKQueryResultList& TopKQueryResultList::default_instance() { - ::PROTOBUF_NAMESPACE_ID::internal::InitSCC(&::scc_info_TopKQueryResultList_milvus_2eproto.base); - return *internal_default_instance(); -} - - -void TopKQueryResultList::Clear() { -// @@protoc_insertion_point(message_clear_start:milvus.grpc.TopKQueryResultList) - ::PROTOBUF_NAMESPACE_ID::uint32 cached_has_bits = 0; - // Prevent compiler warnings about cached_has_bits being unused - (void) cached_has_bits; - - topk_query_result_.Clear(); - if (GetArenaNoVirtual() == nullptr && status_ != nullptr) { - delete status_; - } - status_ = nullptr; - _internal_metadata_.Clear(); -} - -#if GOOGLE_PROTOBUF_ENABLE_EXPERIMENTAL_PARSER -const char* TopKQueryResultList::_InternalParse(const char* ptr, ::PROTOBUF_NAMESPACE_ID::internal::ParseContext* ctx) { -#define CHK_(x) if (PROTOBUF_PREDICT_FALSE(!(x))) goto failure - while (!ctx->Done(&ptr)) { - ::PROTOBUF_NAMESPACE_ID::uint32 tag; - ptr = ::PROTOBUF_NAMESPACE_ID::internal::ReadTag(ptr, &tag); - CHK_(ptr); - switch (tag >> 3) { - // .milvus.grpc.Status status = 1; - case 1: - if (PROTOBUF_PREDICT_TRUE(static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 10)) { - ptr = ctx->ParseMessage(mutable_status(), ptr); - CHK_(ptr); - } else goto handle_unusual; - continue; - // repeated .milvus.grpc.TopKQueryResult topk_query_result = 2; - case 2: - if (PROTOBUF_PREDICT_TRUE(static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 18)) { - ptr -= 1; - do { - ptr += 1; - ptr = ctx->ParseMessage(add_topk_query_result(), ptr); - CHK_(ptr); - if (!ctx->DataAvailable(ptr)) break; - } while (::PROTOBUF_NAMESPACE_ID::internal::UnalignedLoad<::PROTOBUF_NAMESPACE_ID::uint8>(ptr) == 18); - } else goto handle_unusual; - continue; - default: { - handle_unusual: - if ((tag & 7) == 4 || tag == 0) { - ctx->SetLastTag(tag); - goto success; - } - ptr = UnknownFieldParse(tag, &_internal_metadata_, ptr, ctx); - CHK_(ptr != nullptr); - continue; - } - } // switch - } // while -success: - return ptr; -failure: - ptr = nullptr; - goto success; -#undef CHK_ -} -#else // GOOGLE_PROTOBUF_ENABLE_EXPERIMENTAL_PARSER -bool TopKQueryResultList::MergePartialFromCodedStream( - ::PROTOBUF_NAMESPACE_ID::io::CodedInputStream* input) { -#define DO_(EXPRESSION) if (!PROTOBUF_PREDICT_TRUE(EXPRESSION)) goto failure - ::PROTOBUF_NAMESPACE_ID::uint32 tag; - // @@protoc_insertion_point(parse_start:milvus.grpc.TopKQueryResultList) - for (;;) { - ::std::pair<::PROTOBUF_NAMESPACE_ID::uint32, bool> p = input->ReadTagWithCutoffNoLastTag(127u); - tag = p.first; - if (!p.second) goto handle_unusual; - switch (::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::GetTagFieldNumber(tag)) { - // .milvus.grpc.Status status = 1; - case 1: { - if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (10 & 0xFF)) { - DO_(::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadMessage( - input, mutable_status())); - } else { - goto handle_unusual; - } - break; - } - - // repeated .milvus.grpc.TopKQueryResult topk_query_result = 2; - case 2: { - if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (18 & 0xFF)) { - DO_(::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadMessage( - input, add_topk_query_result())); - } else { - goto handle_unusual; - } - break; - } - - default: { - handle_unusual: - if (tag == 0) { - goto success; - } - DO_(::PROTOBUF_NAMESPACE_ID::internal::WireFormat::SkipField( - input, tag, _internal_metadata_.mutable_unknown_fields())); - break; - } - } - } -success: - // @@protoc_insertion_point(parse_success:milvus.grpc.TopKQueryResultList) - return true; -failure: - // @@protoc_insertion_point(parse_failure:milvus.grpc.TopKQueryResultList) - return false; -#undef DO_ -} -#endif // GOOGLE_PROTOBUF_ENABLE_EXPERIMENTAL_PARSER - -void TopKQueryResultList::SerializeWithCachedSizes( - ::PROTOBUF_NAMESPACE_ID::io::CodedOutputStream* output) const { - // @@protoc_insertion_point(serialize_start:milvus.grpc.TopKQueryResultList) - ::PROTOBUF_NAMESPACE_ID::uint32 cached_has_bits = 0; - (void) cached_has_bits; - - // .milvus.grpc.Status status = 1; - if (this->has_status()) { - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteMessageMaybeToArray( - 1, _Internal::status(this), output); - } - - // repeated .milvus.grpc.TopKQueryResult topk_query_result = 2; - for (unsigned int i = 0, - n = static_cast(this->topk_query_result_size()); i < n; i++) { - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteMessageMaybeToArray( - 2, - this->topk_query_result(static_cast(i)), - output); - } - - if (_internal_metadata_.have_unknown_fields()) { - ::PROTOBUF_NAMESPACE_ID::internal::WireFormat::SerializeUnknownFields( - _internal_metadata_.unknown_fields(), output); - } - // @@protoc_insertion_point(serialize_end:milvus.grpc.TopKQueryResultList) -} - -::PROTOBUF_NAMESPACE_ID::uint8* TopKQueryResultList::InternalSerializeWithCachedSizesToArray( - ::PROTOBUF_NAMESPACE_ID::uint8* target) const { - // @@protoc_insertion_point(serialize_to_array_start:milvus.grpc.TopKQueryResultList) - ::PROTOBUF_NAMESPACE_ID::uint32 cached_has_bits = 0; - (void) cached_has_bits; - - // .milvus.grpc.Status status = 1; - if (this->has_status()) { - target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite:: - InternalWriteMessageToArray( - 1, _Internal::status(this), target); - } - - // repeated .milvus.grpc.TopKQueryResult topk_query_result = 2; - for (unsigned int i = 0, - n = static_cast(this->topk_query_result_size()); i < n; i++) { - target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite:: - InternalWriteMessageToArray( - 2, this->topk_query_result(static_cast(i)), target); - } - - if (_internal_metadata_.have_unknown_fields()) { - target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormat::SerializeUnknownFieldsToArray( - _internal_metadata_.unknown_fields(), target); - } - // @@protoc_insertion_point(serialize_to_array_end:milvus.grpc.TopKQueryResultList) - return target; -} - -size_t TopKQueryResultList::ByteSizeLong() const { -// @@protoc_insertion_point(message_byte_size_start:milvus.grpc.TopKQueryResultList) - size_t total_size = 0; - - if (_internal_metadata_.have_unknown_fields()) { - total_size += - ::PROTOBUF_NAMESPACE_ID::internal::WireFormat::ComputeUnknownFieldsSize( - _internal_metadata_.unknown_fields()); - } - ::PROTOBUF_NAMESPACE_ID::uint32 cached_has_bits = 0; - // Prevent compiler warnings about cached_has_bits being unused - (void) cached_has_bits; - - // repeated .milvus.grpc.TopKQueryResult topk_query_result = 2; - { - unsigned int count = static_cast(this->topk_query_result_size()); - total_size += 1UL * count; - for (unsigned int i = 0; i < count; i++) { - total_size += - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::MessageSize( - this->topk_query_result(static_cast(i))); - } - } - - // .milvus.grpc.Status status = 1; - if (this->has_status()) { - total_size += 1 + - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::MessageSize( - *status_); - } - - int cached_size = ::PROTOBUF_NAMESPACE_ID::internal::ToCachedSize(total_size); - SetCachedSize(cached_size); - return total_size; -} - -void TopKQueryResultList::MergeFrom(const ::PROTOBUF_NAMESPACE_ID::Message& from) { -// @@protoc_insertion_point(generalized_merge_from_start:milvus.grpc.TopKQueryResultList) - GOOGLE_DCHECK_NE(&from, this); - const TopKQueryResultList* source = - ::PROTOBUF_NAMESPACE_ID::DynamicCastToGenerated( - &from); - if (source == nullptr) { - // @@protoc_insertion_point(generalized_merge_from_cast_fail:milvus.grpc.TopKQueryResultList) - ::PROTOBUF_NAMESPACE_ID::internal::ReflectionOps::Merge(from, this); - } else { - // @@protoc_insertion_point(generalized_merge_from_cast_success:milvus.grpc.TopKQueryResultList) - MergeFrom(*source); - } -} - -void TopKQueryResultList::MergeFrom(const TopKQueryResultList& from) { -// @@protoc_insertion_point(class_specific_merge_from_start:milvus.grpc.TopKQueryResultList) - GOOGLE_DCHECK_NE(&from, this); - _internal_metadata_.MergeFrom(from._internal_metadata_); - ::PROTOBUF_NAMESPACE_ID::uint32 cached_has_bits = 0; - (void) cached_has_bits; - - topk_query_result_.MergeFrom(from.topk_query_result_); - if (from.has_status()) { - mutable_status()->::milvus::grpc::Status::MergeFrom(from.status()); - } -} - -void TopKQueryResultList::CopyFrom(const ::PROTOBUF_NAMESPACE_ID::Message& from) { -// @@protoc_insertion_point(generalized_copy_from_start:milvus.grpc.TopKQueryResultList) - if (&from == this) return; - Clear(); - MergeFrom(from); -} - -void TopKQueryResultList::CopyFrom(const TopKQueryResultList& from) { -// @@protoc_insertion_point(class_specific_copy_from_start:milvus.grpc.TopKQueryResultList) - if (&from == this) return; - Clear(); - MergeFrom(from); -} - -bool TopKQueryResultList::IsInitialized() const { - return true; -} - -void TopKQueryResultList::InternalSwap(TopKQueryResultList* other) { - using std::swap; - _internal_metadata_.Swap(&other->_internal_metadata_); - CastToBase(&topk_query_result_)->InternalSwap(CastToBase(&other->topk_query_result_)); - swap(status_, other->status_); -} - -::PROTOBUF_NAMESPACE_ID::Metadata TopKQueryResultList::GetMetadata() const { - return GetMetadataStatic(); -} - - // =================================================================== void StringReply::InitAsDefaultInstance() { @@ -7046,15 +6569,9 @@ template<> PROTOBUF_NOINLINE ::milvus::grpc::SearchParam* Arena::CreateMaybeMess template<> PROTOBUF_NOINLINE ::milvus::grpc::SearchInFilesParam* Arena::CreateMaybeMessage< ::milvus::grpc::SearchInFilesParam >(Arena* arena) { return Arena::CreateInternal< ::milvus::grpc::SearchInFilesParam >(arena); } -template<> PROTOBUF_NOINLINE ::milvus::grpc::QueryResult* Arena::CreateMaybeMessage< ::milvus::grpc::QueryResult >(Arena* arena) { - return Arena::CreateInternal< ::milvus::grpc::QueryResult >(arena); -} template<> PROTOBUF_NOINLINE ::milvus::grpc::TopKQueryResult* Arena::CreateMaybeMessage< ::milvus::grpc::TopKQueryResult >(Arena* arena) { return Arena::CreateInternal< ::milvus::grpc::TopKQueryResult >(arena); } -template<> PROTOBUF_NOINLINE ::milvus::grpc::TopKQueryResultList* Arena::CreateMaybeMessage< ::milvus::grpc::TopKQueryResultList >(Arena* arena) { - return Arena::CreateInternal< ::milvus::grpc::TopKQueryResultList >(arena); -} template<> PROTOBUF_NOINLINE ::milvus::grpc::StringReply* Arena::CreateMaybeMessage< ::milvus::grpc::StringReply >(Arena* arena) { return Arena::CreateInternal< ::milvus::grpc::StringReply >(arena); } diff --git a/core/src/grpc/gen-milvus/milvus.pb.h b/core/src/grpc/gen-milvus/milvus.pb.h index 5ac3fda023..9db159dd89 100644 --- a/core/src/grpc/gen-milvus/milvus.pb.h +++ b/core/src/grpc/gen-milvus/milvus.pb.h @@ -48,7 +48,7 @@ struct TableStruct_milvus_2eproto { PROTOBUF_SECTION_VARIABLE(protodesc_cold); static const ::PROTOBUF_NAMESPACE_ID::internal::AuxillaryParseTableField aux[] PROTOBUF_SECTION_VARIABLE(protodesc_cold); - static const ::PROTOBUF_NAMESPACE_ID::internal::ParseTable schema[19] + static const ::PROTOBUF_NAMESPACE_ID::internal::ParseTable schema[17] PROTOBUF_SECTION_VARIABLE(protodesc_cold); static const ::PROTOBUF_NAMESPACE_ID::internal::FieldMetadata field_metadata[]; static const ::PROTOBUF_NAMESPACE_ID::internal::SerializationTable serialization_table[]; @@ -75,9 +75,6 @@ extern IndexParamDefaultTypeInternal _IndexParam_default_instance_; class InsertParam; class InsertParamDefaultTypeInternal; extern InsertParamDefaultTypeInternal _InsertParam_default_instance_; -class QueryResult; -class QueryResultDefaultTypeInternal; -extern QueryResultDefaultTypeInternal _QueryResult_default_instance_; class Range; class RangeDefaultTypeInternal; extern RangeDefaultTypeInternal _Range_default_instance_; @@ -108,9 +105,6 @@ extern TableSchemaDefaultTypeInternal _TableSchema_default_instance_; class TopKQueryResult; class TopKQueryResultDefaultTypeInternal; extern TopKQueryResultDefaultTypeInternal _TopKQueryResult_default_instance_; -class TopKQueryResultList; -class TopKQueryResultListDefaultTypeInternal; -extern TopKQueryResultListDefaultTypeInternal _TopKQueryResultList_default_instance_; class VectorIds; class VectorIdsDefaultTypeInternal; extern VectorIdsDefaultTypeInternal _VectorIds_default_instance_; @@ -123,7 +117,6 @@ template<> ::milvus::grpc::DeleteByRangeParam* Arena::CreateMaybeMessage<::milvu template<> ::milvus::grpc::Index* Arena::CreateMaybeMessage<::milvus::grpc::Index>(Arena*); template<> ::milvus::grpc::IndexParam* Arena::CreateMaybeMessage<::milvus::grpc::IndexParam>(Arena*); template<> ::milvus::grpc::InsertParam* Arena::CreateMaybeMessage<::milvus::grpc::InsertParam>(Arena*); -template<> ::milvus::grpc::QueryResult* Arena::CreateMaybeMessage<::milvus::grpc::QueryResult>(Arena*); template<> ::milvus::grpc::Range* Arena::CreateMaybeMessage<::milvus::grpc::Range>(Arena*); template<> ::milvus::grpc::RowRecord* Arena::CreateMaybeMessage<::milvus::grpc::RowRecord>(Arena*); template<> ::milvus::grpc::SearchInFilesParam* Arena::CreateMaybeMessage<::milvus::grpc::SearchInFilesParam>(Arena*); @@ -134,7 +127,6 @@ template<> ::milvus::grpc::TableNameList* Arena::CreateMaybeMessage<::milvus::gr template<> ::milvus::grpc::TableRowCount* Arena::CreateMaybeMessage<::milvus::grpc::TableRowCount>(Arena*); template<> ::milvus::grpc::TableSchema* Arena::CreateMaybeMessage<::milvus::grpc::TableSchema>(Arena*); template<> ::milvus::grpc::TopKQueryResult* Arena::CreateMaybeMessage<::milvus::grpc::TopKQueryResult>(Arena*); -template<> ::milvus::grpc::TopKQueryResultList* Arena::CreateMaybeMessage<::milvus::grpc::TopKQueryResultList>(Arena*); template<> ::milvus::grpc::VectorIds* Arena::CreateMaybeMessage<::milvus::grpc::VectorIds>(Arena*); PROTOBUF_NAMESPACE_CLOSE namespace milvus { @@ -870,7 +862,8 @@ class RowRecord : void clear_vector_data(); float vector_data(int index) const; void set_vector_data(int index, float value); - void add_vector_data(float value); + //void add_vector_data(float value); + void add_vector_data(std::vector::const_iterator begin, std::vector::const_iterator end); const ::PROTOBUF_NAMESPACE_ID::RepeatedField< float >& vector_data() const; ::PROTOBUF_NAMESPACE_ID::RepeatedField< float >* @@ -1021,7 +1014,9 @@ class InsertParam : void clear_row_id_array(); ::PROTOBUF_NAMESPACE_ID::int64 row_id_array(int index) const; void set_row_id_array(int index, ::PROTOBUF_NAMESPACE_ID::int64 value); - void add_row_id_array(::PROTOBUF_NAMESPACE_ID::int64 value); + //void add_row_id_array(::PROTOBUF_NAMESPACE_ID::int64 value); + void add_row_id_array(std::vector<::PROTOBUF_NAMESPACE_ID::int64>::const_iterator begin, + std::vector<::PROTOBUF_NAMESPACE_ID::int64>::const_iterator end); const ::PROTOBUF_NAMESPACE_ID::RepeatedField< ::PROTOBUF_NAMESPACE_ID::int64 >& row_id_array() const; ::PROTOBUF_NAMESPACE_ID::RepeatedField< ::PROTOBUF_NAMESPACE_ID::int64 >* @@ -1530,144 +1525,6 @@ class SearchInFilesParam : }; // ------------------------------------------------------------------- -class QueryResult : - public ::PROTOBUF_NAMESPACE_ID::Message /* @@protoc_insertion_point(class_definition:milvus.grpc.QueryResult) */ { - public: - QueryResult(); - virtual ~QueryResult(); - - QueryResult(const QueryResult& from); - QueryResult(QueryResult&& from) noexcept - : QueryResult() { - *this = ::std::move(from); - } - - inline QueryResult& operator=(const QueryResult& from) { - CopyFrom(from); - return *this; - } - inline QueryResult& operator=(QueryResult&& from) noexcept { - if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { - if (this != &from) InternalSwap(&from); - } else { - CopyFrom(from); - } - return *this; - } - - static const ::PROTOBUF_NAMESPACE_ID::Descriptor* descriptor() { - return GetDescriptor(); - } - static const ::PROTOBUF_NAMESPACE_ID::Descriptor* GetDescriptor() { - return GetMetadataStatic().descriptor; - } - static const ::PROTOBUF_NAMESPACE_ID::Reflection* GetReflection() { - return GetMetadataStatic().reflection; - } - static const QueryResult& default_instance(); - - static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY - static inline const QueryResult* internal_default_instance() { - return reinterpret_cast( - &_QueryResult_default_instance_); - } - static constexpr int kIndexInFileMessages = - 9; - - friend void swap(QueryResult& a, QueryResult& b) { - a.Swap(&b); - } - inline void Swap(QueryResult* other) { - if (other == this) return; - InternalSwap(other); - } - - // implements Message ---------------------------------------------- - - inline QueryResult* New() const final { - return CreateMaybeMessage(nullptr); - } - - QueryResult* New(::PROTOBUF_NAMESPACE_ID::Arena* arena) const final { - return CreateMaybeMessage(arena); - } - void CopyFrom(const ::PROTOBUF_NAMESPACE_ID::Message& from) final; - void MergeFrom(const ::PROTOBUF_NAMESPACE_ID::Message& from) final; - void CopyFrom(const QueryResult& from); - void MergeFrom(const QueryResult& from); - PROTOBUF_ATTRIBUTE_REINITIALIZES void Clear() final; - bool IsInitialized() const final; - - size_t ByteSizeLong() const final; - #if GOOGLE_PROTOBUF_ENABLE_EXPERIMENTAL_PARSER - const char* _InternalParse(const char* ptr, ::PROTOBUF_NAMESPACE_ID::internal::ParseContext* ctx) final; - #else - bool MergePartialFromCodedStream( - ::PROTOBUF_NAMESPACE_ID::io::CodedInputStream* input) final; - #endif // GOOGLE_PROTOBUF_ENABLE_EXPERIMENTAL_PARSER - void SerializeWithCachedSizes( - ::PROTOBUF_NAMESPACE_ID::io::CodedOutputStream* output) const final; - ::PROTOBUF_NAMESPACE_ID::uint8* InternalSerializeWithCachedSizesToArray( - ::PROTOBUF_NAMESPACE_ID::uint8* target) const final; - int GetCachedSize() const final { return _cached_size_.Get(); } - - private: - inline void SharedCtor(); - inline void SharedDtor(); - void SetCachedSize(int size) const final; - void InternalSwap(QueryResult* other); - friend class ::PROTOBUF_NAMESPACE_ID::internal::AnyMetadata; - static ::PROTOBUF_NAMESPACE_ID::StringPiece FullMessageName() { - return "milvus.grpc.QueryResult"; - } - private: - inline ::PROTOBUF_NAMESPACE_ID::Arena* GetArenaNoVirtual() const { - return nullptr; - } - inline void* MaybeArenaPtr() const { - return nullptr; - } - public: - - ::PROTOBUF_NAMESPACE_ID::Metadata GetMetadata() const final; - private: - static ::PROTOBUF_NAMESPACE_ID::Metadata GetMetadataStatic() { - ::PROTOBUF_NAMESPACE_ID::internal::AssignDescriptors(&::descriptor_table_milvus_2eproto); - return ::descriptor_table_milvus_2eproto.file_level_metadata[kIndexInFileMessages]; - } - - public: - - // nested types ---------------------------------------------------- - - // accessors ------------------------------------------------------- - - enum : int { - kIdFieldNumber = 1, - kDistanceFieldNumber = 2, - }; - // int64 id = 1; - void clear_id(); - ::PROTOBUF_NAMESPACE_ID::int64 id() const; - void set_id(::PROTOBUF_NAMESPACE_ID::int64 value); - - // double distance = 2; - void clear_distance(); - double distance() const; - void set_distance(double value); - - // @@protoc_insertion_point(class_scope:milvus.grpc.QueryResult) - private: - class _Internal; - - ::PROTOBUF_NAMESPACE_ID::internal::InternalMetadataWithArena _internal_metadata_; - ::PROTOBUF_NAMESPACE_ID::int64 id_; - double distance_; - mutable ::PROTOBUF_NAMESPACE_ID::internal::CachedSize _cached_size_; - friend struct ::TableStruct_milvus_2eproto; -}; -// ------------------------------------------------------------------- - class TopKQueryResult : public ::PROTOBUF_NAMESPACE_ID::Message /* @@protoc_insertion_point(class_definition:milvus.grpc.TopKQueryResult) */ { public: @@ -1710,7 +1567,7 @@ class TopKQueryResult : &_TopKQueryResult_default_instance_); } static constexpr int kIndexInFileMessages = - 10; + 9; friend void swap(TopKQueryResult& a, TopKQueryResult& b) { a.Swap(&b); @@ -1781,156 +1638,33 @@ class TopKQueryResult : // accessors ------------------------------------------------------- enum : int { - kQueryResultArraysFieldNumber = 1, - }; - // repeated .milvus.grpc.QueryResult query_result_arrays = 1; - int query_result_arrays_size() const; - void clear_query_result_arrays(); - ::milvus::grpc::QueryResult* mutable_query_result_arrays(int index); - ::PROTOBUF_NAMESPACE_ID::RepeatedPtrField< ::milvus::grpc::QueryResult >* - mutable_query_result_arrays(); - const ::milvus::grpc::QueryResult& query_result_arrays(int index) const; - ::milvus::grpc::QueryResult* add_query_result_arrays(); - const ::PROTOBUF_NAMESPACE_ID::RepeatedPtrField< ::milvus::grpc::QueryResult >& - query_result_arrays() const; - - // @@protoc_insertion_point(class_scope:milvus.grpc.TopKQueryResult) - private: - class _Internal; - - ::PROTOBUF_NAMESPACE_ID::internal::InternalMetadataWithArena _internal_metadata_; - ::PROTOBUF_NAMESPACE_ID::RepeatedPtrField< ::milvus::grpc::QueryResult > query_result_arrays_; - mutable ::PROTOBUF_NAMESPACE_ID::internal::CachedSize _cached_size_; - friend struct ::TableStruct_milvus_2eproto; -}; -// ------------------------------------------------------------------- - -class TopKQueryResultList : - public ::PROTOBUF_NAMESPACE_ID::Message /* @@protoc_insertion_point(class_definition:milvus.grpc.TopKQueryResultList) */ { - public: - TopKQueryResultList(); - virtual ~TopKQueryResultList(); - - TopKQueryResultList(const TopKQueryResultList& from); - TopKQueryResultList(TopKQueryResultList&& from) noexcept - : TopKQueryResultList() { - *this = ::std::move(from); - } - - inline TopKQueryResultList& operator=(const TopKQueryResultList& from) { - CopyFrom(from); - return *this; - } - inline TopKQueryResultList& operator=(TopKQueryResultList&& from) noexcept { - if (GetArenaNoVirtual() == from.GetArenaNoVirtual()) { - if (this != &from) InternalSwap(&from); - } else { - CopyFrom(from); - } - return *this; - } - - static const ::PROTOBUF_NAMESPACE_ID::Descriptor* descriptor() { - return GetDescriptor(); - } - static const ::PROTOBUF_NAMESPACE_ID::Descriptor* GetDescriptor() { - return GetMetadataStatic().descriptor; - } - static const ::PROTOBUF_NAMESPACE_ID::Reflection* GetReflection() { - return GetMetadataStatic().reflection; - } - static const TopKQueryResultList& default_instance(); - - static void InitAsDefaultInstance(); // FOR INTERNAL USE ONLY - static inline const TopKQueryResultList* internal_default_instance() { - return reinterpret_cast( - &_TopKQueryResultList_default_instance_); - } - static constexpr int kIndexInFileMessages = - 11; - - friend void swap(TopKQueryResultList& a, TopKQueryResultList& b) { - a.Swap(&b); - } - inline void Swap(TopKQueryResultList* other) { - if (other == this) return; - InternalSwap(other); - } - - // implements Message ---------------------------------------------- - - inline TopKQueryResultList* New() const final { - return CreateMaybeMessage(nullptr); - } - - TopKQueryResultList* New(::PROTOBUF_NAMESPACE_ID::Arena* arena) const final { - return CreateMaybeMessage(arena); - } - void CopyFrom(const ::PROTOBUF_NAMESPACE_ID::Message& from) final; - void MergeFrom(const ::PROTOBUF_NAMESPACE_ID::Message& from) final; - void CopyFrom(const TopKQueryResultList& from); - void MergeFrom(const TopKQueryResultList& from); - PROTOBUF_ATTRIBUTE_REINITIALIZES void Clear() final; - bool IsInitialized() const final; - - size_t ByteSizeLong() const final; - #if GOOGLE_PROTOBUF_ENABLE_EXPERIMENTAL_PARSER - const char* _InternalParse(const char* ptr, ::PROTOBUF_NAMESPACE_ID::internal::ParseContext* ctx) final; - #else - bool MergePartialFromCodedStream( - ::PROTOBUF_NAMESPACE_ID::io::CodedInputStream* input) final; - #endif // GOOGLE_PROTOBUF_ENABLE_EXPERIMENTAL_PARSER - void SerializeWithCachedSizes( - ::PROTOBUF_NAMESPACE_ID::io::CodedOutputStream* output) const final; - ::PROTOBUF_NAMESPACE_ID::uint8* InternalSerializeWithCachedSizesToArray( - ::PROTOBUF_NAMESPACE_ID::uint8* target) const final; - int GetCachedSize() const final { return _cached_size_.Get(); } - - private: - inline void SharedCtor(); - inline void SharedDtor(); - void SetCachedSize(int size) const final; - void InternalSwap(TopKQueryResultList* other); - friend class ::PROTOBUF_NAMESPACE_ID::internal::AnyMetadata; - static ::PROTOBUF_NAMESPACE_ID::StringPiece FullMessageName() { - return "milvus.grpc.TopKQueryResultList"; - } - private: - inline ::PROTOBUF_NAMESPACE_ID::Arena* GetArenaNoVirtual() const { - return nullptr; - } - inline void* MaybeArenaPtr() const { - return nullptr; - } - public: - - ::PROTOBUF_NAMESPACE_ID::Metadata GetMetadata() const final; - private: - static ::PROTOBUF_NAMESPACE_ID::Metadata GetMetadataStatic() { - ::PROTOBUF_NAMESPACE_ID::internal::AssignDescriptors(&::descriptor_table_milvus_2eproto); - return ::descriptor_table_milvus_2eproto.file_level_metadata[kIndexInFileMessages]; - } - - public: - - // nested types ---------------------------------------------------- - - // accessors ------------------------------------------------------- - - enum : int { - kTopkQueryResultFieldNumber = 2, + kIdsBinaryFieldNumber = 4, + kDistancesBinaryFieldNumber = 5, kStatusFieldNumber = 1, + kNqFieldNumber = 2, + kTopkFieldNumber = 3, }; - // repeated .milvus.grpc.TopKQueryResult topk_query_result = 2; - int topk_query_result_size() const; - void clear_topk_query_result(); - ::milvus::grpc::TopKQueryResult* mutable_topk_query_result(int index); - ::PROTOBUF_NAMESPACE_ID::RepeatedPtrField< ::milvus::grpc::TopKQueryResult >* - mutable_topk_query_result(); - const ::milvus::grpc::TopKQueryResult& topk_query_result(int index) const; - ::milvus::grpc::TopKQueryResult* add_topk_query_result(); - const ::PROTOBUF_NAMESPACE_ID::RepeatedPtrField< ::milvus::grpc::TopKQueryResult >& - topk_query_result() const; + // bytes ids_binary = 4; + void clear_ids_binary(); + const std::string& ids_binary() const; + void set_ids_binary(const std::string& value); + void set_ids_binary(std::string&& value); + void set_ids_binary(const char* value); + void set_ids_binary(const void* value, size_t size); + std::string* mutable_ids_binary(); + std::string* release_ids_binary(); + void set_allocated_ids_binary(std::string* ids_binary); + + // bytes distances_binary = 5; + void clear_distances_binary(); + const std::string& distances_binary() const; + void set_distances_binary(const std::string& value); + void set_distances_binary(std::string&& value); + void set_distances_binary(const char* value); + void set_distances_binary(const void* value, size_t size); + std::string* mutable_distances_binary(); + std::string* release_distances_binary(); + void set_allocated_distances_binary(std::string* distances_binary); // .milvus.grpc.Status status = 1; bool has_status() const; @@ -1940,13 +1674,26 @@ class TopKQueryResultList : ::milvus::grpc::Status* mutable_status(); void set_allocated_status(::milvus::grpc::Status* status); - // @@protoc_insertion_point(class_scope:milvus.grpc.TopKQueryResultList) + // int64 nq = 2; + void clear_nq(); + ::PROTOBUF_NAMESPACE_ID::int64 nq() const; + void set_nq(::PROTOBUF_NAMESPACE_ID::int64 value); + + // int64 topk = 3; + void clear_topk(); + ::PROTOBUF_NAMESPACE_ID::int64 topk() const; + void set_topk(::PROTOBUF_NAMESPACE_ID::int64 value); + + // @@protoc_insertion_point(class_scope:milvus.grpc.TopKQueryResult) private: class _Internal; ::PROTOBUF_NAMESPACE_ID::internal::InternalMetadataWithArena _internal_metadata_; - ::PROTOBUF_NAMESPACE_ID::RepeatedPtrField< ::milvus::grpc::TopKQueryResult > topk_query_result_; + ::PROTOBUF_NAMESPACE_ID::internal::ArenaStringPtr ids_binary_; + ::PROTOBUF_NAMESPACE_ID::internal::ArenaStringPtr distances_binary_; ::milvus::grpc::Status* status_; + ::PROTOBUF_NAMESPACE_ID::int64 nq_; + ::PROTOBUF_NAMESPACE_ID::int64 topk_; mutable ::PROTOBUF_NAMESPACE_ID::internal::CachedSize _cached_size_; friend struct ::TableStruct_milvus_2eproto; }; @@ -1994,7 +1741,7 @@ class StringReply : &_StringReply_default_instance_); } static constexpr int kIndexInFileMessages = - 12; + 10; friend void swap(StringReply& a, StringReply& b) { a.Swap(&b); @@ -2141,7 +1888,7 @@ class BoolReply : &_BoolReply_default_instance_); } static constexpr int kIndexInFileMessages = - 13; + 11; friend void swap(BoolReply& a, BoolReply& b) { a.Swap(&b); @@ -2282,7 +2029,7 @@ class TableRowCount : &_TableRowCount_default_instance_); } static constexpr int kIndexInFileMessages = - 14; + 12; friend void swap(TableRowCount& a, TableRowCount& b) { a.Swap(&b); @@ -2423,7 +2170,7 @@ class Command : &_Command_default_instance_); } static constexpr int kIndexInFileMessages = - 15; + 13; friend void swap(Command& a, Command& b) { a.Swap(&b); @@ -2560,7 +2307,7 @@ class Index : &_Index_default_instance_); } static constexpr int kIndexInFileMessages = - 16; + 14; friend void swap(Index& a, Index& b) { a.Swap(&b); @@ -2698,7 +2445,7 @@ class IndexParam : &_IndexParam_default_instance_); } static constexpr int kIndexInFileMessages = - 17; + 15; friend void swap(IndexParam& a, IndexParam& b) { a.Swap(&b); @@ -2855,7 +2602,7 @@ class DeleteByRangeParam : &_DeleteByRangeParam_default_instance_); } static constexpr int kIndexInFileMessages = - 18; + 16; friend void swap(DeleteByRangeParam& a, DeleteByRangeParam& b) { a.Swap(&b); @@ -3401,8 +3148,13 @@ inline void RowRecord::set_vector_data(int index, float value) { vector_data_.Set(index, value); // @@protoc_insertion_point(field_set:milvus.grpc.RowRecord.vector_data) } -inline void RowRecord::add_vector_data(float value) { - vector_data_.Add(value); +//inline void RowRecord::add_vector_data(float value) { +// vector_data_.Add(value); +// // @@protoc_insertion_point(field_add:milvus.grpc.RowRecord.vector_data) +//} +inline void RowRecord::add_vector_data(std::vector::const_iterator begin, + std::vector::const_iterator end) { + vector_data_.Add(begin, end); // @@protoc_insertion_point(field_add:milvus.grpc.RowRecord.vector_data) } inline const ::PROTOBUF_NAMESPACE_ID::RepeatedField< float >& @@ -3516,8 +3268,13 @@ inline void InsertParam::set_row_id_array(int index, ::PROTOBUF_NAMESPACE_ID::in row_id_array_.Set(index, value); // @@protoc_insertion_point(field_set:milvus.grpc.InsertParam.row_id_array) } -inline void InsertParam::add_row_id_array(::PROTOBUF_NAMESPACE_ID::int64 value) { - row_id_array_.Add(value); +//inline void InsertParam::add_row_id_array(::PROTOBUF_NAMESPACE_ID::int64 value) { +// row_id_array_.Add(value); +// // @@protoc_insertion_point(field_add:milvus.grpc.InsertParam.row_id_array) +//} +inline void InsertParam::add_row_id_array(std::vector<::PROTOBUF_NAMESPACE_ID::int64>::const_iterator begin, + std::vector<::PROTOBUF_NAMESPACE_ID::int64>::const_iterator end) { + row_id_array_.Add(begin, end); // @@protoc_insertion_point(field_add:milvus.grpc.InsertParam.row_id_array) } inline const ::PROTOBUF_NAMESPACE_ID::RepeatedField< ::PROTOBUF_NAMESPACE_ID::int64 >& @@ -3875,101 +3632,35 @@ inline void SearchInFilesParam::set_allocated_search_param(::milvus::grpc::Searc // ------------------------------------------------------------------- -// QueryResult - -// int64 id = 1; -inline void QueryResult::clear_id() { - id_ = PROTOBUF_LONGLONG(0); -} -inline ::PROTOBUF_NAMESPACE_ID::int64 QueryResult::id() const { - // @@protoc_insertion_point(field_get:milvus.grpc.QueryResult.id) - return id_; -} -inline void QueryResult::set_id(::PROTOBUF_NAMESPACE_ID::int64 value) { - - id_ = value; - // @@protoc_insertion_point(field_set:milvus.grpc.QueryResult.id) -} - -// double distance = 2; -inline void QueryResult::clear_distance() { - distance_ = 0; -} -inline double QueryResult::distance() const { - // @@protoc_insertion_point(field_get:milvus.grpc.QueryResult.distance) - return distance_; -} -inline void QueryResult::set_distance(double value) { - - distance_ = value; - // @@protoc_insertion_point(field_set:milvus.grpc.QueryResult.distance) -} - -// ------------------------------------------------------------------- - // TopKQueryResult -// repeated .milvus.grpc.QueryResult query_result_arrays = 1; -inline int TopKQueryResult::query_result_arrays_size() const { - return query_result_arrays_.size(); -} -inline void TopKQueryResult::clear_query_result_arrays() { - query_result_arrays_.Clear(); -} -inline ::milvus::grpc::QueryResult* TopKQueryResult::mutable_query_result_arrays(int index) { - // @@protoc_insertion_point(field_mutable:milvus.grpc.TopKQueryResult.query_result_arrays) - return query_result_arrays_.Mutable(index); -} -inline ::PROTOBUF_NAMESPACE_ID::RepeatedPtrField< ::milvus::grpc::QueryResult >* -TopKQueryResult::mutable_query_result_arrays() { - // @@protoc_insertion_point(field_mutable_list:milvus.grpc.TopKQueryResult.query_result_arrays) - return &query_result_arrays_; -} -inline const ::milvus::grpc::QueryResult& TopKQueryResult::query_result_arrays(int index) const { - // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResult.query_result_arrays) - return query_result_arrays_.Get(index); -} -inline ::milvus::grpc::QueryResult* TopKQueryResult::add_query_result_arrays() { - // @@protoc_insertion_point(field_add:milvus.grpc.TopKQueryResult.query_result_arrays) - return query_result_arrays_.Add(); -} -inline const ::PROTOBUF_NAMESPACE_ID::RepeatedPtrField< ::milvus::grpc::QueryResult >& -TopKQueryResult::query_result_arrays() const { - // @@protoc_insertion_point(field_list:milvus.grpc.TopKQueryResult.query_result_arrays) - return query_result_arrays_; -} - -// ------------------------------------------------------------------- - -// TopKQueryResultList - // .milvus.grpc.Status status = 1; -inline bool TopKQueryResultList::has_status() const { +inline bool TopKQueryResult::has_status() const { return this != internal_default_instance() && status_ != nullptr; } -inline const ::milvus::grpc::Status& TopKQueryResultList::status() const { +inline const ::milvus::grpc::Status& TopKQueryResult::status() const { const ::milvus::grpc::Status* p = status_; - // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResultList.status) + // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResult.status) return p != nullptr ? *p : *reinterpret_cast( &::milvus::grpc::_Status_default_instance_); } -inline ::milvus::grpc::Status* TopKQueryResultList::release_status() { - // @@protoc_insertion_point(field_release:milvus.grpc.TopKQueryResultList.status) +inline ::milvus::grpc::Status* TopKQueryResult::release_status() { + // @@protoc_insertion_point(field_release:milvus.grpc.TopKQueryResult.status) ::milvus::grpc::Status* temp = status_; status_ = nullptr; return temp; } -inline ::milvus::grpc::Status* TopKQueryResultList::mutable_status() { +inline ::milvus::grpc::Status* TopKQueryResult::mutable_status() { if (status_ == nullptr) { auto* p = CreateMaybeMessage<::milvus::grpc::Status>(GetArenaNoVirtual()); status_ = p; } - // @@protoc_insertion_point(field_mutable:milvus.grpc.TopKQueryResultList.status) + // @@protoc_insertion_point(field_mutable:milvus.grpc.TopKQueryResult.status) return status_; } -inline void TopKQueryResultList::set_allocated_status(::milvus::grpc::Status* status) { +inline void TopKQueryResult::set_allocated_status(::milvus::grpc::Status* status) { ::PROTOBUF_NAMESPACE_ID::Arena* message_arena = GetArenaNoVirtual(); if (message_arena == nullptr) { delete reinterpret_cast< ::PROTOBUF_NAMESPACE_ID::MessageLite*>(status_); @@ -3985,37 +3676,137 @@ inline void TopKQueryResultList::set_allocated_status(::milvus::grpc::Status* st } status_ = status; - // @@protoc_insertion_point(field_set_allocated:milvus.grpc.TopKQueryResultList.status) + // @@protoc_insertion_point(field_set_allocated:milvus.grpc.TopKQueryResult.status) } -// repeated .milvus.grpc.TopKQueryResult topk_query_result = 2; -inline int TopKQueryResultList::topk_query_result_size() const { - return topk_query_result_.size(); +// int64 nq = 2; +inline void TopKQueryResult::clear_nq() { + nq_ = PROTOBUF_LONGLONG(0); } -inline void TopKQueryResultList::clear_topk_query_result() { - topk_query_result_.Clear(); +inline ::PROTOBUF_NAMESPACE_ID::int64 TopKQueryResult::nq() const { + // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResult.nq) + return nq_; } -inline ::milvus::grpc::TopKQueryResult* TopKQueryResultList::mutable_topk_query_result(int index) { - // @@protoc_insertion_point(field_mutable:milvus.grpc.TopKQueryResultList.topk_query_result) - return topk_query_result_.Mutable(index); +inline void TopKQueryResult::set_nq(::PROTOBUF_NAMESPACE_ID::int64 value) { + + nq_ = value; + // @@protoc_insertion_point(field_set:milvus.grpc.TopKQueryResult.nq) } -inline ::PROTOBUF_NAMESPACE_ID::RepeatedPtrField< ::milvus::grpc::TopKQueryResult >* -TopKQueryResultList::mutable_topk_query_result() { - // @@protoc_insertion_point(field_mutable_list:milvus.grpc.TopKQueryResultList.topk_query_result) - return &topk_query_result_; + +// int64 topk = 3; +inline void TopKQueryResult::clear_topk() { + topk_ = PROTOBUF_LONGLONG(0); } -inline const ::milvus::grpc::TopKQueryResult& TopKQueryResultList::topk_query_result(int index) const { - // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResultList.topk_query_result) - return topk_query_result_.Get(index); +inline ::PROTOBUF_NAMESPACE_ID::int64 TopKQueryResult::topk() const { + // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResult.topk) + return topk_; } -inline ::milvus::grpc::TopKQueryResult* TopKQueryResultList::add_topk_query_result() { - // @@protoc_insertion_point(field_add:milvus.grpc.TopKQueryResultList.topk_query_result) - return topk_query_result_.Add(); +inline void TopKQueryResult::set_topk(::PROTOBUF_NAMESPACE_ID::int64 value) { + + topk_ = value; + // @@protoc_insertion_point(field_set:milvus.grpc.TopKQueryResult.topk) } -inline const ::PROTOBUF_NAMESPACE_ID::RepeatedPtrField< ::milvus::grpc::TopKQueryResult >& -TopKQueryResultList::topk_query_result() const { - // @@protoc_insertion_point(field_list:milvus.grpc.TopKQueryResultList.topk_query_result) - return topk_query_result_; + +// bytes ids_binary = 4; +inline void TopKQueryResult::clear_ids_binary() { + ids_binary_.ClearToEmptyNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); +} +inline const std::string& TopKQueryResult::ids_binary() const { + // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResult.ids_binary) + return ids_binary_.GetNoArena(); +} +inline void TopKQueryResult::set_ids_binary(const std::string& value) { + + ids_binary_.SetNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:milvus.grpc.TopKQueryResult.ids_binary) +} +inline void TopKQueryResult::set_ids_binary(std::string&& value) { + + ids_binary_.SetNoArena( + &::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:milvus.grpc.TopKQueryResult.ids_binary) +} +inline void TopKQueryResult::set_ids_binary(const char* value) { + GOOGLE_DCHECK(value != nullptr); + + ids_binary_.SetNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:milvus.grpc.TopKQueryResult.ids_binary) +} +inline void TopKQueryResult::set_ids_binary(const void* value, size_t size) { + + ids_binary_.SetNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:milvus.grpc.TopKQueryResult.ids_binary) +} +inline std::string* TopKQueryResult::mutable_ids_binary() { + + // @@protoc_insertion_point(field_mutable:milvus.grpc.TopKQueryResult.ids_binary) + return ids_binary_.MutableNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); +} +inline std::string* TopKQueryResult::release_ids_binary() { + // @@protoc_insertion_point(field_release:milvus.grpc.TopKQueryResult.ids_binary) + + return ids_binary_.ReleaseNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); +} +inline void TopKQueryResult::set_allocated_ids_binary(std::string* ids_binary) { + if (ids_binary != nullptr) { + + } else { + + } + ids_binary_.SetAllocatedNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), ids_binary); + // @@protoc_insertion_point(field_set_allocated:milvus.grpc.TopKQueryResult.ids_binary) +} + +// bytes distances_binary = 5; +inline void TopKQueryResult::clear_distances_binary() { + distances_binary_.ClearToEmptyNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); +} +inline const std::string& TopKQueryResult::distances_binary() const { + // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResult.distances_binary) + return distances_binary_.GetNoArena(); +} +inline void TopKQueryResult::set_distances_binary(const std::string& value) { + + distances_binary_.SetNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), value); + // @@protoc_insertion_point(field_set:milvus.grpc.TopKQueryResult.distances_binary) +} +inline void TopKQueryResult::set_distances_binary(std::string&& value) { + + distances_binary_.SetNoArena( + &::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); + // @@protoc_insertion_point(field_set_rvalue:milvus.grpc.TopKQueryResult.distances_binary) +} +inline void TopKQueryResult::set_distances_binary(const char* value) { + GOOGLE_DCHECK(value != nullptr); + + distances_binary_.SetNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); + // @@protoc_insertion_point(field_set_char:milvus.grpc.TopKQueryResult.distances_binary) +} +inline void TopKQueryResult::set_distances_binary(const void* value, size_t size) { + + distances_binary_.SetNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), + ::std::string(reinterpret_cast(value), size)); + // @@protoc_insertion_point(field_set_pointer:milvus.grpc.TopKQueryResult.distances_binary) +} +inline std::string* TopKQueryResult::mutable_distances_binary() { + + // @@protoc_insertion_point(field_mutable:milvus.grpc.TopKQueryResult.distances_binary) + return distances_binary_.MutableNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); +} +inline std::string* TopKQueryResult::release_distances_binary() { + // @@protoc_insertion_point(field_release:milvus.grpc.TopKQueryResult.distances_binary) + + return distances_binary_.ReleaseNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); +} +inline void TopKQueryResult::set_allocated_distances_binary(std::string* distances_binary) { + if (distances_binary != nullptr) { + + } else { + + } + distances_binary_.SetAllocatedNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), distances_binary); + // @@protoc_insertion_point(field_set_allocated:milvus.grpc.TopKQueryResult.distances_binary) } // ------------------------------------------------------------------- @@ -4623,10 +4414,6 @@ inline void DeleteByRangeParam::set_allocated_table_name(std::string* table_name // ------------------------------------------------------------------- -// ------------------------------------------------------------------- - -// ------------------------------------------------------------------- - // @@protoc_insertion_point(namespace_scope) diff --git a/core/src/grpc/milvus.proto b/core/src/grpc/milvus.proto index 2856dfc6d9..e7fcdc3b2f 100644 --- a/core/src/grpc/milvus.proto +++ b/core/src/grpc/milvus.proto @@ -84,24 +84,12 @@ message SearchInFilesParam { /** * @brief Query result params */ -message QueryResult { - int64 id = 1; - double distance = 2; -} - -/** - * @brief TopK query result - */ message TopKQueryResult { - repeated QueryResult query_result_arrays = 1; -} - -/** - * @brief List of topK query result - */ -message TopKQueryResultList { Status status = 1; - repeated TopKQueryResult topk_query_result = 2; + int64 nq = 2; + int64 topk = 3; + bytes ids_binary = 4; + bytes distances_binary = 5; } /** @@ -227,7 +215,7 @@ service MilvusService { * * @return query result array. */ - rpc Search(SearchParam) returns (TopKQueryResultList) {} + rpc Search(SearchParam) returns (TopKQueryResult) {} /** * @brief Internal use query interface @@ -241,7 +229,7 @@ service MilvusService { * * @return query result array. */ - rpc SearchInFiles(SearchInFilesParam) returns (TopKQueryResultList) {} + rpc SearchInFiles(SearchInFilesParam) returns (TopKQueryResult) {} /** * @brief Get table schema diff --git a/core/src/sdk/examples/grpcsimple/src/ClientTest.cpp b/core/src/sdk/examples/grpcsimple/src/ClientTest.cpp index 069283200f..a6e2b55281 100644 --- a/core/src/sdk/examples/grpcsimple/src/ClientTest.cpp +++ b/core/src/sdk/examples/grpcsimple/src/ClientTest.cpp @@ -57,22 +57,22 @@ PrintTableSchema(const milvus::TableSchema& tb_schema) { void PrintSearchResult(const std::vector>& search_record_array, - const std::vector& topk_query_result_array) { + const milvus::TopKQueryResult& topk_query_result) { BLOCK_SPLITER - std::cout << "Returned result count: " << topk_query_result_array.size() << std::endl; + size_t nq = topk_query_result.row_num; + size_t topk = topk_query_result.topk; + std::cout << "Returned result count: " << nq * topk << std::endl; int32_t index = 0; - for (auto& result : topk_query_result_array) { + for (size_t i = 0; i < nq; i++) { auto search_id = search_record_array[index].first; index++; - std::cout << "No." << std::to_string(index) << " vector " << std::to_string(search_id) << " top " - << std::to_string(result.query_result_arrays.size()) << " search result:" << std::endl; - for (auto& item : result.query_result_arrays) { - std::cout << "\t" << std::to_string(item.id) << "\tdistance:" << std::to_string(item.distance); - std::cout << std::endl; + std::cout << "No." << index << " vector " << search_id << " top " << topk << " search result:" << std::endl; + for (size_t j = 0; j < topk; j++) { + size_t idx = i * nq + j; + std::cout << "\t" << topk_query_result.ids[idx] << "\t" << topk_query_result.distances[idx] << std::endl; } } - BLOCK_SPLITER } @@ -166,11 +166,13 @@ class TimeRecorder { void CheckResult(const std::vector>& search_record_array, - const std::vector& topk_query_result_array) { + const milvus::TopKQueryResult& topk_query_result) { BLOCK_SPLITER + size_t nq = topk_query_result.row_num; + size_t result_k = topk_query_result.topk; int64_t index = 0; - for (auto& result : topk_query_result_array) { - auto result_id = result.query_result_arrays[0].id; + for (size_t i = 0; i < nq; i++) { + auto result_id = topk_query_result.ids[i * result_k]; auto search_id = search_record_array[index++].first; if (result_id != search_id) { std::cout << "The top 1 result is wrong: " << result_id << " vs. " << search_id << std::endl; @@ -196,19 +198,18 @@ DoSearch(std::shared_ptr conn, } auto start = std::chrono::high_resolution_clock::now(); - std::vector topk_query_result_array; + milvus::TopKQueryResult topk_query_result; { TimeRecorder rc(phase_name); - milvus::Status stat = - conn->Search(TABLE_NAME, record_array, query_range_array, TOP_K, 32, topk_query_result_array); + milvus::Status stat = conn->Search(TABLE_NAME, record_array, query_range_array, TOP_K, 32, topk_query_result); std::cout << "SearchVector function call status: " << stat.message() << std::endl; } auto finish = std::chrono::high_resolution_clock::now(); std::cout << "SEARCHVECTOR COST: " << std::chrono::duration_cast>(finish - start).count() << "s\n"; - PrintSearchResult(search_record_array, topk_query_result_array); - CheckResult(search_record_array, topk_query_result_array); + PrintSearchResult(search_record_array, topk_query_result); + CheckResult(search_record_array, topk_query_result); } } // namespace diff --git a/core/src/sdk/grpc/ClientProxy.cpp b/core/src/sdk/grpc/ClientProxy.cpp index c726cfc532..04878acf7f 100644 --- a/core/src/sdk/grpc/ClientProxy.cpp +++ b/core/src/sdk/grpc/ClientProxy.cpp @@ -189,22 +189,19 @@ ClientProxy::Insert(const std::string& table_name, const std::vector& for (auto& record : record_array) { ::milvus::grpc::RowRecord* grpc_record = insert_param.add_row_record_array(); for (size_t i = 0; i < record.data.size(); i++) { - grpc_record->add_vector_data(record.data[i]); + grpc_record->add_vector_data(record.data.begin(), record.data.end()); } } // Single thread ::milvus::grpc::VectorIds vector_ids; if (!id_array.empty()) { - for (auto i = 0; i < id_array.size(); i++) { - insert_param.add_row_id_array(id_array[i]); - } + insert_param.add_row_id_array(id_array.begin(), id_array.end()); client_ptr_->Insert(vector_ids, insert_param, status); } else { client_ptr_->Insert(vector_ids, insert_param, status); - for (size_t i = 0; i < vector_ids.vector_id_array_size(); i++) { - id_array.push_back(vector_ids.vector_id_array(i)); - } + /* return Milvus generated ids back to user */ + id_array.insert(id_array.end(), vector_ids.vector_id_array().begin(), vector_ids.vector_id_array().end()); } #endif } catch (std::exception& ex) { @@ -217,7 +214,7 @@ ClientProxy::Insert(const std::string& table_name, const std::vector& Status ClientProxy::Search(const std::string& table_name, const std::vector& query_record_array, const std::vector& query_range_array, int64_t topk, int64_t nprobe, - std::vector& topk_query_result_array) { + TopKQueryResult& topk_query_result) { try { // step 1: convert vectors data ::milvus::grpc::SearchParam search_param; @@ -226,9 +223,7 @@ ClientProxy::Search(const std::string& table_name, const std::vector& search_param.set_nprobe(nprobe); for (auto& record : query_record_array) { ::milvus::grpc::RowRecord* row_record = search_param.add_query_record_array(); - for (auto& rec : record.data) { - row_record->add_vector_data(rec); - } + row_record->add_vector_data(record.data.begin(), record.data.end()); } // step 2: convert range array @@ -239,21 +234,17 @@ ClientProxy::Search(const std::string& table_name, const std::vector& } // step 3: search vectors - ::milvus::grpc::TopKQueryResultList topk_query_result_list; - Status status = client_ptr_->Search(topk_query_result_list, search_param); + ::milvus::grpc::TopKQueryResult result; + Status status = client_ptr_->Search(result, search_param); // step 4: convert result array - for (uint64_t i = 0; i < topk_query_result_list.topk_query_result_size(); ++i) { - TopKQueryResult result; - for (uint64_t j = 0; j < topk_query_result_list.topk_query_result(i).query_result_arrays_size(); ++j) { - QueryResult query_result; - query_result.id = topk_query_result_list.topk_query_result(i).query_result_arrays(j).id(); - query_result.distance = topk_query_result_list.topk_query_result(i).query_result_arrays(j).distance(); - result.query_result_arrays.emplace_back(query_result); - } + topk_query_result.row_num = result.nq(); + topk_query_result.topk = result.topk(); + topk_query_result.ids.resize(result.ids_binary().size()); + memcpy(topk_query_result.ids.data(), result.ids_binary().data(), result.ids_binary().size()); + topk_query_result.distances.resize(result.distances_binary().size()); + memcpy(topk_query_result.distances.data(), result.distances_binary().data(), result.distances_binary().size()); - topk_query_result_array.emplace_back(result); - } return status; } catch (std::exception& ex) { return Status(StatusCode::UnknownError, "fail to search vectors: " + std::string(ex.what())); diff --git a/core/src/sdk/grpc/ClientProxy.h b/core/src/sdk/grpc/ClientProxy.h index dbeacc1380..8bcece4f20 100644 --- a/core/src/sdk/grpc/ClientProxy.h +++ b/core/src/sdk/grpc/ClientProxy.h @@ -60,7 +60,7 @@ class ClientProxy : public Connection { Status Search(const std::string& table_name, const std::vector& query_record_array, const std::vector& query_range_array, int64_t topk, int64_t nprobe, - std::vector& topk_query_result_array) override; + TopKQueryResult& topk_query_result) override; Status DescribeTable(const std::string& table_name, TableSchema& table_schema) override; diff --git a/core/src/sdk/grpc/GrpcClient.cpp b/core/src/sdk/grpc/GrpcClient.cpp index 5c27c3b73f..5f278eb289 100644 --- a/core/src/sdk/grpc/GrpcClient.cpp +++ b/core/src/sdk/grpc/GrpcClient.cpp @@ -134,20 +134,20 @@ GrpcClient::Insert(::milvus::grpc::VectorIds& vector_ids, const ::milvus::grpc:: } Status -GrpcClient::Search(::milvus::grpc::TopKQueryResultList& topk_query_result_list, +GrpcClient::Search(::milvus::grpc::TopKQueryResult& topk_query_result, const ::milvus::grpc::SearchParam& search_param) { ::milvus::grpc::TopKQueryResult query_result; ClientContext context; - ::grpc::Status grpc_status = stub_->Search(&context, search_param, &topk_query_result_list); + ::grpc::Status grpc_status = stub_->Search(&context, search_param, &topk_query_result); if (!grpc_status.ok()) { std::cerr << "SearchVector rpc failed!" << std::endl; std::cerr << grpc_status.error_message() << std::endl; return Status(StatusCode::RPCFailed, grpc_status.error_message()); } - if (topk_query_result_list.status().error_code() != grpc::SUCCESS) { - std::cerr << topk_query_result_list.status().reason() << std::endl; - return Status(StatusCode::ServerFailed, topk_query_result_list.status().reason()); + if (topk_query_result.status().error_code() != grpc::SUCCESS) { + std::cerr << topk_query_result.status().reason() << std::endl; + return Status(StatusCode::ServerFailed, topk_query_result.status().reason()); } return Status::OK(); diff --git a/core/src/sdk/grpc/GrpcClient.h b/core/src/sdk/grpc/GrpcClient.h index d2e6ae5095..a5e8b44968 100644 --- a/core/src/sdk/grpc/GrpcClient.h +++ b/core/src/sdk/grpc/GrpcClient.h @@ -57,7 +57,7 @@ class GrpcClient { Insert(grpc::VectorIds& vector_ids, const grpc::InsertParam& insert_param, Status& status); Status - Search(::milvus::grpc::TopKQueryResultList& topk_query_result_list, const grpc::SearchParam& search_param); + Search(::milvus::grpc::TopKQueryResult& topk_query_result, const grpc::SearchParam& search_param); Status DescribeTable(grpc::TableSchema& grpc_schema, const std::string& table_name); diff --git a/core/src/sdk/include/MilvusApi.h b/core/src/sdk/include/MilvusApi.h index 68fe0e9d5c..718334fed0 100644 --- a/core/src/sdk/include/MilvusApi.h +++ b/core/src/sdk/include/MilvusApi.h @@ -78,19 +78,14 @@ struct RowRecord { std::vector data; ///< Vector raw data }; -/** - * @brief Query result - */ -struct QueryResult { - int64_t id; ///< Output result - double distance; ///< Vector similarity distance -}; - /** * @brief TopK query result */ struct TopKQueryResult { - std::vector query_result_arrays; ///< TopK query result + int64_t row_num; + int64_t topk; + std::vector ids; + std::vector distances; }; /** @@ -261,7 +256,7 @@ class Connection { virtual Status Search(const std::string& table_name, const std::vector& query_record_array, const std::vector& query_range_array, int64_t topk, int64_t nprobe, - std::vector& topk_query_result_array) = 0; + TopKQueryResult& topk_query_result) = 0; /** * @brief Show table description diff --git a/core/src/sdk/interface/ConnectionImpl.cpp b/core/src/sdk/interface/ConnectionImpl.cpp index 7034ce4a4d..05d3bf6c36 100644 --- a/core/src/sdk/interface/ConnectionImpl.cpp +++ b/core/src/sdk/interface/ConnectionImpl.cpp @@ -91,9 +91,8 @@ ConnectionImpl::Insert(const std::string& table_name, const std::vector& query_record_array, const std::vector& query_range_array, int64_t topk, int64_t nprobe, - std::vector& topk_query_result_array) { - return client_proxy_->Search(table_name, query_record_array, query_range_array, topk, nprobe, - topk_query_result_array); + TopKQueryResult& topk_query_result) { + return client_proxy_->Search(table_name, query_record_array, query_range_array, topk, nprobe, topk_query_result); } Status diff --git a/core/src/sdk/interface/ConnectionImpl.h b/core/src/sdk/interface/ConnectionImpl.h index 6bc3432bc4..b7dd5b6402 100644 --- a/core/src/sdk/interface/ConnectionImpl.h +++ b/core/src/sdk/interface/ConnectionImpl.h @@ -62,7 +62,7 @@ class ConnectionImpl : public Connection { Status Search(const std::string& table_name, const std::vector& query_record_array, const std::vector& query_range_array, int64_t topk, int64_t nprobe, - std::vector& topk_query_result_array) override; + TopKQueryResult& topk_query_result) override; Status DescribeTable(const std::string& table_name, TableSchema& table_schema) override; diff --git a/core/src/server/grpc_impl/GrpcRequestHandler.cpp b/core/src/server/grpc_impl/GrpcRequestHandler.cpp index a9ee3d77d0..f78aea204c 100644 --- a/core/src/server/grpc_impl/GrpcRequestHandler.cpp +++ b/core/src/server/grpc_impl/GrpcRequestHandler.cpp @@ -75,7 +75,7 @@ GrpcRequestHandler::Insert(::grpc::ServerContext* context, const ::milvus::grpc: ::grpc::Status GrpcRequestHandler::Search(::grpc::ServerContext* context, const ::milvus::grpc::SearchParam* request, - ::milvus::grpc::TopKQueryResultList* response) { + ::milvus::grpc::TopKQueryResult* response) { std::vector file_id_array; BaseTaskPtr task_ptr = SearchTask::Create(request, file_id_array, response); ::milvus::grpc::Status grpc_status; @@ -87,7 +87,7 @@ GrpcRequestHandler::Search(::grpc::ServerContext* context, const ::milvus::grpc: ::grpc::Status GrpcRequestHandler::SearchInFiles(::grpc::ServerContext* context, const ::milvus::grpc::SearchInFilesParam* request, - ::milvus::grpc::TopKQueryResultList* response) { + ::milvus::grpc::TopKQueryResult* response) { std::vector file_id_array; for (int i = 0; i < request->file_id_array_size(); i++) { file_id_array.push_back(request->file_id_array(i)); diff --git a/core/src/server/grpc_impl/GrpcRequestHandler.h b/core/src/server/grpc_impl/GrpcRequestHandler.h index 1a9b591154..935600616c 100644 --- a/core/src/server/grpc_impl/GrpcRequestHandler.h +++ b/core/src/server/grpc_impl/GrpcRequestHandler.h @@ -145,7 +145,7 @@ class GrpcRequestHandler final : public ::milvus::grpc::MilvusService::Service { */ ::grpc::Status Search(::grpc::ServerContext* context, const ::milvus::grpc::SearchParam* request, - ::milvus::grpc::TopKQueryResultList* response) override; + ::milvus::grpc::TopKQueryResult* response) override; /** * @brief Internal use query interface @@ -169,7 +169,7 @@ class GrpcRequestHandler final : public ::milvus::grpc::MilvusService::Service { */ ::grpc::Status SearchInFiles(::grpc::ServerContext* context, const ::milvus::grpc::SearchInFilesParam* request, - ::milvus::grpc::TopKQueryResultList* response) override; + ::milvus::grpc::TopKQueryResult* response) override; /** * @brief Get table schema diff --git a/core/src/server/grpc_impl/GrpcRequestTask.cpp b/core/src/server/grpc_impl/GrpcRequestTask.cpp index 77f262bda6..80bc87e6b7 100644 --- a/core/src/server/grpc_impl/GrpcRequestTask.cpp +++ b/core/src/server/grpc_impl/GrpcRequestTask.cpp @@ -20,6 +20,7 @@ #include #include #include +#include #include //#include @@ -541,16 +542,16 @@ InsertTask::OnExecute() { //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// SearchTask::SearchTask(const ::milvus::grpc::SearchParam* search_vector_infos, - const std::vector& file_id_array, ::milvus::grpc::TopKQueryResultList* response) + const std::vector& file_id_array, ::milvus::grpc::TopKQueryResult* response) : GrpcBaseTask(DQL_TASK_GROUP), search_param_(search_vector_infos), file_id_array_(file_id_array), - topk_result_list(response) { + topk_result_(response) { } BaseTaskPtr SearchTask::Create(const ::milvus::grpc::SearchParam* search_vector_infos, - const std::vector& file_id_array, ::milvus::grpc::TopKQueryResultList* response) { + const std::vector& file_id_array, ::milvus::grpc::TopKQueryResult* response) { if (search_vector_infos == nullptr) { SERVER_LOG_ERROR << "grpc input is null!"; return nullptr; @@ -671,15 +672,20 @@ SearchTask::OnExecute() { size_t result_k = result_ids.size() / record_count; // step 7: construct result array - for (size_t i = 0; i < record_count; i++) { - ::milvus::grpc::TopKQueryResult* topk_query_result = topk_result_list->add_topk_query_result(); - for (size_t j = 0; j < result_k; j++) { - ::milvus::grpc::QueryResult* grpc_result = topk_query_result->add_query_result_arrays(); - size_t idx = i * result_k + j; - grpc_result->set_id(result_ids[idx]); - grpc_result->set_distance(result_distances[idx]); - } - } + topk_result_->set_nq(record_count); + topk_result_->set_topk(result_ids.size() / record_count); + + std::string ids_str; + size_t ids_len = sizeof(int64_t) * result_ids.size(); + ids_str.resize(ids_len); + memcpy((void*)(ids_str.data()), result_ids.data(), ids_len); + topk_result_->set_ids_binary(std::move(ids_str)); + + std::string distances_str; + size_t distances_len = sizeof(float) * result_distances.size(); + distances_str.resize(distances_len); + memcpy((void*)(distances_str.data()), result_distances.data(), distances_len); + topk_result_->set_distances_binary(std::move(distances_str)); // step 8: print time cost percent rc.RecordSection("construct result and send"); diff --git a/core/src/server/grpc_impl/GrpcRequestTask.h b/core/src/server/grpc_impl/GrpcRequestTask.h index ad2828ebf3..3732e2ad41 100644 --- a/core/src/server/grpc_impl/GrpcRequestTask.h +++ b/core/src/server/grpc_impl/GrpcRequestTask.h @@ -153,11 +153,11 @@ class SearchTask : public GrpcBaseTask { public: static BaseTaskPtr Create(const ::milvus::grpc::SearchParam* search_param, const std::vector& file_id_array, - ::milvus::grpc::TopKQueryResultList* response); + ::milvus::grpc::TopKQueryResult* response); protected: SearchTask(const ::milvus::grpc::SearchParam* search_param, const std::vector& file_id_array, - ::milvus::grpc::TopKQueryResultList* response); + ::milvus::grpc::TopKQueryResult* response); Status OnExecute() override; @@ -165,7 +165,7 @@ class SearchTask : public GrpcBaseTask { private: const ::milvus::grpc::SearchParam* search_param_; std::vector file_id_array_; - ::milvus::grpc::TopKQueryResultList* topk_result_list; + ::milvus::grpc::TopKQueryResult* topk_result_; }; //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// diff --git a/core/src/server/grpc_impl/GrpcServer.cpp b/core/src/server/grpc_impl/GrpcServer.cpp index 5e0c5f3169..f5aca77798 100644 --- a/core/src/server/grpc_impl/GrpcServer.cpp +++ b/core/src/server/grpc_impl/GrpcServer.cpp @@ -93,7 +93,7 @@ GrpcServer::StartService() { builder.SetCompressionAlgorithmSupportStatus(GRPC_COMPRESS_STREAM_GZIP, true); builder.SetDefaultCompressionAlgorithm(GRPC_COMPRESS_STREAM_GZIP); - builder.SetDefaultCompressionLevel(GRPC_COMPRESS_LEVEL_HIGH); + builder.SetDefaultCompressionLevel(GRPC_COMPRESS_LEVEL_NONE); GrpcRequestHandler service; diff --git a/core/unittest/server/test_rpc.cpp b/core/unittest/server/test_rpc.cpp index ebbcd810c1..01d41c1789 100644 --- a/core/unittest/server/test_rpc.cpp +++ b/core/unittest/server/test_rpc.cpp @@ -202,9 +202,7 @@ TEST_F(RpcHandlerTest, INSERT_TEST) { ::milvus::grpc::VectorIds vector_ids; for (auto& record : record_array) { ::milvus::grpc::RowRecord* grpc_record = request.add_row_record_array(); - for (size_t i = 0; i < record.size(); i++) { - grpc_record->add_vector_data(record[i]); - } + grpc_record->add_vector_data(record.begin(), record.end()); } handler->Insert(&context, &request, &vector_ids); ASSERT_EQ(vector_ids.vector_id_array_size(), VECTOR_COUNT); @@ -213,7 +211,7 @@ TEST_F(RpcHandlerTest, INSERT_TEST) { TEST_F(RpcHandlerTest, SEARCH_TEST) { ::grpc::ServerContext context; ::milvus::grpc::SearchParam request; - ::milvus::grpc::TopKQueryResultList response; + ::milvus::grpc::TopKQueryResult response; //test null input handler->Search(&context, nullptr, &response); @@ -241,9 +239,7 @@ TEST_F(RpcHandlerTest, SEARCH_TEST) { ::milvus::grpc::InsertParam insert_param; for (auto& record : record_array) { ::milvus::grpc::RowRecord* grpc_record = insert_param.add_row_record_array(); - for (size_t i = 0; i < record.size(); i++) { - grpc_record->add_vector_data(record[i]); - } + grpc_record->add_vector_data(record.begin(), record.end()); } //insert vectors insert_param.set_table_name(TABLE_NAME); @@ -254,9 +250,7 @@ TEST_F(RpcHandlerTest, SEARCH_TEST) { BuildVectors(0, 10, record_array); for (auto& record : record_array) { ::milvus::grpc::RowRecord* row_record = request.add_query_record_array(); - for (auto& rec : record) { - row_record->add_vector_data(rec); - } + row_record->add_vector_data(record.begin(), record.end()); } handler->Search(&context, &request, &response); @@ -325,9 +319,7 @@ TEST_F(RpcHandlerTest, TABLES_TEST) { for (auto& record : record_array) { ::milvus::grpc::RowRecord* grpc_record = request.add_row_record_array(); - for (size_t i = 0; i < record.size(); i++) { - grpc_record->add_vector_data(record[i]); - } + grpc_record->add_vector_data(record.begin(), record.end()); } //test vector_id size not equal to row record size vector_ids.clear_vector_id_array(); @@ -342,9 +334,7 @@ TEST_F(RpcHandlerTest, TABLES_TEST) { vector_ids.clear_vector_id_array(); for (uint64_t i = 0; i < 10; ++i) { ::milvus::grpc::RowRecord* grpc_record = request.add_row_record_array(); - for (size_t j = 0; j < 10; j++) { - grpc_record->add_vector_data(record_array[i][j]); - } + grpc_record->add_vector_data(record_array[i].begin(), record_array[i].end()); } handler->Insert(&context, &request, &vector_ids); From dc4e7d4bd08f677880696fbd43ff37c0991257f8 Mon Sep 17 00:00:00 2001 From: JinHai-CN Date: Thu, 7 Nov 2019 15:45:14 +0800 Subject: [PATCH 025/103] Update README --- CHANGELOG.md | 2 +- README.md | 4 ++-- README_CN.md | 4 ++-- ci/jenkins/Jenkinsfile | 2 +- tests/milvus_python_test/test_ping.py | 2 +- 5 files changed, 7 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ffe0e3fc09..256e288360 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ Please mark all change in change log and use the ticket from JIRA. -# Milvus 0.5.2 (TODO) +# Milvus 0.5.2 (2019-11-07) ## Bug - \#194 - Search faild: message="Table file doesn't exist" diff --git a/README.md b/README.md index 311d0d6843..66417b8e84 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ ![LICENSE](https://img.shields.io/badge/license-Apache--2.0-brightgreen) ![Language](https://img.shields.io/badge/language-C%2B%2B-blue) [![codebeat badge](https://codebeat.co/badges/e030a4f6-b126-4475-a938-4723d54ec3a7?style=plastic)](https://codebeat.co/projects/github-com-jinhai-cn-milvus-master) -![Release](https://img.shields.io/badge/release-v0.5.1-yellowgreen) +![Release](https://img.shields.io/badge/release-v0.5.2-yellowgreen) ![Release_date](https://img.shields.io/badge/release%20date-November-yellowgreen) [中文版](README_CN.md) @@ -18,7 +18,7 @@ For more detailed introduction of Milvus and its architecture, see [Milvus overv Milvus provides stable [Python](https://github.com/milvus-io/pymilvus), [Java](https://github.com/milvus-io/milvus-sdk-java) and C++ APIs. -Keep up-to-date with newest releases and latest updates by reading Milvus [release notes](https://www.milvus.io/docs/en/release/v0.5.0/). +Keep up-to-date with newest releases and latest updates by reading Milvus [release notes](https://www.milvus.io/docs/en/release/v0.5.2/). ## Get started diff --git a/README_CN.md b/README_CN.md index 5dad64af9b..1def87452b 100644 --- a/README_CN.md +++ b/README_CN.md @@ -4,7 +4,7 @@ ![Language](https://img.shields.io/badge/language-C%2B%2B-blue) [![codebeat badge](https://codebeat.co/badges/e030a4f6-b126-4475-a938-4723d54ec3a7?style=plastic)](https://codebeat.co/projects/github-com-jinhai-cn-milvus-master) -![Release](https://img.shields.io/badge/release-v0.5.0-orange) +![Release](https://img.shields.io/badge/release-v0.5.2-orange) ![Release_date](https://img.shields.io/badge/release_date-October-yellowgreen) - [Slack 频道](https://join.slack.com/t/milvusio/shared_invite/enQtNzY1OTQ0NDI3NjMzLWNmYmM1NmNjOTQ5MGI5NDhhYmRhMGU5M2NhNzhhMDMzY2MzNDdlYjM5ODQ5MmE3ODFlYzU3YjJkNmVlNDQ2ZTk) @@ -22,7 +22,7 @@ Milvus 是一款开源的、针对海量特征向量的相似性搜索引擎。 Milvus 提供稳定的 Python、Java 以及 C++ 的 API 接口。 -通过 [版本发布说明](https://milvus.io/docs/zh-CN/release/v0.5.0/) 获取最新发行版本的 Milvus。 +通过 [版本发布说明](https://milvus.io/docs/zh-CN/release/v0.5.2/) 获取最新发行版本的 Milvus。 - 异构众核 diff --git a/ci/jenkins/Jenkinsfile b/ci/jenkins/Jenkinsfile index 40d9686415..957b1d3975 100644 --- a/ci/jenkins/Jenkinsfile +++ b/ci/jenkins/Jenkinsfile @@ -2,7 +2,7 @@ String cron_timezone = "TZ=Asia/Shanghai" String cron_string = BRANCH_NAME == "master" ? "H 0 * * * " : "" -cron_string = BRANCH_NAME == "0.5.1" ? "H 1 * * * " : cron_string +cron_string = BRANCH_NAME == "0.5.2" ? "H 1 * * * " : cron_string pipeline { agent none diff --git a/tests/milvus_python_test/test_ping.py b/tests/milvus_python_test/test_ping.py index d63ab93f11..9b9d3b2b25 100644 --- a/tests/milvus_python_test/test_ping.py +++ b/tests/milvus_python_test/test_ping.py @@ -1,7 +1,7 @@ import logging import pytest -__version__ = '0.5.1' +__version__ = '0.5.2' class TestPing: From 77e2be1dcdc9db03d758957002f2dbc80eac6e30 Mon Sep 17 00:00:00 2001 From: JinHai-CN Date: Thu, 7 Nov 2019 16:01:45 +0800 Subject: [PATCH 026/103] Update version info --- README_CN.md | 2 +- README_JP.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README_CN.md b/README_CN.md index ab35586cde..d5de0b1cd6 100644 --- a/README_CN.md +++ b/README_CN.md @@ -17,7 +17,7 @@ Milvus 是一款开源的、针对海量特征向量的相似性搜索引擎。 Milvus 提供稳定的 [Python](https://github.com/milvus-io/pymilvus)、[Java](https://github.com/milvus-io/milvus-sdk-java) 以及 C++ 的 API 接口。 -通过 [版本发布说明](https://milvus.io/docs/zh-CN/release/v0.5.1/) 获取最新版本的功能和更新。 +通过 [版本发布说明](https://milvus.io/docs/zh-CN/release/v0.5.2/) 获取最新版本的功能和更新。 ## 开始使用 Milvus diff --git a/README_JP.md b/README_JP.md index a99c37a845..997b329283 100644 --- a/README_JP.md +++ b/README_JP.md @@ -5,7 +5,7 @@ ![LICENSE](https://img.shields.io/badge/license-Apache--2.0-brightgreen) ![Language](https://img.shields.io/badge/language-C%2B%2B-blue) [![codebeat badge](https://codebeat.co/badges/e030a4f6-b126-4475-a938-4723d54ec3a7?style=plastic)](https://codebeat.co/projects/github-com-jinhai-cn-milvus-master) -![Release](https://img.shields.io/badge/release-v0.5.1-yellowgreen) +![Release](https://img.shields.io/badge/release-v0.5.2-yellowgreen) ![Release_date](https://img.shields.io/badge/release%20date-November-yellowgreen) @@ -17,7 +17,7 @@ Milvus は大規模な特徴ベクトルにむかう類似性検索エンジン Milvus は安定的な Python、Java 又は C++ APIsを提供します。 -Milvus [リリースノート](https://milvus.io/docs/en/release/v0.5.1/)を読んで最新バージョンや更新情報を手に入れます。 +Milvus [リリースノート](https://milvus.io/docs/en/release/v0.5.2/)を読んで最新バージョンや更新情報を手に入れます。 - 不均質な計算アーキテクチャー From 174e16ad26a8d572e96df19b8d3d61302d9061d3 Mon Sep 17 00:00:00 2001 From: fishpenguin Date: Thu, 7 Nov 2019 17:51:01 +0800 Subject: [PATCH 027/103] fix add_vector_data bug in clientproxy --- core/src/grpc/README.md | 4 ++-- core/src/sdk/grpc/ClientProxy.cpp | 5 ++--- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/core/src/grpc/README.md b/core/src/grpc/README.md index d3056fc094..4f66d69f7f 100644 --- a/core/src/grpc/README.md +++ b/core/src/grpc/README.md @@ -1,4 +1,4 @@ We manually add two APIs in "milvus.pd.h": -add_vector_data -add_row_id_array + add_vector_data() + add_row_id_array() If proto files need be generated again, remember to re-add above APIs. \ No newline at end of file diff --git a/core/src/sdk/grpc/ClientProxy.cpp b/core/src/sdk/grpc/ClientProxy.cpp index 04878acf7f..bc572ed0a2 100644 --- a/core/src/sdk/grpc/ClientProxy.cpp +++ b/core/src/sdk/grpc/ClientProxy.cpp @@ -188,14 +188,13 @@ ClientProxy::Insert(const std::string& table_name, const std::vector& for (auto& record : record_array) { ::milvus::grpc::RowRecord* grpc_record = insert_param.add_row_record_array(); - for (size_t i = 0; i < record.data.size(); i++) { - grpc_record->add_vector_data(record.data.begin(), record.data.end()); - } + grpc_record->add_vector_data(record.data.begin(), record.data.end()); } // Single thread ::milvus::grpc::VectorIds vector_ids; if (!id_array.empty()) { + /* set user's ids */ insert_param.add_row_id_array(id_array.begin(), id_array.end()); client_ptr_->Insert(vector_ids, insert_param, status); } else { From 32b609aaf359c0226ebcf710e3c6bf5c377bbd36 Mon Sep 17 00:00:00 2001 From: JinHai-CN Date: Fri, 8 Nov 2019 00:48:05 +0800 Subject: [PATCH 028/103] Update to new local faiss --- core/src/index/cmake/ThirdPartyPackagesCore.cmake | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/core/src/index/cmake/ThirdPartyPackagesCore.cmake b/core/src/index/cmake/ThirdPartyPackagesCore.cmake index 5baf0c5a4b..584283b49b 100644 --- a/core/src/index/cmake/ThirdPartyPackagesCore.cmake +++ b/core/src/index/cmake/ThirdPartyPackagesCore.cmake @@ -247,7 +247,8 @@ if(CUSTOMIZATION) # set(FAISS_MD5 "87fdd86351ffcaf3f80dc26ade63c44b") # commit-id 841a156e67e8e22cd8088e1b58c00afbf2efc30b branch-0.2.1 # set(FAISS_MD5 "f3b2ce3364c3fa7febd3aa7fdd0fe380") # commit-id 694e03458e6b69ce8a62502f71f69a614af5af8f branch-0.3.0 # set(FAISS_MD5 "bb30722c22390ce5f6759ccb216c1b2a") # commit-id d324db297475286afe107847c7fb7a0f9dc7e90e branch-0.3.0 - set(FAISS_MD5 "2293cdb209c3718e3b19f3edae8b32b3") # commit-id a13c1205dc52977a9ad3b33a14efa958604a8bff branch-0.3.0 + # set(FAISS_MD5 "2293cdb209c3718e3b19f3edae8b32b3") # commit-id a13c1205dc52977a9ad3b33a14efa958604a8bff branch-0.3.0 + set(FAISS_MD5 "3632192f06b70e5f3fac8e2f08ced503") # commit-id fbd5b7566466c5665bb646f2b2f911c8544090f5 branch-0.3.0 endif() else() set(FAISS_SOURCE_URL "https://github.com/JinHai-CN/faiss/archive/1.6.0.tar.gz") From c4f56ddd366ffefa0427ebef535f837fe04936fd Mon Sep 17 00:00:00 2001 From: quicksilver Date: Fri, 8 Nov 2019 10:46:25 +0800 Subject: [PATCH 029/103] Delete unittest binary on jenkins ci packaging stage --- ci/jenkins/Jenkinsfile | 2 +- ci/jenkins/scripts/packaging.sh | 13 +++++++++++++ .../step/{package.groovy => packaging.groovy} | 4 +--- 3 files changed, 15 insertions(+), 4 deletions(-) create mode 100755 ci/jenkins/scripts/packaging.sh rename ci/jenkins/step/{package.groovy => packaging.groovy} (71%) diff --git a/ci/jenkins/Jenkinsfile b/ci/jenkins/Jenkinsfile index 40d9686415..c2920358af 100644 --- a/ci/jenkins/Jenkinsfile +++ b/ci/jenkins/Jenkinsfile @@ -75,7 +75,7 @@ pipeline { steps { container('milvus-build-env') { script { - load "${env.WORKSPACE}/ci/jenkins/step/package.groovy" + load "${env.WORKSPACE}/ci/jenkins/step/packaging.groovy" } } } diff --git a/ci/jenkins/scripts/packaging.sh b/ci/jenkins/scripts/packaging.sh new file mode 100755 index 0000000000..7687cbf33f --- /dev/null +++ b/ci/jenkins/scripts/packaging.sh @@ -0,0 +1,13 @@ +#!/usr/bin/env bash + +set -ex + +pip3 install -r requirements.txt +./yaml_processor.py merge -f /opt/milvus/conf/server_config.yaml -m ../yaml/update_server_config.yaml -i && \ +rm /opt/milvus/conf/server_config.yaml.bak + +if [ -d "/opt/milvus/unittest" ]; then + rm -rf "/opt/milvus/unittest" +fi + +tar -zcvf ./${PROJECT_NAME}-${PACKAGE_VERSION}.tar.gz -C /opt/ milvus \ No newline at end of file diff --git a/ci/jenkins/step/package.groovy b/ci/jenkins/step/packaging.groovy similarity index 71% rename from ci/jenkins/step/package.groovy rename to ci/jenkins/step/packaging.groovy index c07dcd9d88..2b8b333213 100644 --- a/ci/jenkins/step/package.groovy +++ b/ci/jenkins/step/packaging.groovy @@ -1,9 +1,7 @@ timeout(time: 5, unit: 'MINUTES') { dir ("ci/jenkins/scripts") { - sh "pip3 install -r requirements.txt" - sh "./yaml_processor.py merge -f /opt/milvus/conf/server_config.yaml -m ../yaml/update_server_config.yaml -i && rm /opt/milvus/conf/server_config.yaml.bak" + sh "./packaging.sh" } - sh "tar -zcvf ./${PROJECT_NAME}-${PACKAGE_VERSION}.tar.gz -C /opt/ milvus" withCredentials([usernamePassword(credentialsId: "${params.JFROG_CREDENTIALS_ID}", usernameVariable: 'JFROG_USERNAME', passwordVariable: 'JFROG_PASSWORD')]) { def uploadStatus = sh(returnStatus: true, script: "curl -u${JFROG_USERNAME}:${JFROG_PASSWORD} -T ./${PROJECT_NAME}-${PACKAGE_VERSION}.tar.gz ${params.JFROG_ARTFACTORY_URL}/milvus/package/${PROJECT_NAME}-${PACKAGE_VERSION}.tar.gz") if (uploadStatus != 0) { From 9194bdfb0bc1eb6d2699117c98b9dc3fcd400fcc Mon Sep 17 00:00:00 2001 From: quicksilver Date: Fri, 8 Nov 2019 10:54:44 +0800 Subject: [PATCH 030/103] Delete unittest binary on jenkins ci packaging stage --- ci/jenkinsfile/packaged_milvus.groovy | 3 +++ 1 file changed, 3 insertions(+) diff --git a/ci/jenkinsfile/packaged_milvus.groovy b/ci/jenkinsfile/packaged_milvus.groovy index 1d30e21910..f185cb4c5f 100644 --- a/ci/jenkinsfile/packaged_milvus.groovy +++ b/ci/jenkinsfile/packaged_milvus.groovy @@ -5,6 +5,9 @@ container('milvus-build-env') { gitlabCommitStatus(name: 'Packaged Engine') { if (fileExists('milvus')) { try { + if (fileExists('milvus/unittest')) { + sh "rm -rf ./milvus/unittest" + } sh "tar -zcvf ./${PROJECT_NAME}-engine-${PACKAGE_VERSION}.tar.gz ./milvus" def fileTransfer = load "${env.WORKSPACE}/ci/function/file_transfer.groovy" fileTransfer.FileTransfer("${PROJECT_NAME}-engine-${PACKAGE_VERSION}.tar.gz", "${PROJECT_NAME}/engine/${JOB_NAME}-${BUILD_ID}", 'nas storage') From fceb3ba714a985ac18ad0c49ffdae82d850a0a49 Mon Sep 17 00:00:00 2001 From: fishpenguin Date: Fri, 8 Nov 2019 11:03:27 +0800 Subject: [PATCH 031/103] Decrease test_rpc run time --- core/unittest/server/test_rpc.cpp | 1 - 1 file changed, 1 deletion(-) diff --git a/core/unittest/server/test_rpc.cpp b/core/unittest/server/test_rpc.cpp index 01d41c1789..3a189d2bf7 100644 --- a/core/unittest/server/test_rpc.cpp +++ b/core/unittest/server/test_rpc.cpp @@ -245,7 +245,6 @@ TEST_F(RpcHandlerTest, SEARCH_TEST) { insert_param.set_table_name(TABLE_NAME); ::milvus::grpc::VectorIds vector_ids; handler->Insert(&context, &insert_param, &vector_ids); - sleep(7); BuildVectors(0, 10, record_array); for (auto& record : record_array) { From c0a009d52888537176ed3ee933eb80a17945fd38 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Fri, 8 Nov 2019 11:17:18 +0800 Subject: [PATCH 032/103] Delete unittest binary on jenkins ci packaging stage --- ci/jenkins/step/packaging.groovy | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/ci/jenkins/step/packaging.groovy b/ci/jenkins/step/packaging.groovy index 2b8b333213..ce1653bb09 100644 --- a/ci/jenkins/step/packaging.groovy +++ b/ci/jenkins/step/packaging.groovy @@ -1,11 +1,11 @@ timeout(time: 5, unit: 'MINUTES') { dir ("ci/jenkins/scripts") { sh "./packaging.sh" - } - withCredentials([usernamePassword(credentialsId: "${params.JFROG_CREDENTIALS_ID}", usernameVariable: 'JFROG_USERNAME', passwordVariable: 'JFROG_PASSWORD')]) { - def uploadStatus = sh(returnStatus: true, script: "curl -u${JFROG_USERNAME}:${JFROG_PASSWORD} -T ./${PROJECT_NAME}-${PACKAGE_VERSION}.tar.gz ${params.JFROG_ARTFACTORY_URL}/milvus/package/${PROJECT_NAME}-${PACKAGE_VERSION}.tar.gz") - if (uploadStatus != 0) { - error("\" ${PROJECT_NAME}-${PACKAGE_VERSION}.tar.gz \" upload to \" ${params.JFROG_ARTFACTORY_URL}/milvus/package/${PROJECT_NAME}-${PACKAGE_VERSION}.tar.gz \" failed!") + withCredentials([usernamePassword(credentialsId: "${params.JFROG_CREDENTIALS_ID}", usernameVariable: 'JFROG_USERNAME', passwordVariable: 'JFROG_PASSWORD')]) { + def uploadStatus = sh(returnStatus: true, script: "curl -u${JFROG_USERNAME}:${JFROG_PASSWORD} -T ./${PROJECT_NAME}-${PACKAGE_VERSION}.tar.gz ${params.JFROG_ARTFACTORY_URL}/milvus/package/${PROJECT_NAME}-${PACKAGE_VERSION}.tar.gz") + if (uploadStatus != 0) { + error("\" ${PROJECT_NAME}-${PACKAGE_VERSION}.tar.gz \" upload to \" ${params.JFROG_ARTFACTORY_URL}/milvus/package/${PROJECT_NAME}-${PACKAGE_VERSION}.tar.gz \" failed!") + } } } } From a76d223a255392da6a03019ec3f1b489c05331b1 Mon Sep 17 00:00:00 2001 From: groot Date: Fri, 8 Nov 2019 12:11:11 +0800 Subject: [PATCH 033/103] #246 Exclude src/external folder from code coverage for jenkin ci --- ci/jenkins/scripts/coverage.sh | 3 +-- core/coverage.sh | 2 -- 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/ci/jenkins/scripts/coverage.sh b/ci/jenkins/scripts/coverage.sh index 07ab210d2f..5c9d010d46 100755 --- a/ci/jenkins/scripts/coverage.sh +++ b/ci/jenkins/scripts/coverage.sh @@ -132,8 +132,7 @@ ${LCOV_CMD} -r "${FILE_INFO_OUTPUT}" -o "${FILE_INFO_OUTPUT_NEW}" \ "*/src/server/Server.cpp" \ "*/src/server/DBWrapper.cpp" \ "*/src/server/grpc_impl/GrpcServer.cpp" \ - "*/src/external/easyloggingpp/easylogging++.h" \ - "*/src/external/easyloggingpp/easylogging++.cc" + "*/src/external/*" if [ $? -ne 0 ]; then echo "gen ${FILE_INFO_OUTPUT_NEW} failed" diff --git a/core/coverage.sh b/core/coverage.sh index 2cb0861de4..9011e290e5 100755 --- a/core/coverage.sh +++ b/core/coverage.sh @@ -122,8 +122,6 @@ ${LCOV_CMD} -r "${FILE_INFO_OUTPUT}" -o "${FILE_INFO_OUTPUT_NEW}" \ "*/src/server/Server.cpp" \ "*/src/server/DBWrapper.cpp" \ "*/src/server/grpc_impl/GrpcServer.cpp" \ - "*/easylogging++.h" \ - "*/easylogging++.cc" \ "*/src/external/*" if [ $? -ne 0 ]; then From 258fd600fae81b488d13b3f1a14fc8a995a2a73e Mon Sep 17 00:00:00 2001 From: groot Date: Fri, 8 Nov 2019 12:19:44 +0800 Subject: [PATCH 034/103] update changelog --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 68bb32f539..906799bcfe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,7 @@ Please mark all change in change log and use the ticket from JIRA. # Milvus 0.6.0 (TODO) ## Bug +- \#246 - Exclude src/external folder from code coverage for jenkin ci ## Feature - \#12 - Pure CPU version for Milvus From fdb816756639d706e97d90c79e47df3b2ac07178 Mon Sep 17 00:00:00 2001 From: fishpenguin Date: Fri, 8 Nov 2019 13:12:14 +0800 Subject: [PATCH 035/103] add README.md for grpc --- core/src/grpc/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/core/src/grpc/README.md b/core/src/grpc/README.md index 4f66d69f7f..781d658842 100644 --- a/core/src/grpc/README.md +++ b/core/src/grpc/README.md @@ -1,4 +1,4 @@ -We manually add two APIs in "milvus.pd.h": +We manually change two APIs in "milvus.pd.h": add_vector_data() add_row_id_array() -If proto files need be generated again, remember to re-add above APIs. \ No newline at end of file +If proto files need be generated again, remember to re-change above APIs. \ No newline at end of file From d9369389f96bdc1286b5a161cc136737aee074e6 Mon Sep 17 00:00:00 2001 From: fishpenguin Date: Fri, 8 Nov 2019 14:17:05 +0800 Subject: [PATCH 036/103] change branche to 0.5.3 in ci/jenkins --- ci/jenkins/step/deploySingle2Dev.groovy | 2 +- ci/jenkins/step/singleDevNightlyTest.groovy | 2 +- ci/jenkins/step/singleDevTest.groovy | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/ci/jenkins/step/deploySingle2Dev.groovy b/ci/jenkins/step/deploySingle2Dev.groovy index f4964df5e2..69be2dfe31 100644 --- a/ci/jenkins/step/deploySingle2Dev.groovy +++ b/ci/jenkins/step/deploySingle2Dev.groovy @@ -1,7 +1,7 @@ sh 'helm init --client-only --skip-refresh --stable-repo-url https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts' sh 'helm repo update' dir ('milvus-helm') { - checkout([$class: 'GitSCM', branches: [[name: "0.5.0"]], userRemoteConfigs: [[url: "https://github.com/milvus-io/milvus-helm.git", name: 'origin', refspec: "+refs/heads/0.5.0:refs/remotes/origin/0.5.0"]]]) + checkout([$class: 'GitSCM', branches: [[name: "0.5.3"]], userRemoteConfigs: [[url: "https://github.com/milvus-io/milvus-helm.git", name: 'origin', refspec: "+refs/heads/0.5.3:refs/remotes/origin/0.5.3"]]]) dir ("milvus-gpu") { sh "helm install --wait --timeout 300 --set engine.image.tag=${DOCKER_VERSION} --set expose.type=clusterIP --name ${env.PIPELINE_NAME}-${env.BUILD_NUMBER}-single-gpu -f ci/db_backend/sqlite_values.yaml -f ci/filebeat/values.yaml --namespace milvus ." } diff --git a/ci/jenkins/step/singleDevNightlyTest.groovy b/ci/jenkins/step/singleDevNightlyTest.groovy index 9aeab2eb4e..201132d2c3 100644 --- a/ci/jenkins/step/singleDevNightlyTest.groovy +++ b/ci/jenkins/step/singleDevNightlyTest.groovy @@ -8,7 +8,7 @@ timeout(time: 90, unit: 'MINUTES') { if (!fileExists('milvus-helm')) { dir ("milvus-helm") { - checkout([$class: 'GitSCM', branches: [[name: "0.5.0"]], userRemoteConfigs: [[url: "https://github.com/milvus-io/milvus-helm.git", name: 'origin', refspec: "+refs/heads/0.5.0:refs/remotes/origin/0.5.0"]]]) + checkout([$class: 'GitSCM', branches: [[name: "0.5.3"]], userRemoteConfigs: [[url: "https://github.com/milvus-io/milvus-helm.git", name: 'origin', refspec: "+refs/heads/0.5.3:refs/remotes/origin/0.5.3"]]]) } } dir ("milvus-helm") { diff --git a/ci/jenkins/step/singleDevTest.groovy b/ci/jenkins/step/singleDevTest.groovy index 86e6f126d9..7f3ce392b1 100644 --- a/ci/jenkins/step/singleDevTest.groovy +++ b/ci/jenkins/step/singleDevTest.groovy @@ -10,7 +10,7 @@ timeout(time: 60, unit: 'MINUTES') { // if (!fileExists('milvus-helm')) { // dir ("milvus-helm") { - // checkout([$class: 'GitSCM', branches: [[name: "0.5.0"]], userRemoteConfigs: [[url: "https://github.com/milvus-io/milvus-helm.git", name: 'origin', refspec: "+refs/heads/0.5.0:refs/remotes/origin/0.5.0"]]]) + // checkout([$class: 'GitSCM', branches: [[name: "0.5.3"]], userRemoteConfigs: [[url: "https://github.com/milvus-io/milvus-helm.git", name: 'origin', refspec: "+refs/heads/0.5.3:refs/remotes/origin/0.5.3"]]]) // } // } // dir ("milvus-helm") { From 87ef2f535a3b86263d426d8ae967ca7be8c8f8f0 Mon Sep 17 00:00:00 2001 From: fishpenguin Date: Fri, 8 Nov 2019 15:55:31 +0800 Subject: [PATCH 037/103] change server version to 0.5.3 in CMakeList.txt --- core/CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/CMakeLists.txt b/core/CMakeLists.txt index 7acf3062fa..822a591c07 100644 --- a/core/CMakeLists.txt +++ b/core/CMakeLists.txt @@ -71,7 +71,7 @@ if(MILVUS_VERSION_MAJOR STREQUAL "" OR MILVUS_VERSION_MINOR STREQUAL "" OR MILVUS_VERSION_PATCH STREQUAL "") message(WARNING "Failed to determine Milvus version from git branch name") - set(MILVUS_VERSION "0.5.2") + set(MILVUS_VERSION "0.5.3") endif() message(STATUS "Build version = ${MILVUS_VERSION}") From 4284e3cdc92dc6f767b787367a91df486abeb9ae Mon Sep 17 00:00:00 2001 From: zhenwu Date: Fri, 8 Nov 2019 16:45:24 +0800 Subject: [PATCH 038/103] Update test case --- tests/milvus_python_test/test_connect.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/milvus_python_test/test_connect.py b/tests/milvus_python_test/test_connect.py index 96ce1d3bdf..7d4f3719c6 100644 --- a/tests/milvus_python_test/test_connect.py +++ b/tests/milvus_python_test/test_connect.py @@ -273,7 +273,8 @@ class TestConnect: milvus = Milvus() uri_value = "tcp://%s:%s" % (args["ip"], args["port"]) with pytest.raises(Exception) as e: - milvus.connect(host=args["ip"], port=19540, uri=uri_value, timeout=1) + res = milvus.connect(host=args["ip"], port=39540, uri=uri_value, timeout=1) + logger.getLogger().info(res) assert not milvus.connected() def _test_add_vector_and_disconnect_concurrently(self): From 105b98499b610fdd68580f5c4ddd3bde9a71eef2 Mon Sep 17 00:00:00 2001 From: fishpenguin Date: Fri, 8 Nov 2019 18:52:56 +0800 Subject: [PATCH 039/103] #258 - Bytes type in proto cause big-endian/little-endian problem --- CHANGELOG.md | 1 + core/src/grpc/README.md | 2 + core/src/grpc/gen-milvus/milvus.pb.cc | 355 +++++++++--------- core/src/grpc/gen-milvus/milvus.pb.h | 259 ++++++------- core/src/grpc/milvus.proto | 7 +- .../examples/grpcsimple/src/ClientTest.cpp | 4 +- core/src/sdk/grpc/ClientProxy.cpp | 12 +- core/src/sdk/include/MilvusApi.h | 1 - core/src/server/grpc_impl/GrpcRequestTask.cpp | 19 +- 9 files changed, 293 insertions(+), 367 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index aad647ed7b..5009c3c32d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ Please mark all change in change log and use the ticket from JIRA. # Milvus 0.5.3 (TODO) ## Bug +- \#258 - Bytes type in proto cause big-endian/little-endian problem ## Feature diff --git a/core/src/grpc/README.md b/core/src/grpc/README.md index 781d658842..44c4e90841 100644 --- a/core/src/grpc/README.md +++ b/core/src/grpc/README.md @@ -1,4 +1,6 @@ We manually change two APIs in "milvus.pd.h": add_vector_data() add_row_id_array() + add_ids() + add_distances() If proto files need be generated again, remember to re-change above APIs. \ No newline at end of file diff --git a/core/src/grpc/gen-milvus/milvus.pb.cc b/core/src/grpc/gen-milvus/milvus.pb.cc index 6cdee4a2e6..753e491663 100644 --- a/core/src/grpc/gen-milvus/milvus.pb.cc +++ b/core/src/grpc/gen-milvus/milvus.pb.cc @@ -423,10 +423,9 @@ const ::PROTOBUF_NAMESPACE_ID::uint32 TableStruct_milvus_2eproto::offsets[] PROT ~0u, // no _oneof_case_ ~0u, // no _weak_field_map_ PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, status_), - PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, nq_), - PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, topk_), - PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, ids_binary_), - PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, distances_binary_), + PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, row_num_), + PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, ids_), + PROTOBUF_FIELD_OFFSET(::milvus::grpc::TopKQueryResult, distances_), ~0u, // no _has_bits_ PROTOBUF_FIELD_OFFSET(::milvus::grpc::StringReply, _internal_metadata_), ~0u, // no _extensions_ @@ -488,13 +487,13 @@ static const ::PROTOBUF_NAMESPACE_ID::internal::MigrationSchema schemas[] PROTOB { 51, -1, sizeof(::milvus::grpc::SearchParam)}, { 61, -1, sizeof(::milvus::grpc::SearchInFilesParam)}, { 68, -1, sizeof(::milvus::grpc::TopKQueryResult)}, - { 78, -1, sizeof(::milvus::grpc::StringReply)}, - { 85, -1, sizeof(::milvus::grpc::BoolReply)}, - { 92, -1, sizeof(::milvus::grpc::TableRowCount)}, - { 99, -1, sizeof(::milvus::grpc::Command)}, - { 105, -1, sizeof(::milvus::grpc::Index)}, - { 112, -1, sizeof(::milvus::grpc::IndexParam)}, - { 120, -1, sizeof(::milvus::grpc::DeleteByRangeParam)}, + { 77, -1, sizeof(::milvus::grpc::StringReply)}, + { 84, -1, sizeof(::milvus::grpc::BoolReply)}, + { 91, -1, sizeof(::milvus::grpc::TableRowCount)}, + { 98, -1, sizeof(::milvus::grpc::Command)}, + { 104, -1, sizeof(::milvus::grpc::Index)}, + { 111, -1, sizeof(::milvus::grpc::IndexParam)}, + { 119, -1, sizeof(::milvus::grpc::DeleteByRangeParam)}, }; static ::PROTOBUF_NAMESPACE_ID::Message const * const file_default_instances[] = { @@ -538,47 +537,46 @@ const char descriptor_table_protodef_milvus_2eproto[] PROTOBUF_SECTION_VARIABLE( "c.Range\022\014\n\004topk\030\004 \001(\003\022\016\n\006nprobe\030\005 \001(\003\"[\n" "\022SearchInFilesParam\022\025\n\rfile_id_array\030\001 \003" "(\t\022.\n\014search_param\030\002 \001(\0132\030.milvus.grpc.S" - "earchParam\"~\n\017TopKQueryResult\022#\n\006status\030" - "\001 \001(\0132\023.milvus.grpc.Status\022\n\n\002nq\030\002 \001(\003\022\014" - "\n\004topk\030\003 \001(\003\022\022\n\nids_binary\030\004 \001(\014\022\030\n\020dist" - "ances_binary\030\005 \001(\014\"H\n\013StringReply\022#\n\006sta" - "tus\030\001 \001(\0132\023.milvus.grpc.Status\022\024\n\014string" - "_reply\030\002 \001(\t\"D\n\tBoolReply\022#\n\006status\030\001 \001(" - "\0132\023.milvus.grpc.Status\022\022\n\nbool_reply\030\002 \001" - "(\010\"M\n\rTableRowCount\022#\n\006status\030\001 \001(\0132\023.mi" - "lvus.grpc.Status\022\027\n\017table_row_count\030\002 \001(" - "\003\"\026\n\007Command\022\013\n\003cmd\030\001 \001(\t\"*\n\005Index\022\022\n\nin" - "dex_type\030\001 \001(\005\022\r\n\005nlist\030\002 \001(\005\"h\n\nIndexPa" - "ram\022#\n\006status\030\001 \001(\0132\023.milvus.grpc.Status" - "\022\022\n\ntable_name\030\002 \001(\t\022!\n\005index\030\003 \001(\0132\022.mi" - "lvus.grpc.Index\"K\n\022DeleteByRangeParam\022!\n" - "\005range\030\001 \001(\0132\022.milvus.grpc.Range\022\022\n\ntabl" - "e_name\030\002 \001(\t2\350\007\n\rMilvusService\022>\n\013Create" - "Table\022\030.milvus.grpc.TableSchema\032\023.milvus" - ".grpc.Status\"\000\022<\n\010HasTable\022\026.milvus.grpc" - ".TableName\032\026.milvus.grpc.BoolReply\"\000\022:\n\t" - "DropTable\022\026.milvus.grpc.TableName\032\023.milv" - "us.grpc.Status\"\000\022=\n\013CreateIndex\022\027.milvus" - ".grpc.IndexParam\032\023.milvus.grpc.Status\"\000\022" - "<\n\006Insert\022\030.milvus.grpc.InsertParam\032\026.mi" - "lvus.grpc.VectorIds\"\000\022B\n\006Search\022\030.milvus" - ".grpc.SearchParam\032\034.milvus.grpc.TopKQuer" - "yResult\"\000\022P\n\rSearchInFiles\022\037.milvus.grpc" - ".SearchInFilesParam\032\034.milvus.grpc.TopKQu" - "eryResult\"\000\022C\n\rDescribeTable\022\026.milvus.gr" - "pc.TableName\032\030.milvus.grpc.TableSchema\"\000" - "\022B\n\nCountTable\022\026.milvus.grpc.TableName\032\032" - ".milvus.grpc.TableRowCount\"\000\022@\n\nShowTabl" - "es\022\024.milvus.grpc.Command\032\032.milvus.grpc.T" - "ableNameList\"\000\0227\n\003Cmd\022\024.milvus.grpc.Comm" - "and\032\030.milvus.grpc.StringReply\"\000\022G\n\rDelet" - "eByRange\022\037.milvus.grpc.DeleteByRangePara" - "m\032\023.milvus.grpc.Status\"\000\022=\n\014PreloadTable" - "\022\026.milvus.grpc.TableName\032\023.milvus.grpc.S" - "tatus\"\000\022B\n\rDescribeIndex\022\026.milvus.grpc.T" - "ableName\032\027.milvus.grpc.IndexParam\"\000\022:\n\tD" - "ropIndex\022\026.milvus.grpc.TableName\032\023.milvu" - "s.grpc.Status\"\000b\006proto3" + "earchParam\"g\n\017TopKQueryResult\022#\n\006status\030" + "\001 \001(\0132\023.milvus.grpc.Status\022\017\n\007row_num\030\002 " + "\001(\003\022\013\n\003ids\030\003 \003(\003\022\021\n\tdistances\030\004 \003(\002\"H\n\013S" + "tringReply\022#\n\006status\030\001 \001(\0132\023.milvus.grpc" + ".Status\022\024\n\014string_reply\030\002 \001(\t\"D\n\tBoolRep" + "ly\022#\n\006status\030\001 \001(\0132\023.milvus.grpc.Status\022" + "\022\n\nbool_reply\030\002 \001(\010\"M\n\rTableRowCount\022#\n\006" + "status\030\001 \001(\0132\023.milvus.grpc.Status\022\027\n\017tab" + "le_row_count\030\002 \001(\003\"\026\n\007Command\022\013\n\003cmd\030\001 \001" + "(\t\"*\n\005Index\022\022\n\nindex_type\030\001 \001(\005\022\r\n\005nlist" + "\030\002 \001(\005\"h\n\nIndexParam\022#\n\006status\030\001 \001(\0132\023.m" + "ilvus.grpc.Status\022\022\n\ntable_name\030\002 \001(\t\022!\n" + "\005index\030\003 \001(\0132\022.milvus.grpc.Index\"K\n\022Dele" + "teByRangeParam\022!\n\005range\030\001 \001(\0132\022.milvus.g" + "rpc.Range\022\022\n\ntable_name\030\002 \001(\t2\350\007\n\rMilvus" + "Service\022>\n\013CreateTable\022\030.milvus.grpc.Tab" + "leSchema\032\023.milvus.grpc.Status\"\000\022<\n\010HasTa" + "ble\022\026.milvus.grpc.TableName\032\026.milvus.grp" + "c.BoolReply\"\000\022:\n\tDropTable\022\026.milvus.grpc" + ".TableName\032\023.milvus.grpc.Status\"\000\022=\n\013Cre" + "ateIndex\022\027.milvus.grpc.IndexParam\032\023.milv" + "us.grpc.Status\"\000\022<\n\006Insert\022\030.milvus.grpc" + ".InsertParam\032\026.milvus.grpc.VectorIds\"\000\022B" + "\n\006Search\022\030.milvus.grpc.SearchParam\032\034.mil" + "vus.grpc.TopKQueryResult\"\000\022P\n\rSearchInFi" + "les\022\037.milvus.grpc.SearchInFilesParam\032\034.m" + "ilvus.grpc.TopKQueryResult\"\000\022C\n\rDescribe" + "Table\022\026.milvus.grpc.TableName\032\030.milvus.g" + "rpc.TableSchema\"\000\022B\n\nCountTable\022\026.milvus" + ".grpc.TableName\032\032.milvus.grpc.TableRowCo" + "unt\"\000\022@\n\nShowTables\022\024.milvus.grpc.Comman" + "d\032\032.milvus.grpc.TableNameList\"\000\0227\n\003Cmd\022\024" + ".milvus.grpc.Command\032\030.milvus.grpc.Strin" + "gReply\"\000\022G\n\rDeleteByRange\022\037.milvus.grpc." + "DeleteByRangeParam\032\023.milvus.grpc.Status\"" + "\000\022=\n\014PreloadTable\022\026.milvus.grpc.TableNam" + "e\032\023.milvus.grpc.Status\"\000\022B\n\rDescribeInde" + "x\022\026.milvus.grpc.TableName\032\027.milvus.grpc." + "IndexParam\"\000\022:\n\tDropIndex\022\026.milvus.grpc." + "TableName\032\023.milvus.grpc.Status\"\000b\006proto3" ; static const ::PROTOBUF_NAMESPACE_ID::internal::DescriptorTable*const descriptor_table_milvus_2eproto_deps[1] = { &::descriptor_table_status_2eproto, @@ -605,7 +603,7 @@ static ::PROTOBUF_NAMESPACE_ID::internal::SCCInfoBase*const descriptor_table_mil static ::PROTOBUF_NAMESPACE_ID::internal::once_flag descriptor_table_milvus_2eproto_once; static bool descriptor_table_milvus_2eproto_initialized = false; const ::PROTOBUF_NAMESPACE_ID::internal::DescriptorTable descriptor_table_milvus_2eproto = { - &descriptor_table_milvus_2eproto_initialized, descriptor_table_protodef_milvus_2eproto, "milvus.proto", 2423, + &descriptor_table_milvus_2eproto_initialized, descriptor_table_protodef_milvus_2eproto, "milvus.proto", 2400, &descriptor_table_milvus_2eproto_once, descriptor_table_milvus_2eproto_sccs, descriptor_table_milvus_2eproto_deps, 17, 1, schemas, file_default_instances, TableStruct_milvus_2eproto::offsets, file_level_metadata_milvus_2eproto, 17, file_level_enum_descriptors_milvus_2eproto, file_level_service_descriptors_milvus_2eproto, @@ -3845,34 +3843,24 @@ TopKQueryResult::TopKQueryResult() } TopKQueryResult::TopKQueryResult(const TopKQueryResult& from) : ::PROTOBUF_NAMESPACE_ID::Message(), - _internal_metadata_(nullptr) { + _internal_metadata_(nullptr), + ids_(from.ids_), + distances_(from.distances_) { _internal_metadata_.MergeFrom(from._internal_metadata_); - ids_binary_.UnsafeSetDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); - if (!from.ids_binary().empty()) { - ids_binary_.AssignWithDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), from.ids_binary_); - } - distances_binary_.UnsafeSetDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); - if (!from.distances_binary().empty()) { - distances_binary_.AssignWithDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), from.distances_binary_); - } if (from.has_status()) { status_ = new ::milvus::grpc::Status(*from.status_); } else { status_ = nullptr; } - ::memcpy(&nq_, &from.nq_, - static_cast(reinterpret_cast(&topk_) - - reinterpret_cast(&nq_)) + sizeof(topk_)); + row_num_ = from.row_num_; // @@protoc_insertion_point(copy_constructor:milvus.grpc.TopKQueryResult) } void TopKQueryResult::SharedCtor() { ::PROTOBUF_NAMESPACE_ID::internal::InitSCC(&scc_info_TopKQueryResult_milvus_2eproto.base); - ids_binary_.UnsafeSetDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); - distances_binary_.UnsafeSetDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); ::memset(&status_, 0, static_cast( - reinterpret_cast(&topk_) - - reinterpret_cast(&status_)) + sizeof(topk_)); + reinterpret_cast(&row_num_) - + reinterpret_cast(&status_)) + sizeof(row_num_)); } TopKQueryResult::~TopKQueryResult() { @@ -3881,8 +3869,6 @@ TopKQueryResult::~TopKQueryResult() { } void TopKQueryResult::SharedDtor() { - ids_binary_.DestroyNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); - distances_binary_.DestroyNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); if (this != internal_default_instance()) delete status_; } @@ -3901,15 +3887,13 @@ void TopKQueryResult::Clear() { // Prevent compiler warnings about cached_has_bits being unused (void) cached_has_bits; - ids_binary_.ClearToEmptyNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); - distances_binary_.ClearToEmptyNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); + ids_.Clear(); + distances_.Clear(); if (GetArenaNoVirtual() == nullptr && status_ != nullptr) { delete status_; } status_ = nullptr; - ::memset(&nq_, 0, static_cast( - reinterpret_cast(&topk_) - - reinterpret_cast(&nq_)) + sizeof(topk_)); + row_num_ = PROTOBUF_LONGLONG(0); _internal_metadata_.Clear(); } @@ -3928,32 +3912,31 @@ const char* TopKQueryResult::_InternalParse(const char* ptr, ::PROTOBUF_NAMESPAC CHK_(ptr); } else goto handle_unusual; continue; - // int64 nq = 2; + // int64 row_num = 2; case 2: if (PROTOBUF_PREDICT_TRUE(static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 16)) { - nq_ = ::PROTOBUF_NAMESPACE_ID::internal::ReadVarint(&ptr); + row_num_ = ::PROTOBUF_NAMESPACE_ID::internal::ReadVarint(&ptr); CHK_(ptr); } else goto handle_unusual; continue; - // int64 topk = 3; + // repeated int64 ids = 3; case 3: - if (PROTOBUF_PREDICT_TRUE(static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 24)) { - topk_ = ::PROTOBUF_NAMESPACE_ID::internal::ReadVarint(&ptr); + if (PROTOBUF_PREDICT_TRUE(static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 26)) { + ptr = ::PROTOBUF_NAMESPACE_ID::internal::PackedInt64Parser(mutable_ids(), ptr, ctx); + CHK_(ptr); + } else if (static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 24) { + add_ids(::PROTOBUF_NAMESPACE_ID::internal::ReadVarint(&ptr)); CHK_(ptr); } else goto handle_unusual; continue; - // bytes ids_binary = 4; + // repeated float distances = 4; case 4: if (PROTOBUF_PREDICT_TRUE(static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 34)) { - ptr = ::PROTOBUF_NAMESPACE_ID::internal::InlineGreedyStringParser(mutable_ids_binary(), ptr, ctx); - CHK_(ptr); - } else goto handle_unusual; - continue; - // bytes distances_binary = 5; - case 5: - if (PROTOBUF_PREDICT_TRUE(static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 42)) { - ptr = ::PROTOBUF_NAMESPACE_ID::internal::InlineGreedyStringParser(mutable_distances_binary(), ptr, ctx); + ptr = ::PROTOBUF_NAMESPACE_ID::internal::PackedFloatParser(mutable_distances(), ptr, ctx); CHK_(ptr); + } else if (static_cast<::PROTOBUF_NAMESPACE_ID::uint8>(tag) == 37) { + add_distances(::PROTOBUF_NAMESPACE_ID::internal::UnalignedLoad(ptr)); + ptr += sizeof(float); } else goto handle_unusual; continue; default: { @@ -3997,48 +3980,45 @@ bool TopKQueryResult::MergePartialFromCodedStream( break; } - // int64 nq = 2; + // int64 row_num = 2; case 2: { if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (16 & 0xFF)) { DO_((::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadPrimitive< ::PROTOBUF_NAMESPACE_ID::int64, ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::TYPE_INT64>( - input, &nq_))); + input, &row_num_))); } else { goto handle_unusual; } break; } - // int64 topk = 3; + // repeated int64 ids = 3; case 3: { - if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (24 & 0xFF)) { - - DO_((::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadPrimitive< + if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (26 & 0xFF)) { + DO_((::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadPackedPrimitive< ::PROTOBUF_NAMESPACE_ID::int64, ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::TYPE_INT64>( - input, &topk_))); + input, this->mutable_ids()))); + } else if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (24 & 0xFF)) { + DO_((::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadRepeatedPrimitiveNoInline< + ::PROTOBUF_NAMESPACE_ID::int64, ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::TYPE_INT64>( + 1, 26u, input, this->mutable_ids()))); } else { goto handle_unusual; } break; } - // bytes ids_binary = 4; + // repeated float distances = 4; case 4: { if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (34 & 0xFF)) { - DO_(::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadBytes( - input, this->mutable_ids_binary())); - } else { - goto handle_unusual; - } - break; - } - - // bytes distances_binary = 5; - case 5: { - if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (42 & 0xFF)) { - DO_(::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadBytes( - input, this->mutable_distances_binary())); + DO_((::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadPackedPrimitive< + float, ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::TYPE_FLOAT>( + input, this->mutable_distances()))); + } else if (static_cast< ::PROTOBUF_NAMESPACE_ID::uint8>(tag) == (37 & 0xFF)) { + DO_((::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::ReadRepeatedPrimitiveNoInline< + float, ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::TYPE_FLOAT>( + 1, 34u, input, this->mutable_distances()))); } else { goto handle_unusual; } @@ -4078,26 +4058,29 @@ void TopKQueryResult::SerializeWithCachedSizes( 1, _Internal::status(this), output); } - // int64 nq = 2; - if (this->nq() != 0) { - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteInt64(2, this->nq(), output); + // int64 row_num = 2; + if (this->row_num() != 0) { + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteInt64(2, this->row_num(), output); } - // int64 topk = 3; - if (this->topk() != 0) { - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteInt64(3, this->topk(), output); + // repeated int64 ids = 3; + if (this->ids_size() > 0) { + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteTag(3, ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WIRETYPE_LENGTH_DELIMITED, output); + output->WriteVarint32(_ids_cached_byte_size_.load( + std::memory_order_relaxed)); + } + for (int i = 0, n = this->ids_size(); i < n; i++) { + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteInt64NoTag( + this->ids(i), output); } - // bytes ids_binary = 4; - if (this->ids_binary().size() > 0) { - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteBytesMaybeAliased( - 4, this->ids_binary(), output); - } - - // bytes distances_binary = 5; - if (this->distances_binary().size() > 0) { - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteBytesMaybeAliased( - 5, this->distances_binary(), output); + // repeated float distances = 4; + if (this->distances_size() > 0) { + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteTag(4, ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WIRETYPE_LENGTH_DELIMITED, output); + output->WriteVarint32(_distances_cached_byte_size_.load( + std::memory_order_relaxed)); + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteFloatArray( + this->distances().data(), this->distances_size(), output); } if (_internal_metadata_.have_unknown_fields()) { @@ -4120,28 +4103,35 @@ void TopKQueryResult::SerializeWithCachedSizes( 1, _Internal::status(this), target); } - // int64 nq = 2; - if (this->nq() != 0) { - target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteInt64ToArray(2, this->nq(), target); + // int64 row_num = 2; + if (this->row_num() != 0) { + target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteInt64ToArray(2, this->row_num(), target); } - // int64 topk = 3; - if (this->topk() != 0) { - target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteInt64ToArray(3, this->topk(), target); + // repeated int64 ids = 3; + if (this->ids_size() > 0) { + target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteTagToArray( + 3, + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WIRETYPE_LENGTH_DELIMITED, + target); + target = ::PROTOBUF_NAMESPACE_ID::io::CodedOutputStream::WriteVarint32ToArray( + _ids_cached_byte_size_.load(std::memory_order_relaxed), + target); + target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite:: + WriteInt64NoTagToArray(this->ids_, target); } - // bytes ids_binary = 4; - if (this->ids_binary().size() > 0) { - target = - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteBytesToArray( - 4, this->ids_binary(), target); - } - - // bytes distances_binary = 5; - if (this->distances_binary().size() > 0) { - target = - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteBytesToArray( - 5, this->distances_binary(), target); + // repeated float distances = 4; + if (this->distances_size() > 0) { + target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WriteTagToArray( + 4, + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::WIRETYPE_LENGTH_DELIMITED, + target); + target = ::PROTOBUF_NAMESPACE_ID::io::CodedOutputStream::WriteVarint32ToArray( + _distances_cached_byte_size_.load(std::memory_order_relaxed), + target); + target = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite:: + WriteFloatNoTagToArray(this->distances_, target); } if (_internal_metadata_.have_unknown_fields()) { @@ -4165,18 +4155,34 @@ size_t TopKQueryResult::ByteSizeLong() const { // Prevent compiler warnings about cached_has_bits being unused (void) cached_has_bits; - // bytes ids_binary = 4; - if (this->ids_binary().size() > 0) { - total_size += 1 + - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::BytesSize( - this->ids_binary()); + // repeated int64 ids = 3; + { + size_t data_size = ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite:: + Int64Size(this->ids_); + if (data_size > 0) { + total_size += 1 + + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::Int32Size( + static_cast<::PROTOBUF_NAMESPACE_ID::int32>(data_size)); + } + int cached_size = ::PROTOBUF_NAMESPACE_ID::internal::ToCachedSize(data_size); + _ids_cached_byte_size_.store(cached_size, + std::memory_order_relaxed); + total_size += data_size; } - // bytes distances_binary = 5; - if (this->distances_binary().size() > 0) { - total_size += 1 + - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::BytesSize( - this->distances_binary()); + // repeated float distances = 4; + { + unsigned int count = static_cast(this->distances_size()); + size_t data_size = 4UL * count; + if (data_size > 0) { + total_size += 1 + + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::Int32Size( + static_cast<::PROTOBUF_NAMESPACE_ID::int32>(data_size)); + } + int cached_size = ::PROTOBUF_NAMESPACE_ID::internal::ToCachedSize(data_size); + _distances_cached_byte_size_.store(cached_size, + std::memory_order_relaxed); + total_size += data_size; } // .milvus.grpc.Status status = 1; @@ -4186,18 +4192,11 @@ size_t TopKQueryResult::ByteSizeLong() const { *status_); } - // int64 nq = 2; - if (this->nq() != 0) { + // int64 row_num = 2; + if (this->row_num() != 0) { total_size += 1 + ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::Int64Size( - this->nq()); - } - - // int64 topk = 3; - if (this->topk() != 0) { - total_size += 1 + - ::PROTOBUF_NAMESPACE_ID::internal::WireFormatLite::Int64Size( - this->topk()); + this->row_num()); } int cached_size = ::PROTOBUF_NAMESPACE_ID::internal::ToCachedSize(total_size); @@ -4227,22 +4226,13 @@ void TopKQueryResult::MergeFrom(const TopKQueryResult& from) { ::PROTOBUF_NAMESPACE_ID::uint32 cached_has_bits = 0; (void) cached_has_bits; - if (from.ids_binary().size() > 0) { - - ids_binary_.AssignWithDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), from.ids_binary_); - } - if (from.distances_binary().size() > 0) { - - distances_binary_.AssignWithDefault(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), from.distances_binary_); - } + ids_.MergeFrom(from.ids_); + distances_.MergeFrom(from.distances_); if (from.has_status()) { mutable_status()->::milvus::grpc::Status::MergeFrom(from.status()); } - if (from.nq() != 0) { - set_nq(from.nq()); - } - if (from.topk() != 0) { - set_topk(from.topk()); + if (from.row_num() != 0) { + set_row_num(from.row_num()); } } @@ -4267,13 +4257,10 @@ bool TopKQueryResult::IsInitialized() const { void TopKQueryResult::InternalSwap(TopKQueryResult* other) { using std::swap; _internal_metadata_.Swap(&other->_internal_metadata_); - ids_binary_.Swap(&other->ids_binary_, &::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), - GetArenaNoVirtual()); - distances_binary_.Swap(&other->distances_binary_, &::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), - GetArenaNoVirtual()); + ids_.InternalSwap(&other->ids_); + distances_.InternalSwap(&other->distances_); swap(status_, other->status_); - swap(nq_, other->nq_); - swap(topk_, other->topk_); + swap(row_num_, other->row_num_); } ::PROTOBUF_NAMESPACE_ID::Metadata TopKQueryResult::GetMetadata() const { diff --git a/core/src/grpc/gen-milvus/milvus.pb.h b/core/src/grpc/gen-milvus/milvus.pb.h index 9db159dd89..5094f22341 100644 --- a/core/src/grpc/gen-milvus/milvus.pb.h +++ b/core/src/grpc/gen-milvus/milvus.pb.h @@ -1638,33 +1638,35 @@ class TopKQueryResult : // accessors ------------------------------------------------------- enum : int { - kIdsBinaryFieldNumber = 4, - kDistancesBinaryFieldNumber = 5, + kIdsFieldNumber = 3, + kDistancesFieldNumber = 4, kStatusFieldNumber = 1, - kNqFieldNumber = 2, - kTopkFieldNumber = 3, + kRowNumFieldNumber = 2, }; - // bytes ids_binary = 4; - void clear_ids_binary(); - const std::string& ids_binary() const; - void set_ids_binary(const std::string& value); - void set_ids_binary(std::string&& value); - void set_ids_binary(const char* value); - void set_ids_binary(const void* value, size_t size); - std::string* mutable_ids_binary(); - std::string* release_ids_binary(); - void set_allocated_ids_binary(std::string* ids_binary); + // repeated int64 ids = 3; + int ids_size() const; + void clear_ids(); + ::PROTOBUF_NAMESPACE_ID::int64 ids(int index) const; + void set_ids(int index, ::PROTOBUF_NAMESPACE_ID::int64 value); +// void add_ids(::PROTOBUF_NAMESPACE_ID::int64 value); + void add_ids(std::vector<::PROTOBUF_NAMESPACE_ID::int64>::const_iterator begin, + std::vector<::PROTOBUF_NAMESPACE_ID::int64>::const_iterator end); + const ::PROTOBUF_NAMESPACE_ID::RepeatedField< ::PROTOBUF_NAMESPACE_ID::int64 >& + ids() const; + ::PROTOBUF_NAMESPACE_ID::RepeatedField< ::PROTOBUF_NAMESPACE_ID::int64 >* + mutable_ids(); - // bytes distances_binary = 5; - void clear_distances_binary(); - const std::string& distances_binary() const; - void set_distances_binary(const std::string& value); - void set_distances_binary(std::string&& value); - void set_distances_binary(const char* value); - void set_distances_binary(const void* value, size_t size); - std::string* mutable_distances_binary(); - std::string* release_distances_binary(); - void set_allocated_distances_binary(std::string* distances_binary); + // repeated float distances = 4; + int distances_size() const; + void clear_distances(); + float distances(int index) const; + void set_distances(int index, float value); +// void add_distances(float value); + void add_distances(std::vector::const_iterator begin, std::vector::const_iterator end); + const ::PROTOBUF_NAMESPACE_ID::RepeatedField< float >& + distances() const; + ::PROTOBUF_NAMESPACE_ID::RepeatedField< float >* + mutable_distances(); // .milvus.grpc.Status status = 1; bool has_status() const; @@ -1674,26 +1676,22 @@ class TopKQueryResult : ::milvus::grpc::Status* mutable_status(); void set_allocated_status(::milvus::grpc::Status* status); - // int64 nq = 2; - void clear_nq(); - ::PROTOBUF_NAMESPACE_ID::int64 nq() const; - void set_nq(::PROTOBUF_NAMESPACE_ID::int64 value); - - // int64 topk = 3; - void clear_topk(); - ::PROTOBUF_NAMESPACE_ID::int64 topk() const; - void set_topk(::PROTOBUF_NAMESPACE_ID::int64 value); + // int64 row_num = 2; + void clear_row_num(); + ::PROTOBUF_NAMESPACE_ID::int64 row_num() const; + void set_row_num(::PROTOBUF_NAMESPACE_ID::int64 value); // @@protoc_insertion_point(class_scope:milvus.grpc.TopKQueryResult) private: class _Internal; ::PROTOBUF_NAMESPACE_ID::internal::InternalMetadataWithArena _internal_metadata_; - ::PROTOBUF_NAMESPACE_ID::internal::ArenaStringPtr ids_binary_; - ::PROTOBUF_NAMESPACE_ID::internal::ArenaStringPtr distances_binary_; + ::PROTOBUF_NAMESPACE_ID::RepeatedField< ::PROTOBUF_NAMESPACE_ID::int64 > ids_; + mutable std::atomic _ids_cached_byte_size_; + ::PROTOBUF_NAMESPACE_ID::RepeatedField< float > distances_; + mutable std::atomic _distances_cached_byte_size_; ::milvus::grpc::Status* status_; - ::PROTOBUF_NAMESPACE_ID::int64 nq_; - ::PROTOBUF_NAMESPACE_ID::int64 topk_; + ::PROTOBUF_NAMESPACE_ID::int64 row_num_; mutable ::PROTOBUF_NAMESPACE_ID::internal::CachedSize _cached_size_; friend struct ::TableStruct_milvus_2eproto; }; @@ -3154,8 +3152,8 @@ inline void RowRecord::set_vector_data(int index, float value) { //} inline void RowRecord::add_vector_data(std::vector::const_iterator begin, std::vector::const_iterator end) { - vector_data_.Add(begin, end); - // @@protoc_insertion_point(field_add:milvus.grpc.RowRecord.vector_data) + vector_data_.Add(begin, end); + // @@protoc_insertion_point(field_add:milvus.grpc.RowRecord.vector_data) } inline const ::PROTOBUF_NAMESPACE_ID::RepeatedField< float >& RowRecord::vector_data() const { @@ -3274,8 +3272,8 @@ inline void InsertParam::set_row_id_array(int index, ::PROTOBUF_NAMESPACE_ID::in //} inline void InsertParam::add_row_id_array(std::vector<::PROTOBUF_NAMESPACE_ID::int64>::const_iterator begin, std::vector<::PROTOBUF_NAMESPACE_ID::int64>::const_iterator end) { - row_id_array_.Add(begin, end); - // @@protoc_insertion_point(field_add:milvus.grpc.InsertParam.row_id_array) + row_id_array_.Add(begin, end); + // @@protoc_insertion_point(field_add:milvus.grpc.InsertParam.row_id_array) } inline const ::PROTOBUF_NAMESPACE_ID::RepeatedField< ::PROTOBUF_NAMESPACE_ID::int64 >& InsertParam::row_id_array() const { @@ -3679,134 +3677,87 @@ inline void TopKQueryResult::set_allocated_status(::milvus::grpc::Status* status // @@protoc_insertion_point(field_set_allocated:milvus.grpc.TopKQueryResult.status) } -// int64 nq = 2; -inline void TopKQueryResult::clear_nq() { - nq_ = PROTOBUF_LONGLONG(0); +// int64 row_num = 2; +inline void TopKQueryResult::clear_row_num() { + row_num_ = PROTOBUF_LONGLONG(0); } -inline ::PROTOBUF_NAMESPACE_ID::int64 TopKQueryResult::nq() const { - // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResult.nq) - return nq_; +inline ::PROTOBUF_NAMESPACE_ID::int64 TopKQueryResult::row_num() const { + // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResult.row_num) + return row_num_; } -inline void TopKQueryResult::set_nq(::PROTOBUF_NAMESPACE_ID::int64 value) { +inline void TopKQueryResult::set_row_num(::PROTOBUF_NAMESPACE_ID::int64 value) { - nq_ = value; - // @@protoc_insertion_point(field_set:milvus.grpc.TopKQueryResult.nq) + row_num_ = value; + // @@protoc_insertion_point(field_set:milvus.grpc.TopKQueryResult.row_num) } -// int64 topk = 3; -inline void TopKQueryResult::clear_topk() { - topk_ = PROTOBUF_LONGLONG(0); +// repeated int64 ids = 3; +inline int TopKQueryResult::ids_size() const { + return ids_.size(); } -inline ::PROTOBUF_NAMESPACE_ID::int64 TopKQueryResult::topk() const { - // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResult.topk) - return topk_; +inline void TopKQueryResult::clear_ids() { + ids_.Clear(); } -inline void TopKQueryResult::set_topk(::PROTOBUF_NAMESPACE_ID::int64 value) { - - topk_ = value; - // @@protoc_insertion_point(field_set:milvus.grpc.TopKQueryResult.topk) +inline ::PROTOBUF_NAMESPACE_ID::int64 TopKQueryResult::ids(int index) const { + // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResult.ids) + return ids_.Get(index); +} +inline void TopKQueryResult::set_ids(int index, ::PROTOBUF_NAMESPACE_ID::int64 value) { + ids_.Set(index, value); + // @@protoc_insertion_point(field_set:milvus.grpc.TopKQueryResult.ids) +} +//inline void TopKQueryResult::add_ids(::PROTOBUF_NAMESPACE_ID::int64 value) { +// ids_.Add(value); +// // @@protoc_insertion_point(field_add:milvus.grpc.TopKQueryResult.ids) +//} +inline void TopKQueryResult::add_ids(std::vector<::PROTOBUF_NAMESPACE_ID::int64>::const_iterator begin, + std::vector<::PROTOBUF_NAMESPACE_ID::int64>::const_iterator end) { + ids_.Add(begin,end); + // @@protoc_insertion_point(field_add:milvus.grpc.TopKQueryResult.ids) +} +inline const ::PROTOBUF_NAMESPACE_ID::RepeatedField< ::PROTOBUF_NAMESPACE_ID::int64 >& +TopKQueryResult::ids() const { + // @@protoc_insertion_point(field_list:milvus.grpc.TopKQueryResult.ids) + return ids_; +} +inline ::PROTOBUF_NAMESPACE_ID::RepeatedField< ::PROTOBUF_NAMESPACE_ID::int64 >* +TopKQueryResult::mutable_ids() { + // @@protoc_insertion_point(field_mutable_list:milvus.grpc.TopKQueryResult.ids) + return &ids_; } -// bytes ids_binary = 4; -inline void TopKQueryResult::clear_ids_binary() { - ids_binary_.ClearToEmptyNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); +// repeated float distances = 4; +inline int TopKQueryResult::distances_size() const { + return distances_.size(); } -inline const std::string& TopKQueryResult::ids_binary() const { - // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResult.ids_binary) - return ids_binary_.GetNoArena(); +inline void TopKQueryResult::clear_distances() { + distances_.Clear(); } -inline void TopKQueryResult::set_ids_binary(const std::string& value) { - - ids_binary_.SetNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), value); - // @@protoc_insertion_point(field_set:milvus.grpc.TopKQueryResult.ids_binary) +inline float TopKQueryResult::distances(int index) const { + // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResult.distances) + return distances_.Get(index); } -inline void TopKQueryResult::set_ids_binary(std::string&& value) { - - ids_binary_.SetNoArena( - &::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); - // @@protoc_insertion_point(field_set_rvalue:milvus.grpc.TopKQueryResult.ids_binary) +inline void TopKQueryResult::set_distances(int index, float value) { + distances_.Set(index, value); + // @@protoc_insertion_point(field_set:milvus.grpc.TopKQueryResult.distances) } -inline void TopKQueryResult::set_ids_binary(const char* value) { - GOOGLE_DCHECK(value != nullptr); - - ids_binary_.SetNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); - // @@protoc_insertion_point(field_set_char:milvus.grpc.TopKQueryResult.ids_binary) +//inline void TopKQueryResult::add_distances(float value) { +// distances_.Add(value); +// // @@protoc_insertion_point(field_add:milvus.grpc.TopKQueryResult.distances) +//} +inline void TopKQueryResult::add_distances(std::vector::const_iterator begin, std::vector::const_iterator end) { + distances_.Add(begin, end); + // @@protoc_insertion_point(field_add:milvus.grpc.TopKQueryResult.distances) } -inline void TopKQueryResult::set_ids_binary(const void* value, size_t size) { - - ids_binary_.SetNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), - ::std::string(reinterpret_cast(value), size)); - // @@protoc_insertion_point(field_set_pointer:milvus.grpc.TopKQueryResult.ids_binary) +inline const ::PROTOBUF_NAMESPACE_ID::RepeatedField< float >& +TopKQueryResult::distances() const { + // @@protoc_insertion_point(field_list:milvus.grpc.TopKQueryResult.distances) + return distances_; } -inline std::string* TopKQueryResult::mutable_ids_binary() { - - // @@protoc_insertion_point(field_mutable:milvus.grpc.TopKQueryResult.ids_binary) - return ids_binary_.MutableNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); -} -inline std::string* TopKQueryResult::release_ids_binary() { - // @@protoc_insertion_point(field_release:milvus.grpc.TopKQueryResult.ids_binary) - - return ids_binary_.ReleaseNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); -} -inline void TopKQueryResult::set_allocated_ids_binary(std::string* ids_binary) { - if (ids_binary != nullptr) { - - } else { - - } - ids_binary_.SetAllocatedNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), ids_binary); - // @@protoc_insertion_point(field_set_allocated:milvus.grpc.TopKQueryResult.ids_binary) -} - -// bytes distances_binary = 5; -inline void TopKQueryResult::clear_distances_binary() { - distances_binary_.ClearToEmptyNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); -} -inline const std::string& TopKQueryResult::distances_binary() const { - // @@protoc_insertion_point(field_get:milvus.grpc.TopKQueryResult.distances_binary) - return distances_binary_.GetNoArena(); -} -inline void TopKQueryResult::set_distances_binary(const std::string& value) { - - distances_binary_.SetNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), value); - // @@protoc_insertion_point(field_set:milvus.grpc.TopKQueryResult.distances_binary) -} -inline void TopKQueryResult::set_distances_binary(std::string&& value) { - - distances_binary_.SetNoArena( - &::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), ::std::move(value)); - // @@protoc_insertion_point(field_set_rvalue:milvus.grpc.TopKQueryResult.distances_binary) -} -inline void TopKQueryResult::set_distances_binary(const char* value) { - GOOGLE_DCHECK(value != nullptr); - - distances_binary_.SetNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), ::std::string(value)); - // @@protoc_insertion_point(field_set_char:milvus.grpc.TopKQueryResult.distances_binary) -} -inline void TopKQueryResult::set_distances_binary(const void* value, size_t size) { - - distances_binary_.SetNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), - ::std::string(reinterpret_cast(value), size)); - // @@protoc_insertion_point(field_set_pointer:milvus.grpc.TopKQueryResult.distances_binary) -} -inline std::string* TopKQueryResult::mutable_distances_binary() { - - // @@protoc_insertion_point(field_mutable:milvus.grpc.TopKQueryResult.distances_binary) - return distances_binary_.MutableNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); -} -inline std::string* TopKQueryResult::release_distances_binary() { - // @@protoc_insertion_point(field_release:milvus.grpc.TopKQueryResult.distances_binary) - - return distances_binary_.ReleaseNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited()); -} -inline void TopKQueryResult::set_allocated_distances_binary(std::string* distances_binary) { - if (distances_binary != nullptr) { - - } else { - - } - distances_binary_.SetAllocatedNoArena(&::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), distances_binary); - // @@protoc_insertion_point(field_set_allocated:milvus.grpc.TopKQueryResult.distances_binary) +inline ::PROTOBUF_NAMESPACE_ID::RepeatedField< float >* +TopKQueryResult::mutable_distances() { + // @@protoc_insertion_point(field_mutable_list:milvus.grpc.TopKQueryResult.distances) + return &distances_; } // ------------------------------------------------------------------- diff --git a/core/src/grpc/milvus.proto b/core/src/grpc/milvus.proto index e7fcdc3b2f..74bdda3f29 100644 --- a/core/src/grpc/milvus.proto +++ b/core/src/grpc/milvus.proto @@ -86,10 +86,9 @@ message SearchInFilesParam { */ message TopKQueryResult { Status status = 1; - int64 nq = 2; - int64 topk = 3; - bytes ids_binary = 4; - bytes distances_binary = 5; + int64 row_num = 2; + repeated int64 ids = 3; + repeated float distances = 4; } /** diff --git a/core/src/sdk/examples/grpcsimple/src/ClientTest.cpp b/core/src/sdk/examples/grpcsimple/src/ClientTest.cpp index a6e2b55281..09021e33b9 100644 --- a/core/src/sdk/examples/grpcsimple/src/ClientTest.cpp +++ b/core/src/sdk/examples/grpcsimple/src/ClientTest.cpp @@ -60,7 +60,7 @@ PrintSearchResult(const std::vector>& sear const milvus::TopKQueryResult& topk_query_result) { BLOCK_SPLITER size_t nq = topk_query_result.row_num; - size_t topk = topk_query_result.topk; + size_t topk = topk_query_result.ids.size() / nq; std::cout << "Returned result count: " << nq * topk << std::endl; int32_t index = 0; @@ -169,7 +169,7 @@ CheckResult(const std::vector>& search_rec const milvus::TopKQueryResult& topk_query_result) { BLOCK_SPLITER size_t nq = topk_query_result.row_num; - size_t result_k = topk_query_result.topk; + size_t result_k = topk_query_result.ids.size() / nq; int64_t index = 0; for (size_t i = 0; i < nq; i++) { auto result_id = topk_query_result.ids[i * result_k]; diff --git a/core/src/sdk/grpc/ClientProxy.cpp b/core/src/sdk/grpc/ClientProxy.cpp index bc572ed0a2..65b5f52a94 100644 --- a/core/src/sdk/grpc/ClientProxy.cpp +++ b/core/src/sdk/grpc/ClientProxy.cpp @@ -237,12 +237,12 @@ ClientProxy::Search(const std::string& table_name, const std::vector& Status status = client_ptr_->Search(result, search_param); // step 4: convert result array - topk_query_result.row_num = result.nq(); - topk_query_result.topk = result.topk(); - topk_query_result.ids.resize(result.ids_binary().size()); - memcpy(topk_query_result.ids.data(), result.ids_binary().data(), result.ids_binary().size()); - topk_query_result.distances.resize(result.distances_binary().size()); - memcpy(topk_query_result.distances.data(), result.distances_binary().data(), result.distances_binary().size()); + topk_query_result.row_num = result.row_num(); + topk_query_result.ids.resize(result.ids().size()); + memcpy(topk_query_result.ids.data(), result.ids().data(), result.ids().size() * sizeof(int64_t)); + topk_query_result.distances.resize(result.distances().size()); + memcpy(topk_query_result.distances.data(), result.distances().data(), + result.distances().size() * sizeof(float)); return status; } catch (std::exception& ex) { diff --git a/core/src/sdk/include/MilvusApi.h b/core/src/sdk/include/MilvusApi.h index 718334fed0..87ce88e460 100644 --- a/core/src/sdk/include/MilvusApi.h +++ b/core/src/sdk/include/MilvusApi.h @@ -83,7 +83,6 @@ struct RowRecord { */ struct TopKQueryResult { int64_t row_num; - int64_t topk; std::vector ids; std::vector distances; }; diff --git a/core/src/server/grpc_impl/GrpcRequestTask.cpp b/core/src/server/grpc_impl/GrpcRequestTask.cpp index 80bc87e6b7..b81fab0645 100644 --- a/core/src/server/grpc_impl/GrpcRequestTask.cpp +++ b/core/src/server/grpc_impl/GrpcRequestTask.cpp @@ -669,23 +669,10 @@ SearchTask::OnExecute() { return Status::OK(); // empty table } - size_t result_k = result_ids.size() / record_count; - // step 7: construct result array - topk_result_->set_nq(record_count); - topk_result_->set_topk(result_ids.size() / record_count); - - std::string ids_str; - size_t ids_len = sizeof(int64_t) * result_ids.size(); - ids_str.resize(ids_len); - memcpy((void*)(ids_str.data()), result_ids.data(), ids_len); - topk_result_->set_ids_binary(std::move(ids_str)); - - std::string distances_str; - size_t distances_len = sizeof(float) * result_distances.size(); - distances_str.resize(distances_len); - memcpy((void*)(distances_str.data()), result_distances.data(), distances_len); - topk_result_->set_distances_binary(std::move(distances_str)); + topk_result_->set_row_num(record_count); + topk_result_->add_ids(result_ids.begin(), result_ids.end()); + topk_result_->add_distances(result_distances.begin(), result_distances.end()); // step 8: print time cost percent rc.RecordSection("construct result and send"); From d4372d28b709956b0f736bb22943cae4e2e7ff70 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Mon, 11 Nov 2019 11:12:59 +0800 Subject: [PATCH 040/103] use ccache when compiling (if available) --- core/CMakeLists.txt | 12 ++++++++++++ core/cmake/DefineOptions.cmake | 2 ++ 2 files changed, 14 insertions(+) diff --git a/core/CMakeLists.txt b/core/CMakeLists.txt index 7b6a115527..65094b19a3 100644 --- a/core/CMakeLists.txt +++ b/core/CMakeLists.txt @@ -118,6 +118,18 @@ include(DefineOptions) include(BuildUtils) include(ThirdPartyPackages) +if(MILVUS_USE_CCACHE) + find_program(CCACHE_FOUND ccache) + if(CCACHE_FOUND) + message(STATUS "Using ccache: ${CCACHE_FOUND}") + set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE ${CCACHE_FOUND}) + set_property(GLOBAL PROPERTY RULE_LAUNCH_LINK ${CCACHE_FOUND}) + # let ccache preserve C++ comments, because some of them may be + # meaningful to the compiler + set(ENV{CCACHE_COMMENTS} "1") + endif(CCACHE_FOUND) +endif() + set(MILVUS_CPU_VERSION false) if (MILVUS_GPU_VERSION) message(STATUS "Building Milvus GPU version") diff --git a/core/cmake/DefineOptions.cmake b/core/cmake/DefineOptions.cmake index 6e05a12dd2..0050de75eb 100644 --- a/core/cmake/DefineOptions.cmake +++ b/core/cmake/DefineOptions.cmake @@ -57,6 +57,8 @@ define_option_string(MILVUS_DEPENDENCY_SOURCE "BUNDLED" "SYSTEM") +define_option(MILVUS_USE_CCACHE "Use ccache when compiling (if available)" ON) + define_option(MILVUS_VERBOSE_THIRDPARTY_BUILD "Show output from ExternalProjects rather than just logging to files" ON) From 57768e77249f28f6d4a93220b34460b0d657da07 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Mon, 11 Nov 2019 11:28:57 +0800 Subject: [PATCH 041/103] add ccache cache in .travis.yml --- .travis.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.travis.yml b/.travis.yml index ec3b1c41fa..ee21c874dd 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,4 +1,5 @@ language: cpp +cache: ccache sudo: required dist: bionic addons: From d94ebbca97ebc1b456f86a38b9aef2455498d04f Mon Sep 17 00:00:00 2001 From: quicksilver Date: Mon, 11 Nov 2019 11:51:06 +0800 Subject: [PATCH 042/103] add before-install.sh in travis --- .travis.yml | 6 ++++++ ci/travis/before-install.sh | 12 ++++++++++++ 2 files changed, 18 insertions(+) create mode 100755 ci/travis/before-install.sh diff --git a/.travis.yml b/.travis.yml index ee21c874dd..aaa390620f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -2,10 +2,16 @@ language: cpp cache: ccache sudo: required dist: bionic + addons: apt: update: true + +before_install: + - source ci/travis/before-install.sh + install: - source $TRAVIS_BUILD_DIR/ci/travis/install_dependency.sh + script: - $TRAVIS_BUILD_DIR/ci/travis/travis_build.sh diff --git a/ci/travis/before-install.sh b/ci/travis/before-install.sh new file mode 100755 index 0000000000..4ded8b8e38 --- /dev/null +++ b/ci/travis/before-install.sh @@ -0,0 +1,12 @@ +#!/bin/bash + +set -ex + +if [[ "${TRAVIS_OS_NAME}" == "linux" ]]; then + export CCACHE_COMPRESS=1 + export CCACHE_COMPRESSLEVEL=5 + export CCACHE_COMPILERCHECK=content + ccache --show-stats +fi + +set +ex From ae9d7ac36551b06ebcd203602c42ea778f05faa6 Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Mon, 11 Nov 2019 13:45:43 +0800 Subject: [PATCH 043/103] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 311d0d6843..6115a5ae57 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ ![Release](https://img.shields.io/badge/release-v0.5.1-yellowgreen) ![Release_date](https://img.shields.io/badge/release%20date-November-yellowgreen) -[中文版](README_CN.md) +[中文版](README_CN.md) | [日本語版](README_JP.md) ## What is Milvus From b837d135550b409751ac47ceed1d72710129efc0 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Mon, 11 Nov 2019 14:12:37 +0800 Subject: [PATCH 044/103] update .travis.yml --- .travis.yml | 5 ++++- ci/travis/before-install.sh | 1 + 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index aaa390620f..c4729a2555 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,8 +1,11 @@ language: cpp -cache: ccache sudo: required dist: bionic +cache: + directories: + - $HOME/.ccache + addons: apt: update: true diff --git a/ci/travis/before-install.sh b/ci/travis/before-install.sh index 4ded8b8e38..70133cfe1d 100755 --- a/ci/travis/before-install.sh +++ b/ci/travis/before-install.sh @@ -6,6 +6,7 @@ if [[ "${TRAVIS_OS_NAME}" == "linux" ]]; then export CCACHE_COMPRESS=1 export CCACHE_COMPRESSLEVEL=5 export CCACHE_COMPILERCHECK=content + export PATH=/usr/lib/ccache/:$PATH ccache --show-stats fi From ac61309c649a39031e9b08ceb1ccccf6a68701de Mon Sep 17 00:00:00 2001 From: quicksilver Date: Mon, 11 Nov 2019 14:15:34 +0800 Subject: [PATCH 045/103] update ci/travis/install_dependency.sh --- ci/travis/install_dependency.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/ci/travis/install_dependency.sh b/ci/travis/install_dependency.sh index cec4e9c30f..54e2f41ab5 100755 --- a/ci/travis/install_dependency.sh +++ b/ci/travis/install_dependency.sh @@ -18,6 +18,7 @@ sudo apt-get install -y -q --no-install-recommends \ lsb-core \ libtool \ automake \ + ccache \ pkg-config \ libboost-filesystem-dev \ libboost-system-dev \ From ba2548a93638f8d413c5fbaf8ac0a841911ac496 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Mon, 11 Nov 2019 16:44:31 +0800 Subject: [PATCH 046/103] add FindArrow.cmake --- core/src/index/cmake/FindArrow.cmake | 431 ++++++++++++++++++ .../index/cmake/ThirdPartyPackagesCore.cmake | 24 +- 2 files changed, 440 insertions(+), 15 deletions(-) create mode 100644 core/src/index/cmake/FindArrow.cmake diff --git a/core/src/index/cmake/FindArrow.cmake b/core/src/index/cmake/FindArrow.cmake new file mode 100644 index 0000000000..fdf7c1437f --- /dev/null +++ b/core/src/index/cmake/FindArrow.cmake @@ -0,0 +1,431 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# - Find Arrow (arrow/api.h, libarrow.a, libarrow.so) +# This module defines +# ARROW_FOUND, whether Arrow has been found +# ARROW_FULL_SO_VERSION, full shared object version of found Arrow "100.0.0" +# ARROW_IMPORT_LIB, path to libarrow's import library (Windows only) +# ARROW_INCLUDE_DIR, directory containing headers +# ARROW_LIBS, deprecated. Use ARROW_LIB_DIR instead +# ARROW_LIB_DIR, directory containing Arrow libraries +# ARROW_SHARED_IMP_LIB, deprecated. Use ARROW_IMPORT_LIB instead +# ARROW_SHARED_LIB, path to libarrow's shared library +# ARROW_SO_VERSION, shared object version of found Arrow such as "100" +# ARROW_STATIC_LIB, path to libarrow.a +# ARROW_VERSION, version of found Arrow +# ARROW_VERSION_MAJOR, major version of found Arrow +# ARROW_VERSION_MINOR, minor version of found Arrow +# ARROW_VERSION_PATCH, patch version of found Arrow + +include(FindPkgConfig) +include(FindPackageHandleStandardArgs) + +set(ARROW_SEARCH_LIB_PATH_SUFFIXES) +if(CMAKE_LIBRARY_ARCHITECTURE) + list(APPEND ARROW_SEARCH_LIB_PATH_SUFFIXES "lib/${CMAKE_LIBRARY_ARCHITECTURE}") +endif() +list(APPEND ARROW_SEARCH_LIB_PATH_SUFFIXES + "lib64" + "lib32" + "lib" + "bin") +set(ARROW_CONFIG_SUFFIXES + "_RELEASE" + "_RELWITHDEBINFO" + "_MINSIZEREL" + "_DEBUG" + "") +if(CMAKE_BUILD_TYPE) + string(TOUPPER ${CMAKE_BUILD_TYPE} ARROW_CONFIG_SUFFIX_PREFERRED) + set(ARROW_CONFIG_SUFFIX_PREFERRED "_${ARROW_CONFIG_SUFFIX_PREFERRED}") + list(INSERT ARROW_CONFIG_SUFFIXES 0 "${ARROW_CONFIG_SUFFIX_PREFERRED}") +endif() + +if(NOT DEFINED ARROW_MSVC_STATIC_LIB_SUFFIX) + if(MSVC) + set(ARROW_MSVC_STATIC_LIB_SUFFIX "_static") + else() + set(ARROW_MSVC_STATIC_LIB_SUFFIX "") + endif() +endif() + +# Internal function. +# +# Set shared library name for ${base_name} to ${output_variable}. +# +# Example: +# arrow_build_shared_library_name(ARROW_SHARED_LIBRARY_NAME arrow) +# # -> ARROW_SHARED_LIBRARY_NAME=libarrow.so on Linux +# # -> ARROW_SHARED_LIBRARY_NAME=libarrow.dylib on macOS +# # -> ARROW_SHARED_LIBRARY_NAME=arrow.dll with MSVC on Windows +# # -> ARROW_SHARED_LIBRARY_NAME=libarrow.dll with MinGW on Windows +function(arrow_build_shared_library_name output_variable base_name) + set(${output_variable} + "${CMAKE_SHARED_LIBRARY_PREFIX}${base_name}${CMAKE_SHARED_LIBRARY_SUFFIX}" + PARENT_SCOPE) +endfunction() + +# Internal function. +# +# Set import library name for ${base_name} to ${output_variable}. +# This is useful only for MSVC build. Import library is used only +# with MSVC build. +# +# Example: +# arrow_build_import_library_name(ARROW_IMPORT_LIBRARY_NAME arrow) +# # -> ARROW_IMPORT_LIBRARY_NAME=arrow on Linux (meaningless) +# # -> ARROW_IMPORT_LIBRARY_NAME=arrow on macOS (meaningless) +# # -> ARROW_IMPORT_LIBRARY_NAME=arrow.lib with MSVC on Windows +# # -> ARROW_IMPORT_LIBRARY_NAME=libarrow.dll.a with MinGW on Windows +function(arrow_build_import_library_name output_variable base_name) + set(${output_variable} + "${CMAKE_IMPORT_LIBRARY_PREFIX}${base_name}${CMAKE_IMPORT_LIBRARY_SUFFIX}" + PARENT_SCOPE) +endfunction() + +# Internal function. +# +# Set static library name for ${base_name} to ${output_variable}. +# +# Example: +# arrow_build_static_library_name(ARROW_STATIC_LIBRARY_NAME arrow) +# # -> ARROW_STATIC_LIBRARY_NAME=libarrow.a on Linux +# # -> ARROW_STATIC_LIBRARY_NAME=libarrow.a on macOS +# # -> ARROW_STATIC_LIBRARY_NAME=arrow.lib with MSVC on Windows +# # -> ARROW_STATIC_LIBRARY_NAME=libarrow.dll.a with MinGW on Windows +function(arrow_build_static_library_name output_variable base_name) + set( + ${output_variable} + "${CMAKE_STATIC_LIBRARY_PREFIX}${base_name}${ARROW_MSVC_STATIC_LIB_SUFFIX}${CMAKE_STATIC_LIBRARY_SUFFIX}" + PARENT_SCOPE) +endfunction() + +# Internal function. +# +# Set macro value for ${macro_name} in ${header_content} to ${output_variable}. +# +# Example: +# arrow_extract_macro_value(version_major +# "ARROW_VERSION_MAJOR" +# "#define ARROW_VERSION_MAJOR 1.0.0") +# # -> version_major=1.0.0 +function(arrow_extract_macro_value output_variable macro_name header_content) + string(REGEX MATCH "#define +${macro_name} +[^\r\n]+" macro_definition + "${header_content}") + string(REGEX + REPLACE "^#define +${macro_name} +(.+)$" "\\1" macro_value "${macro_definition}") + set(${output_variable} "${macro_value}" PARENT_SCOPE) +endfunction() + +# Internal macro only for arrow_find_package. +# +# Find package in HOME. +macro(arrow_find_package_home) + find_path(${prefix}_include_dir "${header_path}" + PATHS "${home}" + PATH_SUFFIXES "include" + NO_DEFAULT_PATH) + set(include_dir "${${prefix}_include_dir}") + set(${prefix}_INCLUDE_DIR "${include_dir}" PARENT_SCOPE) + + if(MSVC) + set(CMAKE_SHARED_LIBRARY_SUFFIXES_ORIGINAL ${CMAKE_FIND_LIBRARY_SUFFIXES}) + # .dll isn't found by find_library with MSVC because .dll isn't included in + # CMAKE_FIND_LIBRARY_SUFFIXES. + list(APPEND CMAKE_FIND_LIBRARY_SUFFIXES "${CMAKE_SHARED_LIBRARY_SUFFIX}") + endif() + find_library(${prefix}_shared_lib + NAMES "${shared_lib_name}" + PATHS "${home}" + PATH_SUFFIXES ${ARROW_SEARCH_LIB_PATH_SUFFIXES} + NO_DEFAULT_PATH) + if(MSVC) + set(CMAKE_SHARED_LIBRARY_SUFFIXES ${CMAKE_FIND_LIBRARY_SUFFIXES_ORIGINAL}) + endif() + set(shared_lib "${${prefix}_shared_lib}") + set(${prefix}_SHARED_LIB "${shared_lib}" PARENT_SCOPE) + if(shared_lib) + add_library(${target_shared} SHARED IMPORTED) + set_target_properties(${target_shared} PROPERTIES IMPORTED_LOCATION "${shared_lib}") + if(include_dir) + set_target_properties(${target_shared} + PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${include_dir}") + endif() + find_library(${prefix}_import_lib + NAMES "${import_lib_name}" + PATHS "${home}" + PATH_SUFFIXES ${ARROW_SEARCH_LIB_PATH_SUFFIXES} + NO_DEFAULT_PATH) + set(import_lib "${${prefix}_import_lib}") + set(${prefix}_IMPORT_LIB "${import_lib}" PARENT_SCOPE) + if(import_lib) + set_target_properties(${target_shared} PROPERTIES IMPORTED_IMPLIB "${import_lib}") + endif() + endif() + + find_library(${prefix}_static_lib + NAMES "${static_lib_name}" + PATHS "${home}" + PATH_SUFFIXES ${ARROW_SEARCH_LIB_PATH_SUFFIXES} + NO_DEFAULT_PATH) + set(static_lib "${${prefix}_static_lib}") + set(${prefix}_STATIC_LIB "${static_lib}" PARENT_SCOPE) + if(static_lib) + add_library(${target_static} STATIC IMPORTED) + set_target_properties(${target_static} PROPERTIES IMPORTED_LOCATION "${static_lib}") + if(include_dir) + set_target_properties(${target_static} + PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${include_dir}") + endif() + endif() +endmacro() + +# Internal macro only for arrow_find_package. +# +# Find package by CMake package configuration. +macro(arrow_find_package_cmake_package_configuration) + # ARROW-5575: We need to split target files for each component + if(TARGET ${target_shared} OR TARGET ${target_static}) + set(${cmake_package_name}_FOUND TRUE) + else() + find_package(${cmake_package_name} CONFIG) + endif() + if(${cmake_package_name}_FOUND) + set(${prefix}_USE_CMAKE_PACKAGE_CONFIG TRUE PARENT_SCOPE) + if(TARGET ${target_shared}) + foreach(suffix ${ARROW_CONFIG_SUFFIXES}) + get_target_property(shared_lib ${target_shared} IMPORTED_LOCATION${suffix}) + if(shared_lib) + # Remove shared library version: + # libarrow.so.100.0.0 -> libarrow.so + # Because ARROW_HOME and pkg-config approaches don't add + # shared library version. + string(REGEX + REPLACE "(${CMAKE_SHARED_LIBRARY_SUFFIX})[.0-9]+$" "\\1" shared_lib + "${shared_lib}") + set(${prefix}_SHARED_LIB "${shared_lib}" PARENT_SCOPE) + break() + endif() + endforeach() + endif() + if(TARGET ${target_static}) + foreach(suffix ${ARROW_CONFIG_SUFFIXES}) + get_target_property(static_lib ${target_static} IMPORTED_LOCATION${suffix}) + if(static_lib) + set(${prefix}_STATIC_LIB "${static_lib}" PARENT_SCOPE) + break() + endif() + endforeach() + endif() + endif() +endmacro() + +# Internal macro only for arrow_find_package. +# +# Find package by pkg-config. +macro(arrow_find_package_pkg_config) + pkg_check_modules(${prefix}_PC ${pkg_config_name}) + if(${prefix}_PC_FOUND) + set(${prefix}_USE_PKG_CONFIG TRUE PARENT_SCOPE) + + set(include_dir "${${prefix}_PC_INCLUDEDIR}") + set(lib_dir "${${prefix}_PC_LIBDIR}") + set(shared_lib_paths "${${prefix}_PC_LINK_LIBRARIES}") + # Use the first shared library path as the IMPORTED_LOCATION + # for ${target_shared}. This assumes that the first shared library + # path is the shared library path for this module. + list(GET shared_lib_paths 0 first_shared_lib_path) + # Use the rest shared library paths as the INTERFACE_LINK_LIBRARIES + # for ${target_shared}. This assumes that the rest shared library + # paths are dependency library paths for this module. + list(LENGTH shared_lib_paths n_shared_lib_paths) + if(n_shared_lib_paths LESS_EQUAL 1) + set(rest_shared_lib_paths) + else() + list(SUBLIST + shared_lib_paths + 1 + -1 + rest_shared_lib_paths) + endif() + + set(${prefix}_VERSION "${${prefix}_PC_VERSION}" PARENT_SCOPE) + set(${prefix}_INCLUDE_DIR "${include_dir}" PARENT_SCOPE) + set(${prefix}_SHARED_LIB "${first_shared_lib_path}" PARENT_SCOPE) + + add_library(${target_shared} SHARED IMPORTED) + set_target_properties(${target_shared} + PROPERTIES INTERFACE_INCLUDE_DIRECTORIES + "${include_dir}" + INTERFACE_LINK_LIBRARIES + "${rest_shared_lib_paths}" + IMPORTED_LOCATION + "${first_shared_lib_path}") + + find_library(${prefix}_static_lib + NAMES "${static_lib_name}" + PATHS "${lib_dir}" + NO_DEFAULT_PATH) + set(static_lib "${${prefix}_static_lib}") + set(${prefix}_STATIC_LIB "${static_lib}" PARENT_SCOPE) + if(static_lib) + add_library(${target_static} STATIC IMPORTED) + set_target_properties(${target_static} + PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${include_dir}" + IMPORTED_LOCATION "${static_lib}") + endif() + endif() +endmacro() + +function(arrow_find_package + prefix + home + base_name + header_path + cmake_package_name + pkg_config_name) + arrow_build_shared_library_name(shared_lib_name ${base_name}) + arrow_build_import_library_name(import_lib_name ${base_name}) + arrow_build_static_library_name(static_lib_name ${base_name}) + + set(target_shared ${base_name}_shared) + set(target_static ${base_name}_static) + + if(home) + arrow_find_package_home() + set(${prefix}_FIND_APPROACH "HOME: ${home}" PARENT_SCOPE) + else() + arrow_find_package_cmake_package_configuration() + if(${cmake_package_name}_FOUND) + set(${prefix}_FIND_APPROACH + "CMake package configuration: ${cmake_package_name}" + PARENT_SCOPE) + else() + arrow_find_package_pkg_config() + set(${prefix}_FIND_APPROACH "pkg-config: ${pkg_config_name}" PARENT_SCOPE) + endif() + endif() + + if(NOT include_dir) + if(TARGET ${target_shared}) + get_target_property(include_dir ${target_shared} INTERFACE_INCLUDE_DIRECTORIES) + elseif(TARGET ${target_static}) + get_target_property(include_dir ${target_static} INTERFACE_INCLUDE_DIRECTORIES) + endif() + endif() + if(include_dir) + set(${prefix}_INCLUDE_DIR "${include_dir}" PARENT_SCOPE) + endif() + + if(shared_lib) + get_filename_component(lib_dir "${shared_lib}" DIRECTORY) + elseif(static_lib) + get_filename_component(lib_dir "${static_lib}" DIRECTORY) + else() + set(lib_dir NOTFOUND) + endif() + set(${prefix}_LIB_DIR "${lib_dir}" PARENT_SCOPE) + # For backward compatibility + set(${prefix}_LIBS "${lib_dir}" PARENT_SCOPE) +endfunction() + +if(NOT "$ENV{ARROW_HOME}" STREQUAL "") + file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME) +endif() +arrow_find_package(ARROW + "${ARROW_HOME}" + arrow + arrow/api.h + Arrow + arrow) + +if(ARROW_HOME) + if(ARROW_INCLUDE_DIR) + file(READ "${ARROW_INCLUDE_DIR}/arrow/util/config.h" ARROW_CONFIG_H_CONTENT) + arrow_extract_macro_value(ARROW_VERSION_MAJOR "ARROW_VERSION_MAJOR" + "${ARROW_CONFIG_H_CONTENT}") + arrow_extract_macro_value(ARROW_VERSION_MINOR "ARROW_VERSION_MINOR" + "${ARROW_CONFIG_H_CONTENT}") + arrow_extract_macro_value(ARROW_VERSION_PATCH "ARROW_VERSION_PATCH" + "${ARROW_CONFIG_H_CONTENT}") + if("${ARROW_VERSION_MAJOR}" STREQUAL "" + OR "${ARROW_VERSION_MINOR}" STREQUAL "" + OR "${ARROW_VERSION_PATCH}" STREQUAL "") + set(ARROW_VERSION "0.0.0") + else() + set(ARROW_VERSION + "${ARROW_VERSION_MAJOR}.${ARROW_VERSION_MINOR}.${ARROW_VERSION_PATCH}") + endif() + + arrow_extract_macro_value(ARROW_SO_VERSION_QUOTED "ARROW_SO_VERSION" + "${ARROW_CONFIG_H_CONTENT}") + string(REGEX REPLACE "^\"(.+)\"$" "\\1" ARROW_SO_VERSION "${ARROW_SO_VERSION_QUOTED}") + arrow_extract_macro_value(ARROW_FULL_SO_VERSION_QUOTED "ARROW_FULL_SO_VERSION" + "${ARROW_CONFIG_H_CONTENT}") + string(REGEX + REPLACE "^\"(.+)\"$" "\\1" ARROW_FULL_SO_VERSION + "${ARROW_FULL_SO_VERSION_QUOTED}") + endif() +else() + if(ARROW_USE_CMAKE_PACKAGE_CONFIG) + find_package(Arrow CONFIG) + elseif(ARROW_USE_PKG_CONFIG) + pkg_get_variable(ARROW_SO_VERSION arrow so_version) + pkg_get_variable(ARROW_FULL_SO_VERSION arrow full_so_version) + endif() +endif() + +set(ARROW_ABI_VERSION ${ARROW_SO_VERSION}) + +mark_as_advanced(ARROW_ABI_VERSION + ARROW_CONFIG_SUFFIXES + ARROW_FULL_SO_VERSION + ARROW_IMPORT_LIB + ARROW_INCLUDE_DIR + ARROW_LIBS + ARROW_LIB_DIR + ARROW_SEARCH_LIB_PATH_SUFFIXES + ARROW_SHARED_IMP_LIB + ARROW_SHARED_LIB + ARROW_SO_VERSION + ARROW_STATIC_LIB + ARROW_VERSION + ARROW_VERSION_MAJOR + ARROW_VERSION_MINOR + ARROW_VERSION_PATCH) + +find_package_handle_standard_args(Arrow REQUIRED_VARS + # The first required variable is shown + # in the found message. So this list is + # not sorted alphabetically. + ARROW_INCLUDE_DIR + ARROW_LIB_DIR + ARROW_FULL_SO_VERSION + ARROW_SO_VERSION + VERSION_VAR + ARROW_VERSION) +set(ARROW_FOUND ${Arrow_FOUND}) + +if(Arrow_FOUND AND NOT Arrow_FIND_QUIETLY) + message(STATUS "Arrow version: ${ARROW_VERSION} (${ARROW_FIND_APPROACH})") + message(STATUS "Arrow SO and ABI version: ${ARROW_SO_VERSION}") + message(STATUS "Arrow full SO version: ${ARROW_FULL_SO_VERSION}") + message(STATUS "Found the Arrow core shared library: ${ARROW_SHARED_LIB}") + message(STATUS "Found the Arrow core import library: ${ARROW_IMPORT_LIB}") + message(STATUS "Found the Arrow core static library: ${ARROW_STATIC_LIB}") +endif() diff --git a/core/src/index/cmake/ThirdPartyPackagesCore.cmake b/core/src/index/cmake/ThirdPartyPackagesCore.cmake index 625fc0d6e2..ac3eb672e2 100644 --- a/core/src/index/cmake/ThirdPartyPackagesCore.cmake +++ b/core/src/index/cmake/ThirdPartyPackagesCore.cmake @@ -50,9 +50,14 @@ endmacro() macro(resolve_dependency DEPENDENCY_NAME) if (${DEPENDENCY_NAME}_SOURCE STREQUAL "AUTO") - #message(STATUS "Finding ${DEPENDENCY_NAME} package") - #message(STATUS "${DEPENDENCY_NAME} package not found") - build_dependency(${DEPENDENCY_NAME}) + if (${DEPENDENCY_NAME} STREQUAL "ARROW") + find_package(${DEPENDENCY_NAME} MODULE) + if (NOT ${${DEPENDENCY_NAME}_FOUND}) + build_dependency(${DEPENDENCY_NAME}) + endif () + else() + build_dependency(${DEPENDENCY_NAME}) + endif() elseif (${DEPENDENCY_NAME}_SOURCE STREQUAL "BUNDLED") build_dependency(${DEPENDENCY_NAME}) elseif (${DEPENDENCY_NAME}_SOURCE STREQUAL "SYSTEM") @@ -131,17 +136,6 @@ if (USE_JFROG_CACHE STREQUAL "ON") endif () endif () -macro(resolve_dependency DEPENDENCY_NAME) - if (${DEPENDENCY_NAME}_SOURCE STREQUAL "AUTO") - #disable find_package for now - build_dependency(${DEPENDENCY_NAME}) - elseif (${DEPENDENCY_NAME}_SOURCE STREQUAL "BUNDLED") - build_dependency(${DEPENDENCY_NAME}) - elseif (${DEPENDENCY_NAME}_SOURCE STREQUAL "SYSTEM") - find_package(${DEPENDENCY_NAME} REQUIRED) - endif () -endmacro() - # ---------------------------------------------------------------------- # ExternalProject options @@ -412,7 +406,7 @@ if (KNOWHERE_WITH_ARROW AND NOT TARGET arrow_ep) resolve_dependency(ARROW) - link_directories(SYSTEM ${ARROW_PREFIX}/lib/) + link_directories(SYSTEM ${ARROW_LIBRARY_DIRS}) include_directories(SYSTEM ${ARROW_INCLUDE_DIR}) endif () From 8ba423db030d47ea4f5d25925dffc18cdc366ccd Mon Sep 17 00:00:00 2001 From: quicksilver Date: Mon, 11 Nov 2019 16:55:02 +0800 Subject: [PATCH 047/103] update Index ThirdPartyPackagesCore.cmake --- core/src/index/cmake/ThirdPartyPackagesCore.cmake | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/src/index/cmake/ThirdPartyPackagesCore.cmake b/core/src/index/cmake/ThirdPartyPackagesCore.cmake index ac3eb672e2..e23d101128 100644 --- a/core/src/index/cmake/ThirdPartyPackagesCore.cmake +++ b/core/src/index/cmake/ThirdPartyPackagesCore.cmake @@ -51,7 +51,7 @@ endmacro() macro(resolve_dependency DEPENDENCY_NAME) if (${DEPENDENCY_NAME}_SOURCE STREQUAL "AUTO") if (${DEPENDENCY_NAME} STREQUAL "ARROW") - find_package(${DEPENDENCY_NAME} MODULE) + find_package(Arrow MODULE) if (NOT ${${DEPENDENCY_NAME}_FOUND}) build_dependency(${DEPENDENCY_NAME}) endif () From e29cf5a0fe158394ebfb6376ed0a6fb9b98e90f0 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Mon, 11 Nov 2019 17:00:29 +0800 Subject: [PATCH 048/103] Install libarrow-dev on System --- ci/scripts/build.sh | 1 + ci/travis/install_dependency.sh | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/ci/scripts/build.sh b/ci/scripts/build.sh index 253ee5893d..118d6a4785 100755 --- a/ci/scripts/build.sh +++ b/ci/scripts/build.sh @@ -89,6 +89,7 @@ CMAKE_CMD="cmake \ -DBUILD_UNIT_TEST=${BUILD_UNITTEST} \ -DBUILD_COVERAGE=${BUILD_COVERAGE} \ -DUSE_JFROG_CACHE=${USE_JFROG_CACHE} \ +-DARROW_SOURCE=AUTO \ ${MILVUS_CORE_DIR}" echo ${CMAKE_CMD} ${CMAKE_CMD} diff --git a/ci/travis/install_dependency.sh b/ci/travis/install_dependency.sh index 54e2f41ab5..627da9e38a 100755 --- a/ci/travis/install_dependency.sh +++ b/ci/travis/install_dependency.sh @@ -9,6 +9,13 @@ sudo apt-key add /tmp/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB echo "deb https://apt.repos.intel.com/mkl all main" | \ sudo tee /etc/apt/sources.list.d/intel-mkl.list +sudo wget -O /usr/share/keyrings/apache-arrow-keyring.gpg https://dl.bintray.com/apache/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-keyring.gpg + +sudo tee /etc/apt/sources.list.d/apache-arrow.list < Date: Mon, 11 Nov 2019 17:26:05 +0800 Subject: [PATCH 049/103] fix link arrow library path --- core/src/index/knowhere/CMakeLists.txt | 2 +- core/src/index/unittest/CMakeLists.txt | 2 +- core/src/index/unittest/faiss_benchmark/CMakeLists.txt | 2 +- core/src/index/unittest/faiss_ori/CMakeLists.txt | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/core/src/index/knowhere/CMakeLists.txt b/core/src/index/knowhere/CMakeLists.txt index 11c79e5466..373965acf0 100644 --- a/core/src/index/knowhere/CMakeLists.txt +++ b/core/src/index/knowhere/CMakeLists.txt @@ -49,7 +49,7 @@ set(depend_libs SPTAGLibStatic faiss arrow - ${ARROW_PREFIX}/lib/libjemalloc_pic.a + ${ARROW_LIB_DIR}/libjemalloc_pic.a gomp gfortran pthread diff --git a/core/src/index/unittest/CMakeLists.txt b/core/src/index/unittest/CMakeLists.txt index 145278a636..13277aa40e 100644 --- a/core/src/index/unittest/CMakeLists.txt +++ b/core/src/index/unittest/CMakeLists.txt @@ -6,7 +6,7 @@ include_directories(${INDEX_SOURCE_DIR}) set(depend_libs gtest gmock gtest_main gmock_main faiss - arrow "${ARROW_PREFIX}/lib/libjemalloc_pic.a" + arrow "${ARROW_LIB_DIR}/libjemalloc_pic.a" ) if (BUILD_FAISS_WITH_MKL) set(depend_libs ${depend_libs} diff --git a/core/src/index/unittest/faiss_benchmark/CMakeLists.txt b/core/src/index/unittest/faiss_benchmark/CMakeLists.txt index 72eb7e7a7e..6cad5abda5 100644 --- a/core/src/index/unittest/faiss_benchmark/CMakeLists.txt +++ b/core/src/index/unittest/faiss_benchmark/CMakeLists.txt @@ -13,7 +13,7 @@ if (KNOWHERE_GPU_VERSION) set(depend_libs faiss hdf5 - arrow ${ARROW_PREFIX}/lib/libjemalloc_pic.a + arrow ${ARROW_LIB_DIR}/libjemalloc_pic.a ) if (BUILD_FAISS_WITH_MKL) set(depend_libs ${depend_libs} diff --git a/core/src/index/unittest/faiss_ori/CMakeLists.txt b/core/src/index/unittest/faiss_ori/CMakeLists.txt index 8216764ab7..829a27fd87 100644 --- a/core/src/index/unittest/faiss_ori/CMakeLists.txt +++ b/core/src/index/unittest/faiss_ori/CMakeLists.txt @@ -8,7 +8,7 @@ if (KNOWHERE_GPU_VERSION) set(depend_libs faiss - arrow ${ARROW_PREFIX}/lib/libjemalloc_pic.a + arrow ${ARROW_LIB_DIR}/libjemalloc_pic.a ) if (BUILD_FAISS_WITH_MKL) set(depend_libs ${depend_libs} From 77dd0ad9af7051be82ee83b6fdf39eaef5e97c9f Mon Sep 17 00:00:00 2001 From: quicksilver Date: Mon, 11 Nov 2019 17:39:01 +0800 Subject: [PATCH 050/103] update ci/travis/install_dependency.sh --- ci/travis/install_dependency.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/ci/travis/install_dependency.sh b/ci/travis/install_dependency.sh index 627da9e38a..a2424727f8 100755 --- a/ci/travis/install_dependency.sh +++ b/ci/travis/install_dependency.sh @@ -28,6 +28,7 @@ sudo apt-get install -y -q --no-install-recommends \ ccache \ pkg-config \ libarrow-dev \ + libjemalloc-dev \ libboost-filesystem-dev \ libboost-system-dev \ libboost-regex-dev \ From 439456f8cddb952825c8736c6daec57e3918680e Mon Sep 17 00:00:00 2001 From: quicksilver Date: Mon, 11 Nov 2019 19:48:29 +0800 Subject: [PATCH 051/103] add build with mkl option in build.sh --- ci/scripts/build.sh | 10 ++++++++-- ci/travis/travis_build.sh | 4 ++-- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/ci/scripts/build.sh b/ci/scripts/build.sh index 118d6a4785..d077325fa8 100755 --- a/ci/scripts/build.sh +++ b/ci/scripts/build.sh @@ -19,9 +19,10 @@ BUILD_COVERAGE="OFF" USE_JFROG_CACHE="OFF" RUN_CPPLINT="OFF" CPU_VERSION="ON" +WITH_MKL="OFF" CUDA_COMPILER=/usr/local/cuda/bin/nvcc -while getopts "o:t:b:gulcjh" arg +while getopts "o:t:b:gulcjmh" arg do case $arg in o) @@ -49,6 +50,9 @@ do j) USE_JFROG_CACHE="ON" ;; + m) + WITH_MKL="ON" + ;; h) # help echo " @@ -61,10 +65,11 @@ parameter: -l: run cpplint, clang-format and clang-tidy(default: OFF) -c: code coverage(default: OFF) -j: use jfrog cache build directory(default: OFF) +-m: build with MKL(default: OFF) -h: help usage: -./build.sh -o \${INSTALL_PREFIX} -t \${BUILD_TYPE} -b \${CORE_BUILD_DIR} [-u] [-l] [-c] [-j] [-h] +./build.sh -o \${INSTALL_PREFIX} -t \${BUILD_TYPE} -b \${CORE_BUILD_DIR} [-u] [-l] [-c] [-j] [-m] [-h] " exit 0 ;; @@ -89,6 +94,7 @@ CMAKE_CMD="cmake \ -DBUILD_UNIT_TEST=${BUILD_UNITTEST} \ -DBUILD_COVERAGE=${BUILD_COVERAGE} \ -DUSE_JFROG_CACHE=${USE_JFROG_CACHE} \ +-DBUILD_FAISS_WITH_MKL=${WITH_MKL} \ -DARROW_SOURCE=AUTO \ ${MILVUS_CORE_DIR}" echo ${CMAKE_CMD} diff --git a/ci/travis/travis_build.sh b/ci/travis/travis_build.sh index 3cde1d5a4d..3f49750b6d 100755 --- a/ci/travis/travis_build.sh +++ b/ci/travis/travis_build.sh @@ -18,7 +18,7 @@ done BUILD_COMMON_FLAGS="-t ${MILVUS_BUILD_TYPE} -o ${MILVUS_INSTALL_PREFIX} -b ${MILVUS_BUILD_DIR}" if [ $only_library_mode == "yes" ]; then - ${TRAVIS_BUILD_DIR}/ci/scripts/build.sh ${BUILD_COMMON_FLAGS} + ${TRAVIS_BUILD_DIR}/ci/scripts/build.sh ${BUILD_COMMON_FLAGS} -m else - ${TRAVIS_BUILD_DIR}/ci/scripts/build.sh ${BUILD_COMMON_FLAGS} -u -c + ${TRAVIS_BUILD_DIR}/ci/scripts/build.sh ${BUILD_COMMON_FLAGS} -m -u -c fi \ No newline at end of file From 89f094f7cb00000f46d0ef6ce1df55e5479badb1 Mon Sep 17 00:00:00 2001 From: ZhifengZhang-CN Date: Mon, 11 Nov 2019 21:35:45 +0800 Subject: [PATCH 052/103] fix mkl library path error --- core/src/index/cmake/ThirdPartyPackagesCore.cmake | 1 + 1 file changed, 1 insertion(+) diff --git a/core/src/index/cmake/ThirdPartyPackagesCore.cmake b/core/src/index/cmake/ThirdPartyPackagesCore.cmake index e23d101128..8840db7621 100644 --- a/core/src/index/cmake/ThirdPartyPackagesCore.cmake +++ b/core/src/index/cmake/ThirdPartyPackagesCore.cmake @@ -716,6 +716,7 @@ macro(build_faiss) find_path(MKL_LIB_PATH NAMES "libmkl_intel_ilp64.a" "libmkl_gnu_thread.a" "libmkl_core.a" + PATHS /opt PATH_SUFFIXES "intel/compilers_and_libraries_${MKL_VERSION}/linux/mkl/lib/intel64/") if (${MKL_LIB_PATH} STREQUAL "MKL_LIB_PATH-NOTFOUND") message(FATAL_ERROR "Could not find MKL libraries") From af68a2f4d3b7e98ed45353b057d208704a16f1bf Mon Sep 17 00:00:00 2001 From: ZhifengZhang-CN Date: Mon, 11 Nov 2019 21:58:22 +0800 Subject: [PATCH 053/103] fix mkl library path error --- ci/travis/install_dependency.sh | 6 +++--- core/src/index/cmake/ThirdPartyPackagesCore.cmake | 3 +-- core/src/index/thirdparty/versions.txt | 3 ++- core/thirdparty/versions.txt | 1 - 4 files changed, 6 insertions(+), 7 deletions(-) diff --git a/ci/travis/install_dependency.sh b/ci/travis/install_dependency.sh index a2424727f8..6120fccda9 100755 --- a/ci/travis/install_dependency.sh +++ b/ci/travis/install_dependency.sh @@ -32,8 +32,8 @@ sudo apt-get install -y -q --no-install-recommends \ libboost-filesystem-dev \ libboost-system-dev \ libboost-regex-dev \ - intel-mkl-gnu-2019.4-243 \ - intel-mkl-core-2019.4-243 \ + intel-mkl-gnu-2019.5-281 \ + intel-mkl-core-2019.5-281 \ libmysqlclient-dev \ clang-format-6.0 \ clang-tidy-6.0 \ @@ -42,4 +42,4 @@ sudo apt-get install -y -q --no-install-recommends \ sudo ln -s /usr/lib/x86_64-linux-gnu/libmysqlclient.so \ /usr/lib/x86_64-linux-gnu/libmysqlclient_r.so -export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/opt/intel/compilers_and_libraries_2019.4.243/linux/mkl/lib/intel64 +export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/opt/intel/compilers_and_libraries_2019.5.281/linux/mkl/lib/intel64 diff --git a/core/src/index/cmake/ThirdPartyPackagesCore.cmake b/core/src/index/cmake/ThirdPartyPackagesCore.cmake index 8840db7621..3173a37d34 100644 --- a/core/src/index/cmake/ThirdPartyPackagesCore.cmake +++ b/core/src/index/cmake/ThirdPartyPackagesCore.cmake @@ -714,9 +714,8 @@ macro(build_faiss) if (BUILD_FAISS_WITH_MKL) - find_path(MKL_LIB_PATH + find_library(MKL_LIB_PATH NAMES "libmkl_intel_ilp64.a" "libmkl_gnu_thread.a" "libmkl_core.a" - PATHS /opt PATH_SUFFIXES "intel/compilers_and_libraries_${MKL_VERSION}/linux/mkl/lib/intel64/") if (${MKL_LIB_PATH} STREQUAL "MKL_LIB_PATH-NOTFOUND") message(FATAL_ERROR "Could not find MKL libraries") diff --git a/core/src/index/thirdparty/versions.txt b/core/src/index/thirdparty/versions.txt index a2b16414c2..f328ec437a 100644 --- a/core/src/index/thirdparty/versions.txt +++ b/core/src/index/thirdparty/versions.txt @@ -3,4 +3,5 @@ BOOST_VERSION=1.70.0 GTEST_VERSION=1.8.1 LAPACK_VERSION=v3.8.0 OPENBLAS_VERSION=v0.3.6 -FAISS_VERSION=branch-0.3.0 \ No newline at end of file +FAISS_VERSION=branch-0.3.0 +MKL_VERSION=2019.5.281 diff --git a/core/thirdparty/versions.txt b/core/thirdparty/versions.txt index 68023d4072..4faaf119e4 100644 --- a/core/thirdparty/versions.txt +++ b/core/thirdparty/versions.txt @@ -9,6 +9,5 @@ LIBUNWIND_VERSION=1.3.1 GPERFTOOLS_VERSION=2.7 GRPC_VERSION=master ZLIB_VERSION=v1.2.11 -MKL_VERSION=2019.5.281 # vim: set filetype=sh: From ce0d32fb53f8bfb0391fa682d31b73459c68842d Mon Sep 17 00:00:00 2001 From: ZhifengZhang-CN Date: Mon, 11 Nov 2019 22:38:41 +0800 Subject: [PATCH 054/103] fix mkl library path error --- core/src/index/cmake/ThirdPartyPackagesCore.cmake | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/src/index/cmake/ThirdPartyPackagesCore.cmake b/core/src/index/cmake/ThirdPartyPackagesCore.cmake index 3173a37d34..e23d101128 100644 --- a/core/src/index/cmake/ThirdPartyPackagesCore.cmake +++ b/core/src/index/cmake/ThirdPartyPackagesCore.cmake @@ -714,7 +714,7 @@ macro(build_faiss) if (BUILD_FAISS_WITH_MKL) - find_library(MKL_LIB_PATH + find_path(MKL_LIB_PATH NAMES "libmkl_intel_ilp64.a" "libmkl_gnu_thread.a" "libmkl_core.a" PATH_SUFFIXES "intel/compilers_and_libraries_${MKL_VERSION}/linux/mkl/lib/intel64/") if (${MKL_LIB_PATH} STREQUAL "MKL_LIB_PATH-NOTFOUND") From 1c99b2612d0ebd97d9050c196a8e14f67830d226 Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Tue, 12 Nov 2019 09:31:10 +0800 Subject: [PATCH 055/103] [skip ci] Add C++ SDK link --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index aa645c7ca1..08ac3de4a0 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ Milvus is the world's fastest similarity search engine for massive-scale feature For more detailed introduction of Milvus and its architecture, see [Milvus overview](https://www.milvus.io/docs/en/aboutmilvus/overview/). -Milvus provides stable [Python](https://github.com/milvus-io/pymilvus), [Java](https://github.com/milvus-io/milvus-sdk-java) and C++ APIs. +Milvus provides stable [Python](https://github.com/milvus-io/pymilvus), [Java](https://github.com/milvus-io/milvus-sdk-java) and [C++](https://github.com/milvus-io/milvus/tree/master/core/src/sdk) APIs. Keep up-to-date with newest releases and latest updates by reading Milvus [release notes](https://www.milvus.io/docs/en/release/v0.5.2/). From c0a728d244983c82aeb9fe9e71792c19f687045b Mon Sep 17 00:00:00 2001 From: groot Date: Tue, 12 Nov 2019 10:29:37 +0800 Subject: [PATCH 056/103] #284 Change C++ SDK to shread library --- CHANGELOG.md | 1 + core/src/sdk/CMakeLists.txt | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 906799bcfe..9359be28a2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,7 @@ Please mark all change in change log and use the ticket from JIRA. - \#12 - Pure CPU version for Milvus - \#77 - Support table partition - \#226 - Experimental shards middleware for Milvus +- \#284 - Change C++ SDK to shread library ## Improvement diff --git a/core/src/sdk/CMakeLists.txt b/core/src/sdk/CMakeLists.txt index c68712d34c..71a03c38d0 100644 --- a/core/src/sdk/CMakeLists.txt +++ b/core/src/sdk/CMakeLists.txt @@ -22,7 +22,7 @@ include_directories(include) aux_source_directory(interface interface_files) aux_source_directory(grpc grpc_client_files) -add_library(milvus_sdk STATIC +add_library(milvus_sdk SHARED ${interface_files} ${grpc_client_files} ${grpc_service_files} From 2aa6af2fd8ede0e882c2738c62f5adc41eebface Mon Sep 17 00:00:00 2001 From: groot Date: Tue, 12 Nov 2019 10:45:49 +0800 Subject: [PATCH 057/103] #284 Change C++ SDK to shread library --- CHANGELOG.md | 1 + core/src/config.h | 20 ++++++++++++++++++++ core/src/sdk/CMakeLists.txt | 2 +- 3 files changed, 22 insertions(+), 1 deletion(-) create mode 100644 core/src/config.h diff --git a/CHANGELOG.md b/CHANGELOG.md index 1ae461f4ee..1087c1b0e0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,7 @@ Please mark all change in change log and use the ticket from JIRA. - \#204 - improve grpc performance in search - \#207 - Add more unittest for config set/get - \#208 - optimize unittest to support run single test more easily +- \#284 - Change C++ SDK to shread library ## Task diff --git a/core/src/config.h b/core/src/config.h new file mode 100644 index 0000000000..745e6c5a64 --- /dev/null +++ b/core/src/config.h @@ -0,0 +1,20 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#define MILVUS_VERSION "0.6.0" +#define BUILD_TYPE "Debug" +#define BUILD_TIME "2019-11-12 10:25.02" diff --git a/core/src/sdk/CMakeLists.txt b/core/src/sdk/CMakeLists.txt index c68712d34c..71a03c38d0 100644 --- a/core/src/sdk/CMakeLists.txt +++ b/core/src/sdk/CMakeLists.txt @@ -22,7 +22,7 @@ include_directories(include) aux_source_directory(interface interface_files) aux_source_directory(grpc grpc_client_files) -add_library(milvus_sdk STATIC +add_library(milvus_sdk SHARED ${interface_files} ${grpc_client_files} ${grpc_service_files} From eb4b7c77f399236716f436b619d41bbf712fb574 Mon Sep 17 00:00:00 2001 From: groot Date: Tue, 12 Nov 2019 10:47:09 +0800 Subject: [PATCH 058/103] #284 Change C++ SDK to shread library --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9359be28a2..a951cb041f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,9 +11,9 @@ Please mark all change in change log and use the ticket from JIRA. - \#12 - Pure CPU version for Milvus - \#77 - Support table partition - \#226 - Experimental shards middleware for Milvus -- \#284 - Change C++ SDK to shread library ## Improvement +- \#284 - Change C++ SDK to shread library ## Task From a1ae44e602928efa0329db0ab8e7f99c108a8076 Mon Sep 17 00:00:00 2001 From: groot Date: Tue, 12 Nov 2019 10:48:44 +0800 Subject: [PATCH 059/103] #284 Change C++ SDK to shread library --- core/src/config.h | 20 -------------------- 1 file changed, 20 deletions(-) delete mode 100644 core/src/config.h diff --git a/core/src/config.h b/core/src/config.h deleted file mode 100644 index 745e6c5a64..0000000000 --- a/core/src/config.h +++ /dev/null @@ -1,20 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -#define MILVUS_VERSION "0.6.0" -#define BUILD_TYPE "Debug" -#define BUILD_TIME "2019-11-12 10:25.02" From f8e66f12a9f4246ee59b2a6e273d8f116760f83d Mon Sep 17 00:00:00 2001 From: quicksilver Date: Tue, 12 Nov 2019 11:12:03 +0800 Subject: [PATCH 060/103] add FindGTest.cmake --- ci/scripts/build.sh | 3 +- ci/travis/install_dependency.sh | 1 + core/cmake/DefineOptions.cmake | 2 +- core/cmake/FindGTest.cmake | 278 ++++++++++++++++++ core/cmake/ThirdPartyPackages.cmake | 6 +- core/src/index/cmake/DefineOptionsCore.cmake | 2 +- core/src/index/cmake/FindGTest.cmake | 278 ++++++++++++++++++ .../index/cmake/ThirdPartyPackagesCore.cmake | 14 +- 8 files changed, 570 insertions(+), 14 deletions(-) create mode 100644 core/cmake/FindGTest.cmake create mode 100644 core/src/index/cmake/FindGTest.cmake diff --git a/ci/scripts/build.sh b/ci/scripts/build.sh index d077325fa8..0875ac1dd1 100755 --- a/ci/scripts/build.sh +++ b/ci/scripts/build.sh @@ -95,7 +95,8 @@ CMAKE_CMD="cmake \ -DBUILD_COVERAGE=${BUILD_COVERAGE} \ -DUSE_JFROG_CACHE=${USE_JFROG_CACHE} \ -DBUILD_FAISS_WITH_MKL=${WITH_MKL} \ --DARROW_SOURCE=AUTO \ +-DArrow_SOURCE=AUTO \ +-DGTest_SOURCE=AUTO \ ${MILVUS_CORE_DIR}" echo ${CMAKE_CMD} ${CMAKE_CMD} diff --git a/ci/travis/install_dependency.sh b/ci/travis/install_dependency.sh index 6120fccda9..694d78b8f7 100755 --- a/ci/travis/install_dependency.sh +++ b/ci/travis/install_dependency.sh @@ -27,6 +27,7 @@ sudo apt-get install -y -q --no-install-recommends \ automake \ ccache \ pkg-config \ + libgtest-dev \ libarrow-dev \ libjemalloc-dev \ libboost-filesystem-dev \ diff --git a/core/cmake/DefineOptions.cmake b/core/cmake/DefineOptions.cmake index 0050de75eb..4f4c8b17b5 100644 --- a/core/cmake/DefineOptions.cmake +++ b/core/cmake/DefineOptions.cmake @@ -48,7 +48,7 @@ define_option(MILVUS_GPU_VERSION "Build GPU version" OFF) #---------------------------------------------------------------------- set_option_category("Thirdparty") -set(MILVUS_DEPENDENCY_SOURCE_DEFAULT "AUTO") +set(MILVUS_DEPENDENCY_SOURCE_DEFAULT "BUNDLED") define_option_string(MILVUS_DEPENDENCY_SOURCE "Method to use for acquiring MILVUS's build dependencies" diff --git a/core/cmake/FindGTest.cmake b/core/cmake/FindGTest.cmake new file mode 100644 index 0000000000..d746e40b05 --- /dev/null +++ b/core/cmake/FindGTest.cmake @@ -0,0 +1,278 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# Originally imported from the CMake project at commit +# df4ed1e9ffcdb6b99ccff9e6f44808fdd2abda56 with the following license header: +# +# Distributed under the OSI-approved BSD 3-Clause License. See accompanying +# file Copyright.txt or https://cmake.org/licensing for details. + + +#[=======================================================================[.rst: +FindGTest +--------- + +Locate the Google C++ Testing Framework. + +Imported targets +^^^^^^^^^^^^^^^^ + +This module defines the following :prop_tgt:`IMPORTED` targets: + +``GTest::GTest`` + The Google Test ``gtest`` library, if found; adds Thread::Thread + automatically +``GTest::Main`` + The Google Test ``gtest_main`` library, if found +``GMock::GMock`` + The Google Mock ``gmock`` library, if found + + +Result variables +^^^^^^^^^^^^^^^^ + +This module will set the following variables in your project: + +``GTEST_FOUND`` + Found the Google Testing framework +``GTEST_INCLUDE_DIRS`` + the directory containing the Google Test headers + +The library variables below are set as normal variables. These +contain debug/optimized keywords when a debugging library is found. + +``GTEST_LIBRARIES`` + The Google Test ``gtest`` library; note it also requires linking + with an appropriate thread library +``GTEST_MAIN_LIBRARIES`` + The Google Test ``gtest_main`` library +``GTEST_BOTH_LIBRARIES`` + Both ``gtest`` and ``gtest_main`` + +Cache variables +^^^^^^^^^^^^^^^ + +The following cache variables may also be set: + +``GTEST_ROOT`` + The root directory of the Google Test installation (may also be + set as an environment variable) +``GTEST_MSVC_SEARCH`` + If compiling with MSVC, this variable can be set to ``MT`` or + ``MD`` (the default) to enable searching a GTest build tree + + +Example usage +^^^^^^^^^^^^^ + +:: + + enable_testing() + find_package(GTest REQUIRED) + + add_executable(foo foo.cc) + target_link_libraries(foo GTest::GTest GTest::Main) + + add_test(AllTestsInFoo foo) + + +Deeper integration with CTest +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +See :module:`GoogleTest` for information on the :command:`gtest_add_tests` +and :command:`gtest_discover_tests` commands. +#]=======================================================================] + +# include(${CMAKE_CURRENT_LIST_DIR}/GoogleTest.cmake) + +function(__gtest_append_debugs _endvar _library) + if(${_library} AND ${_library}_DEBUG) + set(_output optimized ${${_library}} debug ${${_library}_DEBUG}) + else() + set(_output ${${_library}}) + endif() + set(${_endvar} ${_output} PARENT_SCOPE) +endfunction() + +function(__gtest_find_library _name) + find_library(${_name} + NAMES ${ARGN} + HINTS + ENV GTEST_ROOT + ${GTEST_ROOT} + PATH_SUFFIXES ${_gtest_libpath_suffixes} + ) + mark_as_advanced(${_name}) +endfunction() + +macro(__gtest_determine_windows_library_type _var) + if(EXISTS "${${_var}}") + file(TO_NATIVE_PATH "${${_var}}" _lib_path) + get_filename_component(_name "${${_var}}" NAME_WE) + file(STRINGS "${${_var}}" _match REGEX "${_name}\\.dll" LIMIT_COUNT 1) + if(NOT _match STREQUAL "") + set(${_var}_TYPE SHARED PARENT_SCOPE) + else() + set(${_var}_TYPE UNKNOWN PARENT_SCOPE) + endif() + return() + endif() +endmacro() + +function(__gtest_determine_library_type _var) + if(WIN32) + # For now, at least, only Windows really needs to know the library type + __gtest_determine_windows_library_type(${_var}) + __gtest_determine_windows_library_type(${_var}_RELEASE) + __gtest_determine_windows_library_type(${_var}_DEBUG) + endif() + # If we get here, no determination was made from the above checks + set(${_var}_TYPE UNKNOWN PARENT_SCOPE) +endfunction() + +function(__gtest_import_library _target _var _config) + if(_config) + set(_config_suffix "_${_config}") + else() + set(_config_suffix "") + endif() + + set(_lib "${${_var}${_config_suffix}}") + if(EXISTS "${_lib}") + if(_config) + set_property(TARGET ${_target} APPEND PROPERTY + IMPORTED_CONFIGURATIONS ${_config}) + endif() + set_target_properties(${_target} PROPERTIES + IMPORTED_LINK_INTERFACE_LANGUAGES${_config_suffix} "CXX") + if(WIN32 AND ${_var}_TYPE STREQUAL SHARED) + set_target_properties(${_target} PROPERTIES + IMPORTED_IMPLIB${_config_suffix} "${_lib}") + else() + set_target_properties(${_target} PROPERTIES + IMPORTED_LOCATION${_config_suffix} "${_lib}") + endif() + endif() +endfunction() + +# + +if(NOT DEFINED GTEST_MSVC_SEARCH) + set(GTEST_MSVC_SEARCH MD) +endif() + +set(_gtest_libpath_suffixes lib) +if(MSVC) + if(GTEST_MSVC_SEARCH STREQUAL "MD") + list(APPEND _gtest_libpath_suffixes + msvc/gtest-md/Debug + msvc/gtest-md/Release + msvc/x64/Debug + msvc/x64/Release + ) + elseif(GTEST_MSVC_SEARCH STREQUAL "MT") + list(APPEND _gtest_libpath_suffixes + msvc/gtest/Debug + msvc/gtest/Release + msvc/x64/Debug + msvc/x64/Release + ) + endif() +endif() + + +find_path(GTEST_INCLUDE_DIR gtest/gtest.h + HINTS + $ENV{GTEST_ROOT}/include + ${GTEST_ROOT}/include + PATH_SUFFIXES ${LIB_PATH_SUFFIXES} +) +mark_as_advanced(GTEST_INCLUDE_DIR) + +if(MSVC AND GTEST_MSVC_SEARCH STREQUAL "MD") + # The provided /MD project files for Google Test add -md suffixes to the + # library names. + __gtest_find_library(GTEST_LIBRARY gtest-md gtest) + __gtest_find_library(GTEST_LIBRARY_DEBUG gtest-mdd gtestd) + __gtest_find_library(GTEST_MAIN_LIBRARY gtest_main-md gtest_main) + __gtest_find_library(GTEST_MAIN_LIBRARY_DEBUG gtest_main-mdd gtest_maind) + __gtest_find_library(GMOCK_LIBRARY gmock-md gmock) + __gtest_find_library(GMOCK_LIBRARY_DEBUG gmock-mdd gmockd) +else() + __gtest_find_library(GTEST_LIBRARY gtest) + __gtest_find_library(GTEST_LIBRARY_DEBUG gtestd) + __gtest_find_library(GTEST_MAIN_LIBRARY gtest_main) + __gtest_find_library(GTEST_MAIN_LIBRARY_DEBUG gtest_maind) + __gtest_find_library(GMOCK_LIBRARY gmock) + __gtest_find_library(GMOCK_LIBRARY_DEBUG gtestd) +endif() + +include(FindPackageHandleStandardArgs) +FIND_PACKAGE_HANDLE_STANDARD_ARGS(GTest DEFAULT_MSG GTEST_LIBRARY GTEST_INCLUDE_DIR GTEST_MAIN_LIBRARY GMOCK_LIBRARY) + +if(GTEST_FOUND) + set(GTEST_INCLUDE_DIRS ${GTEST_INCLUDE_DIR}) + __gtest_append_debugs(GTEST_LIBRARIES GTEST_LIBRARY) + __gtest_append_debugs(GTEST_MAIN_LIBRARIES GTEST_MAIN_LIBRARY) + __gtest_append_debugs(GMOCK_LIBRARIES GMOCK_LIBRARY) + set(GTEST_BOTH_LIBRARIES ${GTEST_LIBRARIES} ${GTEST_MAIN_LIBRARIES}) + + find_package(Threads QUIET) + + if(NOT TARGET GTest::GTest) + __gtest_determine_library_type(GTEST_LIBRARY) + add_library(GTest::GTest ${GTEST_LIBRARY_TYPE} IMPORTED) + if(TARGET Threads::Threads) + set_target_properties(GTest::GTest PROPERTIES + INTERFACE_LINK_LIBRARIES Threads::Threads) + endif() + if(GTEST_LIBRARY_TYPE STREQUAL "SHARED") + set_target_properties(GTest::GTest PROPERTIES + INTERFACE_COMPILE_DEFINITIONS "GTEST_LINKED_AS_SHARED_LIBRARY=1") + endif() + if(GTEST_INCLUDE_DIRS) + set_target_properties(GTest::GTest PROPERTIES + INTERFACE_INCLUDE_DIRECTORIES "${GTEST_INCLUDE_DIRS}") + endif() + __gtest_import_library(GTest::GTest GTEST_LIBRARY "") + __gtest_import_library(GTest::GTest GTEST_LIBRARY "RELEASE") + __gtest_import_library(GTest::GTest GTEST_LIBRARY "DEBUG") + endif() + if(NOT TARGET GTest::Main) + __gtest_determine_library_type(GTEST_MAIN_LIBRARY) + add_library(GTest::Main ${GTEST_MAIN_LIBRARY_TYPE} IMPORTED) + set_target_properties(GTest::Main PROPERTIES + INTERFACE_LINK_LIBRARIES "GTest::GTest") + __gtest_import_library(GTest::Main GTEST_MAIN_LIBRARY "") + __gtest_import_library(GTest::Main GTEST_MAIN_LIBRARY "RELEASE") + __gtest_import_library(GTest::Main GTEST_MAIN_LIBRARY "DEBUG") + endif() + if(NOT TARGET GTest::GMock) + __gtest_determine_library_type(GMOCK_LIBRARY) + add_library(GTest::GMock ${GMOCK_LIBRARY_TYPE} IMPORTED) + if(TARGET Threads::Threads) + set_target_properties(GTest::GMock PROPERTIES + INTERFACE_LINK_LIBRARIES Threads::Threads) + endif() + if(GMOCK_LIBRARY_TYPE STREQUAL "SHARED") + set_target_properties(GTest::GMock PROPERTIES + INTERFACE_COMPILE_DEFINITIONS "GTEST_LINKED_AS_SHARED_LIBRARY=1") + endif() + if(GTEST_INCLUDE_DIRS) + set_target_properties(GTest::GMock PROPERTIES + INTERFACE_INCLUDE_DIRECTORIES "${GTEST_INCLUDE_DIRS}") + endif() + __gtest_import_library(GTest::GMock GMOCK_LIBRARY "") + __gtest_import_library(GTest::GMock GMOCK_LIBRARY "RELEASE") + __gtest_import_library(GTest::GMock GMOCK_LIBRARY "DEBUG") + endif() +endif() diff --git a/core/cmake/ThirdPartyPackages.cmake b/core/cmake/ThirdPartyPackages.cmake index 657efa2eef..2092879b17 100644 --- a/core/cmake/ThirdPartyPackages.cmake +++ b/core/cmake/ThirdPartyPackages.cmake @@ -164,8 +164,10 @@ endif () macro(resolve_dependency DEPENDENCY_NAME) if (${DEPENDENCY_NAME}_SOURCE STREQUAL "AUTO") - #disable find_package for now - build_dependency(${DEPENDENCY_NAME}) + find_package(${DEPENDENCY_NAME} MODULE) + if(NOT ${${DEPENDENCY_NAME}_FOUND}) + build_dependency(${DEPENDENCY_NAME}) + endif() elseif (${DEPENDENCY_NAME}_SOURCE STREQUAL "BUNDLED") build_dependency(${DEPENDENCY_NAME}) elseif (${DEPENDENCY_NAME}_SOURCE STREQUAL "SYSTEM") diff --git a/core/src/index/cmake/DefineOptionsCore.cmake b/core/src/index/cmake/DefineOptionsCore.cmake index e49b3a779a..2f050cdf4e 100644 --- a/core/src/index/cmake/DefineOptionsCore.cmake +++ b/core/src/index/cmake/DefineOptionsCore.cmake @@ -52,7 +52,7 @@ endif () #---------------------------------------------------------------------- set_option_category("Thirdparty") -set(KNOWHERE_DEPENDENCY_SOURCE_DEFAULT "AUTO") +set(KNOWHERE_DEPENDENCY_SOURCE_DEFAULT "BUNDLED") define_option_string(KNOWHERE_DEPENDENCY_SOURCE "Method to use for acquiring KNOWHERE's build dependencies" diff --git a/core/src/index/cmake/FindGTest.cmake b/core/src/index/cmake/FindGTest.cmake new file mode 100644 index 0000000000..d746e40b05 --- /dev/null +++ b/core/src/index/cmake/FindGTest.cmake @@ -0,0 +1,278 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# Originally imported from the CMake project at commit +# df4ed1e9ffcdb6b99ccff9e6f44808fdd2abda56 with the following license header: +# +# Distributed under the OSI-approved BSD 3-Clause License. See accompanying +# file Copyright.txt or https://cmake.org/licensing for details. + + +#[=======================================================================[.rst: +FindGTest +--------- + +Locate the Google C++ Testing Framework. + +Imported targets +^^^^^^^^^^^^^^^^ + +This module defines the following :prop_tgt:`IMPORTED` targets: + +``GTest::GTest`` + The Google Test ``gtest`` library, if found; adds Thread::Thread + automatically +``GTest::Main`` + The Google Test ``gtest_main`` library, if found +``GMock::GMock`` + The Google Mock ``gmock`` library, if found + + +Result variables +^^^^^^^^^^^^^^^^ + +This module will set the following variables in your project: + +``GTEST_FOUND`` + Found the Google Testing framework +``GTEST_INCLUDE_DIRS`` + the directory containing the Google Test headers + +The library variables below are set as normal variables. These +contain debug/optimized keywords when a debugging library is found. + +``GTEST_LIBRARIES`` + The Google Test ``gtest`` library; note it also requires linking + with an appropriate thread library +``GTEST_MAIN_LIBRARIES`` + The Google Test ``gtest_main`` library +``GTEST_BOTH_LIBRARIES`` + Both ``gtest`` and ``gtest_main`` + +Cache variables +^^^^^^^^^^^^^^^ + +The following cache variables may also be set: + +``GTEST_ROOT`` + The root directory of the Google Test installation (may also be + set as an environment variable) +``GTEST_MSVC_SEARCH`` + If compiling with MSVC, this variable can be set to ``MT`` or + ``MD`` (the default) to enable searching a GTest build tree + + +Example usage +^^^^^^^^^^^^^ + +:: + + enable_testing() + find_package(GTest REQUIRED) + + add_executable(foo foo.cc) + target_link_libraries(foo GTest::GTest GTest::Main) + + add_test(AllTestsInFoo foo) + + +Deeper integration with CTest +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +See :module:`GoogleTest` for information on the :command:`gtest_add_tests` +and :command:`gtest_discover_tests` commands. +#]=======================================================================] + +# include(${CMAKE_CURRENT_LIST_DIR}/GoogleTest.cmake) + +function(__gtest_append_debugs _endvar _library) + if(${_library} AND ${_library}_DEBUG) + set(_output optimized ${${_library}} debug ${${_library}_DEBUG}) + else() + set(_output ${${_library}}) + endif() + set(${_endvar} ${_output} PARENT_SCOPE) +endfunction() + +function(__gtest_find_library _name) + find_library(${_name} + NAMES ${ARGN} + HINTS + ENV GTEST_ROOT + ${GTEST_ROOT} + PATH_SUFFIXES ${_gtest_libpath_suffixes} + ) + mark_as_advanced(${_name}) +endfunction() + +macro(__gtest_determine_windows_library_type _var) + if(EXISTS "${${_var}}") + file(TO_NATIVE_PATH "${${_var}}" _lib_path) + get_filename_component(_name "${${_var}}" NAME_WE) + file(STRINGS "${${_var}}" _match REGEX "${_name}\\.dll" LIMIT_COUNT 1) + if(NOT _match STREQUAL "") + set(${_var}_TYPE SHARED PARENT_SCOPE) + else() + set(${_var}_TYPE UNKNOWN PARENT_SCOPE) + endif() + return() + endif() +endmacro() + +function(__gtest_determine_library_type _var) + if(WIN32) + # For now, at least, only Windows really needs to know the library type + __gtest_determine_windows_library_type(${_var}) + __gtest_determine_windows_library_type(${_var}_RELEASE) + __gtest_determine_windows_library_type(${_var}_DEBUG) + endif() + # If we get here, no determination was made from the above checks + set(${_var}_TYPE UNKNOWN PARENT_SCOPE) +endfunction() + +function(__gtest_import_library _target _var _config) + if(_config) + set(_config_suffix "_${_config}") + else() + set(_config_suffix "") + endif() + + set(_lib "${${_var}${_config_suffix}}") + if(EXISTS "${_lib}") + if(_config) + set_property(TARGET ${_target} APPEND PROPERTY + IMPORTED_CONFIGURATIONS ${_config}) + endif() + set_target_properties(${_target} PROPERTIES + IMPORTED_LINK_INTERFACE_LANGUAGES${_config_suffix} "CXX") + if(WIN32 AND ${_var}_TYPE STREQUAL SHARED) + set_target_properties(${_target} PROPERTIES + IMPORTED_IMPLIB${_config_suffix} "${_lib}") + else() + set_target_properties(${_target} PROPERTIES + IMPORTED_LOCATION${_config_suffix} "${_lib}") + endif() + endif() +endfunction() + +# + +if(NOT DEFINED GTEST_MSVC_SEARCH) + set(GTEST_MSVC_SEARCH MD) +endif() + +set(_gtest_libpath_suffixes lib) +if(MSVC) + if(GTEST_MSVC_SEARCH STREQUAL "MD") + list(APPEND _gtest_libpath_suffixes + msvc/gtest-md/Debug + msvc/gtest-md/Release + msvc/x64/Debug + msvc/x64/Release + ) + elseif(GTEST_MSVC_SEARCH STREQUAL "MT") + list(APPEND _gtest_libpath_suffixes + msvc/gtest/Debug + msvc/gtest/Release + msvc/x64/Debug + msvc/x64/Release + ) + endif() +endif() + + +find_path(GTEST_INCLUDE_DIR gtest/gtest.h + HINTS + $ENV{GTEST_ROOT}/include + ${GTEST_ROOT}/include + PATH_SUFFIXES ${LIB_PATH_SUFFIXES} +) +mark_as_advanced(GTEST_INCLUDE_DIR) + +if(MSVC AND GTEST_MSVC_SEARCH STREQUAL "MD") + # The provided /MD project files for Google Test add -md suffixes to the + # library names. + __gtest_find_library(GTEST_LIBRARY gtest-md gtest) + __gtest_find_library(GTEST_LIBRARY_DEBUG gtest-mdd gtestd) + __gtest_find_library(GTEST_MAIN_LIBRARY gtest_main-md gtest_main) + __gtest_find_library(GTEST_MAIN_LIBRARY_DEBUG gtest_main-mdd gtest_maind) + __gtest_find_library(GMOCK_LIBRARY gmock-md gmock) + __gtest_find_library(GMOCK_LIBRARY_DEBUG gmock-mdd gmockd) +else() + __gtest_find_library(GTEST_LIBRARY gtest) + __gtest_find_library(GTEST_LIBRARY_DEBUG gtestd) + __gtest_find_library(GTEST_MAIN_LIBRARY gtest_main) + __gtest_find_library(GTEST_MAIN_LIBRARY_DEBUG gtest_maind) + __gtest_find_library(GMOCK_LIBRARY gmock) + __gtest_find_library(GMOCK_LIBRARY_DEBUG gtestd) +endif() + +include(FindPackageHandleStandardArgs) +FIND_PACKAGE_HANDLE_STANDARD_ARGS(GTest DEFAULT_MSG GTEST_LIBRARY GTEST_INCLUDE_DIR GTEST_MAIN_LIBRARY GMOCK_LIBRARY) + +if(GTEST_FOUND) + set(GTEST_INCLUDE_DIRS ${GTEST_INCLUDE_DIR}) + __gtest_append_debugs(GTEST_LIBRARIES GTEST_LIBRARY) + __gtest_append_debugs(GTEST_MAIN_LIBRARIES GTEST_MAIN_LIBRARY) + __gtest_append_debugs(GMOCK_LIBRARIES GMOCK_LIBRARY) + set(GTEST_BOTH_LIBRARIES ${GTEST_LIBRARIES} ${GTEST_MAIN_LIBRARIES}) + + find_package(Threads QUIET) + + if(NOT TARGET GTest::GTest) + __gtest_determine_library_type(GTEST_LIBRARY) + add_library(GTest::GTest ${GTEST_LIBRARY_TYPE} IMPORTED) + if(TARGET Threads::Threads) + set_target_properties(GTest::GTest PROPERTIES + INTERFACE_LINK_LIBRARIES Threads::Threads) + endif() + if(GTEST_LIBRARY_TYPE STREQUAL "SHARED") + set_target_properties(GTest::GTest PROPERTIES + INTERFACE_COMPILE_DEFINITIONS "GTEST_LINKED_AS_SHARED_LIBRARY=1") + endif() + if(GTEST_INCLUDE_DIRS) + set_target_properties(GTest::GTest PROPERTIES + INTERFACE_INCLUDE_DIRECTORIES "${GTEST_INCLUDE_DIRS}") + endif() + __gtest_import_library(GTest::GTest GTEST_LIBRARY "") + __gtest_import_library(GTest::GTest GTEST_LIBRARY "RELEASE") + __gtest_import_library(GTest::GTest GTEST_LIBRARY "DEBUG") + endif() + if(NOT TARGET GTest::Main) + __gtest_determine_library_type(GTEST_MAIN_LIBRARY) + add_library(GTest::Main ${GTEST_MAIN_LIBRARY_TYPE} IMPORTED) + set_target_properties(GTest::Main PROPERTIES + INTERFACE_LINK_LIBRARIES "GTest::GTest") + __gtest_import_library(GTest::Main GTEST_MAIN_LIBRARY "") + __gtest_import_library(GTest::Main GTEST_MAIN_LIBRARY "RELEASE") + __gtest_import_library(GTest::Main GTEST_MAIN_LIBRARY "DEBUG") + endif() + if(NOT TARGET GTest::GMock) + __gtest_determine_library_type(GMOCK_LIBRARY) + add_library(GTest::GMock ${GMOCK_LIBRARY_TYPE} IMPORTED) + if(TARGET Threads::Threads) + set_target_properties(GTest::GMock PROPERTIES + INTERFACE_LINK_LIBRARIES Threads::Threads) + endif() + if(GMOCK_LIBRARY_TYPE STREQUAL "SHARED") + set_target_properties(GTest::GMock PROPERTIES + INTERFACE_COMPILE_DEFINITIONS "GTEST_LINKED_AS_SHARED_LIBRARY=1") + endif() + if(GTEST_INCLUDE_DIRS) + set_target_properties(GTest::GMock PROPERTIES + INTERFACE_INCLUDE_DIRECTORIES "${GTEST_INCLUDE_DIRS}") + endif() + __gtest_import_library(GTest::GMock GMOCK_LIBRARY "") + __gtest_import_library(GTest::GMock GMOCK_LIBRARY "RELEASE") + __gtest_import_library(GTest::GMock GMOCK_LIBRARY "DEBUG") + endif() +endif() diff --git a/core/src/index/cmake/ThirdPartyPackagesCore.cmake b/core/src/index/cmake/ThirdPartyPackagesCore.cmake index e23d101128..65615e885e 100644 --- a/core/src/index/cmake/ThirdPartyPackagesCore.cmake +++ b/core/src/index/cmake/ThirdPartyPackagesCore.cmake @@ -16,7 +16,7 @@ set(KNOWHERE_THIRDPARTY_DEPENDENCIES - ARROW + Arrow FAISS GTest LAPACK @@ -33,7 +33,7 @@ foreach (DEPENDENCY ${KNOWHERE_THIRDPARTY_DEPENDENCIES}) endforeach () macro(build_dependency DEPENDENCY_NAME) - if ("${DEPENDENCY_NAME}" STREQUAL "ARROW") + if ("${DEPENDENCY_NAME}" STREQUAL "Arrow") build_arrow() elseif ("${DEPENDENCY_NAME}" STREQUAL "LAPACK") build_lapack() @@ -50,13 +50,9 @@ endmacro() macro(resolve_dependency DEPENDENCY_NAME) if (${DEPENDENCY_NAME}_SOURCE STREQUAL "AUTO") - if (${DEPENDENCY_NAME} STREQUAL "ARROW") - find_package(Arrow MODULE) - if (NOT ${${DEPENDENCY_NAME}_FOUND}) - build_dependency(${DEPENDENCY_NAME}) - endif () - else() - build_dependency(${DEPENDENCY_NAME}) + find_package(${DEPENDENCY_NAME} MODULE) + if(NOT ${${DEPENDENCY_NAME}_FOUND}) + build_dependency(${DEPENDENCY_NAME}) endif() elseif (${DEPENDENCY_NAME}_SOURCE STREQUAL "BUNDLED") build_dependency(${DEPENDENCY_NAME}) From 583ec7a2b2012c5c8f3fb20dd4712bc9cab5310c Mon Sep 17 00:00:00 2001 From: quicksilver Date: Tue, 12 Nov 2019 11:34:21 +0800 Subject: [PATCH 061/103] format ARROW_LIB_DIR --- core/src/index/cmake/ThirdPartyPackagesCore.cmake | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/core/src/index/cmake/ThirdPartyPackagesCore.cmake b/core/src/index/cmake/ThirdPartyPackagesCore.cmake index 65615e885e..2fcc7d1001 100644 --- a/core/src/index/cmake/ThirdPartyPackagesCore.cmake +++ b/core/src/index/cmake/ThirdPartyPackagesCore.cmake @@ -286,6 +286,7 @@ macro(build_arrow) set(ARROW_STATIC_LIB "${ARROW_PREFIX}/lib/${CMAKE_STATIC_LIBRARY_PREFIX}${ARROW_STATIC_LIB_NAME}${CMAKE_STATIC_LIBRARY_SUFFIX}" ) + set(ARROW_LIB_DIR "${ARROW_PREFIX}/lib") set(ARROW_INCLUDE_DIR "${ARROW_PREFIX}/include") set(ARROW_CMAKE_ARGS @@ -382,7 +383,7 @@ macro(build_arrow) ) endif () - file(MAKE_DIRECTORY "${ARROW_PREFIX}/include") + file(MAKE_DIRECTORY "${ARROW_INCLUDE_DIR}") add_library(arrow STATIC IMPORTED) set_target_properties(arrow PROPERTIES IMPORTED_LOCATION "${ARROW_STATIC_LIB}" @@ -392,8 +393,8 @@ macro(build_arrow) set(JEMALLOC_PREFIX "${INDEX_BINARY_DIR}/arrow_ep-prefix/src/arrow_ep-build/jemalloc_ep-prefix/src/jemalloc_ep") add_custom_command(TARGET arrow_ep POST_BUILD - COMMAND ${CMAKE_COMMAND} -E make_directory ${ARROW_PREFIX}/lib/ - COMMAND ${CMAKE_COMMAND} -E copy ${JEMALLOC_PREFIX}/lib/libjemalloc_pic.a ${ARROW_PREFIX}/lib/ + COMMAND ${CMAKE_COMMAND} -E make_directory ${ARROW_LIB_DIR} + COMMAND ${CMAKE_COMMAND} -E copy ${JEMALLOC_PREFIX}/lib/libjemalloc_pic.a ${ARROW_LIB_DIR} DEPENDS ${JEMALLOC_PREFIX}/lib/libjemalloc_pic.a) endmacro() @@ -402,7 +403,7 @@ if (KNOWHERE_WITH_ARROW AND NOT TARGET arrow_ep) resolve_dependency(ARROW) - link_directories(SYSTEM ${ARROW_LIBRARY_DIRS}) + link_directories(SYSTEM ${ARROW_LIB_DIR}) include_directories(SYSTEM ${ARROW_INCLUDE_DIR}) endif () From d6fe3c468fe763dc18d2808b186e010c55ed75e2 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Tue, 12 Nov 2019 11:56:42 +0800 Subject: [PATCH 062/103] fix bug --- core/src/index/cmake/ThirdPartyPackagesCore.cmake | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/src/index/cmake/ThirdPartyPackagesCore.cmake b/core/src/index/cmake/ThirdPartyPackagesCore.cmake index 2fcc7d1001..60ac4b9eac 100644 --- a/core/src/index/cmake/ThirdPartyPackagesCore.cmake +++ b/core/src/index/cmake/ThirdPartyPackagesCore.cmake @@ -401,7 +401,7 @@ endmacro() if (KNOWHERE_WITH_ARROW AND NOT TARGET arrow_ep) - resolve_dependency(ARROW) + resolve_dependency(Arrow) link_directories(SYSTEM ${ARROW_LIB_DIR}) include_directories(SYSTEM ${ARROW_INCLUDE_DIR}) From 1a02484b21717cac956ea43822a9fd944f95ec17 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Tue, 12 Nov 2019 14:45:29 +0800 Subject: [PATCH 063/103] update arrow version to 0.15.1 --- core/src/index/thirdparty/versions.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/src/index/thirdparty/versions.txt b/core/src/index/thirdparty/versions.txt index f328ec437a..380c9dedad 100644 --- a/core/src/index/thirdparty/versions.txt +++ b/core/src/index/thirdparty/versions.txt @@ -1,4 +1,4 @@ -ARROW_VERSION=apache-arrow-0.14.0 +ARROW_VERSION=apache-arrow-0.15.1 BOOST_VERSION=1.70.0 GTEST_VERSION=1.8.1 LAPACK_VERSION=v3.8.0 From d82826965881cab66f1d75429f90a676d5911454 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Tue, 12 Nov 2019 14:49:49 +0800 Subject: [PATCH 064/103] update travis install_dependency.sh --- ci/travis/install_dependency.sh | 2 -- 1 file changed, 2 deletions(-) diff --git a/ci/travis/install_dependency.sh b/ci/travis/install_dependency.sh index 694d78b8f7..5b4a5e1482 100755 --- a/ci/travis/install_dependency.sh +++ b/ci/travis/install_dependency.sh @@ -19,8 +19,6 @@ APT_LINE sudo apt-get update -qq sudo apt-get install -y -q --no-install-recommends \ - flex \ - bison \ gfortran \ lsb-core \ libtool \ From bd41630b1f596992920821bf71bc7f7fc43b08a2 Mon Sep 17 00:00:00 2001 From: groot Date: Tue, 12 Nov 2019 15:29:04 +0800 Subject: [PATCH 065/103] fix typo --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a951cb041f..363c2cc385 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,7 +13,7 @@ Please mark all change in change log and use the ticket from JIRA. - \#226 - Experimental shards middleware for Milvus ## Improvement -- \#284 - Change C++ SDK to shread library +- \#284 - Change C++ SDK to shared library ## Task From c9bb9d7e58a06a9db2301076b3926c4bc2cfb028 Mon Sep 17 00:00:00 2001 From: groot Date: Tue, 12 Nov 2019 15:30:19 +0800 Subject: [PATCH 066/103] fix typo --- CHANGELOG.md | 2 +- core/src/config.h | 20 ++++++++++++++++++++ 2 files changed, 21 insertions(+), 1 deletion(-) create mode 100644 core/src/config.h diff --git a/CHANGELOG.md b/CHANGELOG.md index 1087c1b0e0..92167c3922 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,7 +12,7 @@ Please mark all change in change log and use the ticket from JIRA. - \#204 - improve grpc performance in search - \#207 - Add more unittest for config set/get - \#208 - optimize unittest to support run single test more easily -- \#284 - Change C++ SDK to shread library +- \#284 - Change C++ SDK to shared library ## Task diff --git a/core/src/config.h b/core/src/config.h new file mode 100644 index 0000000000..d935c36faf --- /dev/null +++ b/core/src/config.h @@ -0,0 +1,20 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#define MILVUS_VERSION "0.6.0" +#define BUILD_TYPE "Debug" +#define BUILD_TIME "2019-11-12 14:06.18" From 4d6f69927a5b575bf80321092b00f97fd84206f9 Mon Sep 17 00:00:00 2001 From: ZhifengZhang-CN Date: Tue, 12 Nov 2019 15:38:32 +0800 Subject: [PATCH 067/103] update travis install_dependency.sh --- ci/travis/install_dependency.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/ci/travis/install_dependency.sh b/ci/travis/install_dependency.sh index 5b4a5e1482..a1bff278a0 100755 --- a/ci/travis/install_dependency.sh +++ b/ci/travis/install_dependency.sh @@ -28,6 +28,7 @@ sudo apt-get install -y -q --no-install-recommends \ libgtest-dev \ libarrow-dev \ libjemalloc-dev \ + libboost_serialization-dev \ libboost-filesystem-dev \ libboost-system-dev \ libboost-regex-dev \ From 1266faef4a71670f56d98647c3c4eec8670135a9 Mon Sep 17 00:00:00 2001 From: Lutkin Wang Date: Tue, 12 Nov 2019 16:03:49 +0800 Subject: [PATCH 068/103] [skip ci] Updated the Japanese readme. --- README_JP.md | 153 +++++---------------------------------------------- 1 file changed, 15 insertions(+), 138 deletions(-) diff --git a/README_JP.md b/README_JP.md index 997b329283..fd80b5d2ca 100644 --- a/README_JP.md +++ b/README_JP.md @@ -13,142 +13,24 @@ ## 概要 -Milvus は大規模な特徴ベクトルにむかう類似性検索エンジンです。不均質な計算アーキテクチャーに基づいて効率を最大化出来ます。数十億のベクタの中に目標を検索できるまで数ミリ秒しかかからず、最低限の計算資源だけが必要です。 +Milvusは世界中一番早い特徴ベクトルにむかう類似性検索エンジンです。不均質な計算アーキテクチャーに基づいて効率を最大化出来ます。数十億のベクタの中に目標を検索できるまで数ミリ秒しかかからず、最低限の計算資源だけが必要です。 -Milvus は安定的な Python、Java 又は C++ APIsを提供します。 +Milvusは安定的なPython、Java又は C++ APIsを提供します。 Milvus [リリースノート](https://milvus.io/docs/en/release/v0.5.2/)を読んで最新バージョンや更新情報を手に入れます。 -- 不均質な計算アーキテクチャー - - Milvusは不均質な計算アーキテクチャーに基づいて効率を最大化出来ます。 - -- 様々な索引方法 - - Milvusはさまざまな索引方法を支えます。量子化、 木、 グラフに基づいて索引を作成できます。 - -- 知能てきな資源管理 - - Milvusはデータセットと利用できる資源を基づいて、自動的に検索アルゴリズムと索引作成方法を選びます。 - -- 水平拡張 - - Milvusはオンラインとオフラインに記憶域と計算を簡単な命令で拡張できます。 - -- 高い可用性 - - MilvusはKubernetes枠組みと統合するので、 単一障害点を避けられます。 - -- 高い互換性 - - Milvusはほぼ全ての深層学習モデルと主要なプログラミング言語と互換性があります。 - -- やすい使い方 - - Milvusは簡単にインストールできます。ベクタ以外のことを心配する必要がありますん。 - -- 視覚化てきな監視モード - - Prometheusに基づいてGUIでシステムの性能を監視出来ます。 - -## アーキテクチャー - -![Milvus_arch](https://github.com/milvus-io/docs/blob/master/assets/milvus_arch.png) ## はじめに -### ハードウェア要件 +DockerでMilvusをインストールすることは簡単です。[Milvusインストール案内](https://milvus.io/docs/en/userguide/install_milvus/) を参考してください。ソースからMilvusを構築するために、[ソースから構築する](install.md)を参考してください。 -| コンポーネント | お勧めの配置 | -| --------- | ----------------------------------- | -| CPU | Intel CPU Haswell 以上 | -| GPU | NVIDIA Pascal series 以上 | -| RAM メモリ | 8 GB 以上 (データ規模に関わる) | -| ハードディスク | SATA 3.0 SSD 以上 | +Milvusをコンフィグするために、[Milvusコンフィグ](https://github.com/milvus-io/docs/blob/master/reference/milvus_config.md)を読んでください。 -### Dockerでインストールする +### 初めてのMilvusプログラムを試す -DockerでMilvusをインストールすることは簡単です。 [Milvusインストール案内](https://milvus.io/docs/en/userguide/install_milvus/) を参考してください。 +[Python](https://www.milvus.io/docs/en/userguide/example_code/)や[Java](https://github.com/milvus-io/milvus-sdk-java/tree/master/examples)などのサンプルコードを使ってMilvusプログラムを試す。 -### ソースから構築する - -#### ソフトウェア要件 - -- Ubuntu 18.04 以上 -- CMake 3.14 以上 -- CUDA 10.0 以上 -- NVIDIA driver 418 以上 - -#### コンパイル - -##### 1 依存コンポーネントをインストールする - -```shell -$ cd [Milvus sourcecode path]/core -./ubuntu_build_deps.sh -``` - -##### 2 構築する - -```shell -$ cd [Milvus sourcecode path]/core -$ ./build.sh -t Debug -or -$ ./build.sh -t Release -``` - -構築が完成するとき、 Milvusを実行するために必要なものは全てこのディレクトリにあります： `[Milvus root path]/core/milvus`。 - -#### Milvusサーバーを実行する - -```shell -$ cd [Milvus root path]/core/milvus -``` - -`lib/`　ディレクトリを　`LD_LIBRARY_PATH`　に添付する。 - -```shell -$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/milvus/lib -``` - -Milvusサーバーを実行する。 - -```shell -$ cd scripts -$ ./start_server.sh -``` - -Milvusサーバーを止めるために、次のコードを実行する： - -```shell -$ ./stop_server.sh -``` - -`conf/server_config.yaml` と `conf/log_config.conf` でMilvusをコンフィグするために、 [Milvusコンフィグ](https://github.com/milvus-io/docs/blob/master/reference/milvus_config.md)を読んでください。 - -### 初めてのMilvusプログラムを実行する - -#### Pythonサンプルコードを実行する - -[Python 3.5](https://www.python.org/downloads/)以上のバージョンがインストールされていることを確かめてください。 - -Milvus Python SDK　をインストールする。 - -```shell -# Install Milvus Python SDK -$ pip install pymilvus==0.2.3 -``` - -新しいファイル　`example.py`　を作成し、 [Pythonサンプルコード]( https://github.com/milvus-io/pymilvus/blob/master/examples/advanced_example.py)を添付してください。 - -サンプルコードを実行する。 - -```shell -# Run Milvus Python example -$ python3 example.py -``` - -#### C++サンプルコードを実行する +C++サンプルコードを実行するために、次のコマンドをつかってください。 ```shell # Run Milvus C++ example @@ -156,11 +38,9 @@ $ python3 example.py $ ./sdk_simple ``` -#### Javaサンプルコードを実行する +## Milvusロードマップ -Java 8以上のバージョンがインストールされていることを確かめてください。 - -[このリンク](https://github.com/milvus-io/milvus-sdk-java/tree/master/examples)でサンプルコードを手に入れます。 +[ロードマップ](https://milvus.io/docs/en/roadmap/)を読んで、追加する予定の特性が分かります。 ## 貢献規約 @@ -172,23 +52,20 @@ Java 8以上のバージョンがインストールされていることを確 他の貢献者と交流したい場合は、Milvusの [slackチャンネル](https://join.slack.com/t/milvusio/shared_invite/enQtNzY1OTQ0NDI3NjMzLWNmYmM1NmNjOTQ5MGI5NDhhYmRhMGU5M2NhNzhhMDMzY2MzNDdlYjM5ODQ5MmE3ODFlYzU3YjJkNmVlNDQ2ZTk)に参加してください。 -## Milvusロードマップ - -[ロードマップ](https://milvus.io/docs/en/roadmap/)を読んで、追加する予定の特性が分かります。 ## 参考情報 -[Milvus公式サイト](https://www.milvus.io) +- [Milvus.io](https://www.milvus.io) -[Milvusドキュメント](https://www.milvus.io/docs/en/userguide/install_milvus/) +- [Milvus](https://github.com/milvus-io/bootcamp) -[Milvusトレーニングセンター](https://github.com/milvus-io/bootcamp) +- [Milvus Medium](https://medium.com/@milvusio) -[Milvusブロック](https://www.milvus.io/blog/) +- [Milvus CSDN](https://zilliz.blog.csdn.net/) -[Milvus CSDN](https://zilliz.blog.csdn.net/) +- [Milvus ツイッター](https://twitter.com/milvusio) -[Milvusロードマップ](https://milvus.io/docs/en/roadmap/) +- [Milvus Facebook](https://www.facebook.com/io.milvus.5) ## ライセンス From ff406d0300c23bf6f70b42b4b6ecd723d576c2b6 Mon Sep 17 00:00:00 2001 From: ZhifengZhang-CN Date: Tue, 12 Nov 2019 16:45:27 +0800 Subject: [PATCH 069/103] update travis install_dependency.sh --- ci/travis/install_dependency.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/travis/install_dependency.sh b/ci/travis/install_dependency.sh index a1bff278a0..95d7a698db 100755 --- a/ci/travis/install_dependency.sh +++ b/ci/travis/install_dependency.sh @@ -28,7 +28,7 @@ sudo apt-get install -y -q --no-install-recommends \ libgtest-dev \ libarrow-dev \ libjemalloc-dev \ - libboost_serialization-dev \ + libboost-serialization-dev \ libboost-filesystem-dev \ libboost-system-dev \ libboost-regex-dev \ From 636313622c3186b195129d8aeefe42033239d9c7 Mon Sep 17 00:00:00 2001 From: ZhifengZhang-CN Date: Tue, 12 Nov 2019 16:50:22 +0800 Subject: [PATCH 070/103] update travis travis_env_common.sh --- ci/travis/travis_env_common.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/travis/travis_env_common.sh b/ci/travis/travis_env_common.sh index ac63d2950b..f5f9cbbcf7 100644 --- a/ci/travis/travis_env_common.sh +++ b/ci/travis/travis_env_common.sh @@ -3,7 +3,7 @@ export MILVUS_BUILD_DIR=${TRAVIS_BUILD_DIR}/core/cmake_build export MILVUS_INSTALL_PREFIX=/opt/milvus export MILVUS_TRAVIS_COVERAGE=${MILVUS_TRAVIS_COVERAGE:=0} -if ["$MILVUS_TRAVIS_COVERAGE" == "1"]; then +if [ "${MILVUS_TRAVIS_COVERAGE}" == "1" ]; then export MILVUS_CPP_COVERAGE_FILE=${TRAVIS_BUILD_DIR}/output_new.info fi From d04f7b56a2de36fad250a5aa599ce9bd696bc5f6 Mon Sep 17 00:00:00 2001 From: ZhifengZhang-CN Date: Tue, 12 Nov 2019 17:11:32 +0800 Subject: [PATCH 071/103] update FindGTest.cmake --- core/cmake/FindGTest.cmake | 52 ++++++++++++++-------------- core/src/index/cmake/FindGTest.cmake | 52 ++++++++++++++-------------- 2 files changed, 52 insertions(+), 52 deletions(-) diff --git a/core/cmake/FindGTest.cmake b/core/cmake/FindGTest.cmake index d746e40b05..3e14b49b0e 100644 --- a/core/cmake/FindGTest.cmake +++ b/core/cmake/FindGTest.cmake @@ -28,10 +28,10 @@ Imported targets This module defines the following :prop_tgt:`IMPORTED` targets: -``GTest::GTest`` +``gtest`` The Google Test ``gtest`` library, if found; adds Thread::Thread automatically -``GTest::Main`` +``gtest_main`` The Google Test ``gtest_main`` library, if found ``GMock::GMock`` The Google Mock ``gmock`` library, if found @@ -80,7 +80,7 @@ Example usage find_package(GTest REQUIRED) add_executable(foo foo.cc) - target_link_libraries(foo GTest::GTest GTest::Main) + target_link_libraries(foo gtest gtest_main) add_test(AllTestsInFoo foo) @@ -228,51 +228,51 @@ if(GTEST_FOUND) find_package(Threads QUIET) - if(NOT TARGET GTest::GTest) + if(NOT TARGET gtest) __gtest_determine_library_type(GTEST_LIBRARY) - add_library(GTest::GTest ${GTEST_LIBRARY_TYPE} IMPORTED) + add_library(gtest ${GTEST_LIBRARY_TYPE} IMPORTED) if(TARGET Threads::Threads) - set_target_properties(GTest::GTest PROPERTIES + set_target_properties(gtest PROPERTIES INTERFACE_LINK_LIBRARIES Threads::Threads) endif() if(GTEST_LIBRARY_TYPE STREQUAL "SHARED") - set_target_properties(GTest::GTest PROPERTIES + set_target_properties(gtest PROPERTIES INTERFACE_COMPILE_DEFINITIONS "GTEST_LINKED_AS_SHARED_LIBRARY=1") endif() if(GTEST_INCLUDE_DIRS) - set_target_properties(GTest::GTest PROPERTIES + set_target_properties(gtest PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${GTEST_INCLUDE_DIRS}") endif() - __gtest_import_library(GTest::GTest GTEST_LIBRARY "") - __gtest_import_library(GTest::GTest GTEST_LIBRARY "RELEASE") - __gtest_import_library(GTest::GTest GTEST_LIBRARY "DEBUG") + __gtest_import_library(gtest GTEST_LIBRARY "") + __gtest_import_library(gtest GTEST_LIBRARY "RELEASE") + __gtest_import_library(gtest GTEST_LIBRARY "DEBUG") endif() - if(NOT TARGET GTest::Main) + if(NOT TARGET gtest_main) __gtest_determine_library_type(GTEST_MAIN_LIBRARY) - add_library(GTest::Main ${GTEST_MAIN_LIBRARY_TYPE} IMPORTED) - set_target_properties(GTest::Main PROPERTIES - INTERFACE_LINK_LIBRARIES "GTest::GTest") - __gtest_import_library(GTest::Main GTEST_MAIN_LIBRARY "") - __gtest_import_library(GTest::Main GTEST_MAIN_LIBRARY "RELEASE") - __gtest_import_library(GTest::Main GTEST_MAIN_LIBRARY "DEBUG") + add_library(gtest_main ${GTEST_MAIN_LIBRARY_TYPE} IMPORTED) + set_target_properties(gtest_main PROPERTIES + INTERFACE_LINK_LIBRARIES "gtest") + __gtest_import_library(gtest_main GTEST_MAIN_LIBRARY "") + __gtest_import_library(gtest_main GTEST_MAIN_LIBRARY "RELEASE") + __gtest_import_library(gtest_main GTEST_MAIN_LIBRARY "DEBUG") endif() - if(NOT TARGET GTest::GMock) + if(NOT TARGET gmock) __gtest_determine_library_type(GMOCK_LIBRARY) - add_library(GTest::GMock ${GMOCK_LIBRARY_TYPE} IMPORTED) + add_library(gmock ${GMOCK_LIBRARY_TYPE} IMPORTED) if(TARGET Threads::Threads) - set_target_properties(GTest::GMock PROPERTIES + set_target_properties(gmock PROPERTIES INTERFACE_LINK_LIBRARIES Threads::Threads) endif() if(GMOCK_LIBRARY_TYPE STREQUAL "SHARED") - set_target_properties(GTest::GMock PROPERTIES + set_target_properties(gmock PROPERTIES INTERFACE_COMPILE_DEFINITIONS "GTEST_LINKED_AS_SHARED_LIBRARY=1") endif() if(GTEST_INCLUDE_DIRS) - set_target_properties(GTest::GMock PROPERTIES + set_target_properties(gmock PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${GTEST_INCLUDE_DIRS}") endif() - __gtest_import_library(GTest::GMock GMOCK_LIBRARY "") - __gtest_import_library(GTest::GMock GMOCK_LIBRARY "RELEASE") - __gtest_import_library(GTest::GMock GMOCK_LIBRARY "DEBUG") + __gtest_import_library(gmock GMOCK_LIBRARY "") + __gtest_import_library(gmock GMOCK_LIBRARY "RELEASE") + __gtest_import_library(gmock GMOCK_LIBRARY "DEBUG") endif() endif() diff --git a/core/src/index/cmake/FindGTest.cmake b/core/src/index/cmake/FindGTest.cmake index d746e40b05..3e14b49b0e 100644 --- a/core/src/index/cmake/FindGTest.cmake +++ b/core/src/index/cmake/FindGTest.cmake @@ -28,10 +28,10 @@ Imported targets This module defines the following :prop_tgt:`IMPORTED` targets: -``GTest::GTest`` +``gtest`` The Google Test ``gtest`` library, if found; adds Thread::Thread automatically -``GTest::Main`` +``gtest_main`` The Google Test ``gtest_main`` library, if found ``GMock::GMock`` The Google Mock ``gmock`` library, if found @@ -80,7 +80,7 @@ Example usage find_package(GTest REQUIRED) add_executable(foo foo.cc) - target_link_libraries(foo GTest::GTest GTest::Main) + target_link_libraries(foo gtest gtest_main) add_test(AllTestsInFoo foo) @@ -228,51 +228,51 @@ if(GTEST_FOUND) find_package(Threads QUIET) - if(NOT TARGET GTest::GTest) + if(NOT TARGET gtest) __gtest_determine_library_type(GTEST_LIBRARY) - add_library(GTest::GTest ${GTEST_LIBRARY_TYPE} IMPORTED) + add_library(gtest ${GTEST_LIBRARY_TYPE} IMPORTED) if(TARGET Threads::Threads) - set_target_properties(GTest::GTest PROPERTIES + set_target_properties(gtest PROPERTIES INTERFACE_LINK_LIBRARIES Threads::Threads) endif() if(GTEST_LIBRARY_TYPE STREQUAL "SHARED") - set_target_properties(GTest::GTest PROPERTIES + set_target_properties(gtest PROPERTIES INTERFACE_COMPILE_DEFINITIONS "GTEST_LINKED_AS_SHARED_LIBRARY=1") endif() if(GTEST_INCLUDE_DIRS) - set_target_properties(GTest::GTest PROPERTIES + set_target_properties(gtest PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${GTEST_INCLUDE_DIRS}") endif() - __gtest_import_library(GTest::GTest GTEST_LIBRARY "") - __gtest_import_library(GTest::GTest GTEST_LIBRARY "RELEASE") - __gtest_import_library(GTest::GTest GTEST_LIBRARY "DEBUG") + __gtest_import_library(gtest GTEST_LIBRARY "") + __gtest_import_library(gtest GTEST_LIBRARY "RELEASE") + __gtest_import_library(gtest GTEST_LIBRARY "DEBUG") endif() - if(NOT TARGET GTest::Main) + if(NOT TARGET gtest_main) __gtest_determine_library_type(GTEST_MAIN_LIBRARY) - add_library(GTest::Main ${GTEST_MAIN_LIBRARY_TYPE} IMPORTED) - set_target_properties(GTest::Main PROPERTIES - INTERFACE_LINK_LIBRARIES "GTest::GTest") - __gtest_import_library(GTest::Main GTEST_MAIN_LIBRARY "") - __gtest_import_library(GTest::Main GTEST_MAIN_LIBRARY "RELEASE") - __gtest_import_library(GTest::Main GTEST_MAIN_LIBRARY "DEBUG") + add_library(gtest_main ${GTEST_MAIN_LIBRARY_TYPE} IMPORTED) + set_target_properties(gtest_main PROPERTIES + INTERFACE_LINK_LIBRARIES "gtest") + __gtest_import_library(gtest_main GTEST_MAIN_LIBRARY "") + __gtest_import_library(gtest_main GTEST_MAIN_LIBRARY "RELEASE") + __gtest_import_library(gtest_main GTEST_MAIN_LIBRARY "DEBUG") endif() - if(NOT TARGET GTest::GMock) + if(NOT TARGET gmock) __gtest_determine_library_type(GMOCK_LIBRARY) - add_library(GTest::GMock ${GMOCK_LIBRARY_TYPE} IMPORTED) + add_library(gmock ${GMOCK_LIBRARY_TYPE} IMPORTED) if(TARGET Threads::Threads) - set_target_properties(GTest::GMock PROPERTIES + set_target_properties(gmock PROPERTIES INTERFACE_LINK_LIBRARIES Threads::Threads) endif() if(GMOCK_LIBRARY_TYPE STREQUAL "SHARED") - set_target_properties(GTest::GMock PROPERTIES + set_target_properties(gmock PROPERTIES INTERFACE_COMPILE_DEFINITIONS "GTEST_LINKED_AS_SHARED_LIBRARY=1") endif() if(GTEST_INCLUDE_DIRS) - set_target_properties(GTest::GMock PROPERTIES + set_target_properties(gmock PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${GTEST_INCLUDE_DIRS}") endif() - __gtest_import_library(GTest::GMock GMOCK_LIBRARY "") - __gtest_import_library(GTest::GMock GMOCK_LIBRARY "RELEASE") - __gtest_import_library(GTest::GMock GMOCK_LIBRARY "DEBUG") + __gtest_import_library(gmock GMOCK_LIBRARY "") + __gtest_import_library(gmock GMOCK_LIBRARY "RELEASE") + __gtest_import_library(gmock GMOCK_LIBRARY "DEBUG") endif() endif() From 9c954b562ab8c8f3204235ee03e589f8874b0ac4 Mon Sep 17 00:00:00 2001 From: ZhifengZhang-CN Date: Tue, 12 Nov 2019 17:30:19 +0800 Subject: [PATCH 072/103] delete FindGTest.cmake --- ci/scripts/build.sh | 1 - ci/travis/install_dependency.sh | 1 - core/cmake/FindGTest.cmake | 278 --------------------------- core/src/index/cmake/FindGTest.cmake | 278 --------------------------- 4 files changed, 558 deletions(-) delete mode 100644 core/cmake/FindGTest.cmake delete mode 100644 core/src/index/cmake/FindGTest.cmake diff --git a/ci/scripts/build.sh b/ci/scripts/build.sh index 0875ac1dd1..a8bd339ff5 100755 --- a/ci/scripts/build.sh +++ b/ci/scripts/build.sh @@ -96,7 +96,6 @@ CMAKE_CMD="cmake \ -DUSE_JFROG_CACHE=${USE_JFROG_CACHE} \ -DBUILD_FAISS_WITH_MKL=${WITH_MKL} \ -DArrow_SOURCE=AUTO \ --DGTest_SOURCE=AUTO \ ${MILVUS_CORE_DIR}" echo ${CMAKE_CMD} ${CMAKE_CMD} diff --git a/ci/travis/install_dependency.sh b/ci/travis/install_dependency.sh index 95d7a698db..e9efd1f441 100755 --- a/ci/travis/install_dependency.sh +++ b/ci/travis/install_dependency.sh @@ -25,7 +25,6 @@ sudo apt-get install -y -q --no-install-recommends \ automake \ ccache \ pkg-config \ - libgtest-dev \ libarrow-dev \ libjemalloc-dev \ libboost-serialization-dev \ diff --git a/core/cmake/FindGTest.cmake b/core/cmake/FindGTest.cmake deleted file mode 100644 index 3e14b49b0e..0000000000 --- a/core/cmake/FindGTest.cmake +++ /dev/null @@ -1,278 +0,0 @@ -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -# Originally imported from the CMake project at commit -# df4ed1e9ffcdb6b99ccff9e6f44808fdd2abda56 with the following license header: -# -# Distributed under the OSI-approved BSD 3-Clause License. See accompanying -# file Copyright.txt or https://cmake.org/licensing for details. - - -#[=======================================================================[.rst: -FindGTest ---------- - -Locate the Google C++ Testing Framework. - -Imported targets -^^^^^^^^^^^^^^^^ - -This module defines the following :prop_tgt:`IMPORTED` targets: - -``gtest`` - The Google Test ``gtest`` library, if found; adds Thread::Thread - automatically -``gtest_main`` - The Google Test ``gtest_main`` library, if found -``GMock::GMock`` - The Google Mock ``gmock`` library, if found - - -Result variables -^^^^^^^^^^^^^^^^ - -This module will set the following variables in your project: - -``GTEST_FOUND`` - Found the Google Testing framework -``GTEST_INCLUDE_DIRS`` - the directory containing the Google Test headers - -The library variables below are set as normal variables. These -contain debug/optimized keywords when a debugging library is found. - -``GTEST_LIBRARIES`` - The Google Test ``gtest`` library; note it also requires linking - with an appropriate thread library -``GTEST_MAIN_LIBRARIES`` - The Google Test ``gtest_main`` library -``GTEST_BOTH_LIBRARIES`` - Both ``gtest`` and ``gtest_main`` - -Cache variables -^^^^^^^^^^^^^^^ - -The following cache variables may also be set: - -``GTEST_ROOT`` - The root directory of the Google Test installation (may also be - set as an environment variable) -``GTEST_MSVC_SEARCH`` - If compiling with MSVC, this variable can be set to ``MT`` or - ``MD`` (the default) to enable searching a GTest build tree - - -Example usage -^^^^^^^^^^^^^ - -:: - - enable_testing() - find_package(GTest REQUIRED) - - add_executable(foo foo.cc) - target_link_libraries(foo gtest gtest_main) - - add_test(AllTestsInFoo foo) - - -Deeper integration with CTest -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -See :module:`GoogleTest` for information on the :command:`gtest_add_tests` -and :command:`gtest_discover_tests` commands. -#]=======================================================================] - -# include(${CMAKE_CURRENT_LIST_DIR}/GoogleTest.cmake) - -function(__gtest_append_debugs _endvar _library) - if(${_library} AND ${_library}_DEBUG) - set(_output optimized ${${_library}} debug ${${_library}_DEBUG}) - else() - set(_output ${${_library}}) - endif() - set(${_endvar} ${_output} PARENT_SCOPE) -endfunction() - -function(__gtest_find_library _name) - find_library(${_name} - NAMES ${ARGN} - HINTS - ENV GTEST_ROOT - ${GTEST_ROOT} - PATH_SUFFIXES ${_gtest_libpath_suffixes} - ) - mark_as_advanced(${_name}) -endfunction() - -macro(__gtest_determine_windows_library_type _var) - if(EXISTS "${${_var}}") - file(TO_NATIVE_PATH "${${_var}}" _lib_path) - get_filename_component(_name "${${_var}}" NAME_WE) - file(STRINGS "${${_var}}" _match REGEX "${_name}\\.dll" LIMIT_COUNT 1) - if(NOT _match STREQUAL "") - set(${_var}_TYPE SHARED PARENT_SCOPE) - else() - set(${_var}_TYPE UNKNOWN PARENT_SCOPE) - endif() - return() - endif() -endmacro() - -function(__gtest_determine_library_type _var) - if(WIN32) - # For now, at least, only Windows really needs to know the library type - __gtest_determine_windows_library_type(${_var}) - __gtest_determine_windows_library_type(${_var}_RELEASE) - __gtest_determine_windows_library_type(${_var}_DEBUG) - endif() - # If we get here, no determination was made from the above checks - set(${_var}_TYPE UNKNOWN PARENT_SCOPE) -endfunction() - -function(__gtest_import_library _target _var _config) - if(_config) - set(_config_suffix "_${_config}") - else() - set(_config_suffix "") - endif() - - set(_lib "${${_var}${_config_suffix}}") - if(EXISTS "${_lib}") - if(_config) - set_property(TARGET ${_target} APPEND PROPERTY - IMPORTED_CONFIGURATIONS ${_config}) - endif() - set_target_properties(${_target} PROPERTIES - IMPORTED_LINK_INTERFACE_LANGUAGES${_config_suffix} "CXX") - if(WIN32 AND ${_var}_TYPE STREQUAL SHARED) - set_target_properties(${_target} PROPERTIES - IMPORTED_IMPLIB${_config_suffix} "${_lib}") - else() - set_target_properties(${_target} PROPERTIES - IMPORTED_LOCATION${_config_suffix} "${_lib}") - endif() - endif() -endfunction() - -# - -if(NOT DEFINED GTEST_MSVC_SEARCH) - set(GTEST_MSVC_SEARCH MD) -endif() - -set(_gtest_libpath_suffixes lib) -if(MSVC) - if(GTEST_MSVC_SEARCH STREQUAL "MD") - list(APPEND _gtest_libpath_suffixes - msvc/gtest-md/Debug - msvc/gtest-md/Release - msvc/x64/Debug - msvc/x64/Release - ) - elseif(GTEST_MSVC_SEARCH STREQUAL "MT") - list(APPEND _gtest_libpath_suffixes - msvc/gtest/Debug - msvc/gtest/Release - msvc/x64/Debug - msvc/x64/Release - ) - endif() -endif() - - -find_path(GTEST_INCLUDE_DIR gtest/gtest.h - HINTS - $ENV{GTEST_ROOT}/include - ${GTEST_ROOT}/include - PATH_SUFFIXES ${LIB_PATH_SUFFIXES} -) -mark_as_advanced(GTEST_INCLUDE_DIR) - -if(MSVC AND GTEST_MSVC_SEARCH STREQUAL "MD") - # The provided /MD project files for Google Test add -md suffixes to the - # library names. - __gtest_find_library(GTEST_LIBRARY gtest-md gtest) - __gtest_find_library(GTEST_LIBRARY_DEBUG gtest-mdd gtestd) - __gtest_find_library(GTEST_MAIN_LIBRARY gtest_main-md gtest_main) - __gtest_find_library(GTEST_MAIN_LIBRARY_DEBUG gtest_main-mdd gtest_maind) - __gtest_find_library(GMOCK_LIBRARY gmock-md gmock) - __gtest_find_library(GMOCK_LIBRARY_DEBUG gmock-mdd gmockd) -else() - __gtest_find_library(GTEST_LIBRARY gtest) - __gtest_find_library(GTEST_LIBRARY_DEBUG gtestd) - __gtest_find_library(GTEST_MAIN_LIBRARY gtest_main) - __gtest_find_library(GTEST_MAIN_LIBRARY_DEBUG gtest_maind) - __gtest_find_library(GMOCK_LIBRARY gmock) - __gtest_find_library(GMOCK_LIBRARY_DEBUG gtestd) -endif() - -include(FindPackageHandleStandardArgs) -FIND_PACKAGE_HANDLE_STANDARD_ARGS(GTest DEFAULT_MSG GTEST_LIBRARY GTEST_INCLUDE_DIR GTEST_MAIN_LIBRARY GMOCK_LIBRARY) - -if(GTEST_FOUND) - set(GTEST_INCLUDE_DIRS ${GTEST_INCLUDE_DIR}) - __gtest_append_debugs(GTEST_LIBRARIES GTEST_LIBRARY) - __gtest_append_debugs(GTEST_MAIN_LIBRARIES GTEST_MAIN_LIBRARY) - __gtest_append_debugs(GMOCK_LIBRARIES GMOCK_LIBRARY) - set(GTEST_BOTH_LIBRARIES ${GTEST_LIBRARIES} ${GTEST_MAIN_LIBRARIES}) - - find_package(Threads QUIET) - - if(NOT TARGET gtest) - __gtest_determine_library_type(GTEST_LIBRARY) - add_library(gtest ${GTEST_LIBRARY_TYPE} IMPORTED) - if(TARGET Threads::Threads) - set_target_properties(gtest PROPERTIES - INTERFACE_LINK_LIBRARIES Threads::Threads) - endif() - if(GTEST_LIBRARY_TYPE STREQUAL "SHARED") - set_target_properties(gtest PROPERTIES - INTERFACE_COMPILE_DEFINITIONS "GTEST_LINKED_AS_SHARED_LIBRARY=1") - endif() - if(GTEST_INCLUDE_DIRS) - set_target_properties(gtest PROPERTIES - INTERFACE_INCLUDE_DIRECTORIES "${GTEST_INCLUDE_DIRS}") - endif() - __gtest_import_library(gtest GTEST_LIBRARY "") - __gtest_import_library(gtest GTEST_LIBRARY "RELEASE") - __gtest_import_library(gtest GTEST_LIBRARY "DEBUG") - endif() - if(NOT TARGET gtest_main) - __gtest_determine_library_type(GTEST_MAIN_LIBRARY) - add_library(gtest_main ${GTEST_MAIN_LIBRARY_TYPE} IMPORTED) - set_target_properties(gtest_main PROPERTIES - INTERFACE_LINK_LIBRARIES "gtest") - __gtest_import_library(gtest_main GTEST_MAIN_LIBRARY "") - __gtest_import_library(gtest_main GTEST_MAIN_LIBRARY "RELEASE") - __gtest_import_library(gtest_main GTEST_MAIN_LIBRARY "DEBUG") - endif() - if(NOT TARGET gmock) - __gtest_determine_library_type(GMOCK_LIBRARY) - add_library(gmock ${GMOCK_LIBRARY_TYPE} IMPORTED) - if(TARGET Threads::Threads) - set_target_properties(gmock PROPERTIES - INTERFACE_LINK_LIBRARIES Threads::Threads) - endif() - if(GMOCK_LIBRARY_TYPE STREQUAL "SHARED") - set_target_properties(gmock PROPERTIES - INTERFACE_COMPILE_DEFINITIONS "GTEST_LINKED_AS_SHARED_LIBRARY=1") - endif() - if(GTEST_INCLUDE_DIRS) - set_target_properties(gmock PROPERTIES - INTERFACE_INCLUDE_DIRECTORIES "${GTEST_INCLUDE_DIRS}") - endif() - __gtest_import_library(gmock GMOCK_LIBRARY "") - __gtest_import_library(gmock GMOCK_LIBRARY "RELEASE") - __gtest_import_library(gmock GMOCK_LIBRARY "DEBUG") - endif() -endif() diff --git a/core/src/index/cmake/FindGTest.cmake b/core/src/index/cmake/FindGTest.cmake deleted file mode 100644 index 3e14b49b0e..0000000000 --- a/core/src/index/cmake/FindGTest.cmake +++ /dev/null @@ -1,278 +0,0 @@ -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -# Originally imported from the CMake project at commit -# df4ed1e9ffcdb6b99ccff9e6f44808fdd2abda56 with the following license header: -# -# Distributed under the OSI-approved BSD 3-Clause License. See accompanying -# file Copyright.txt or https://cmake.org/licensing for details. - - -#[=======================================================================[.rst: -FindGTest ---------- - -Locate the Google C++ Testing Framework. - -Imported targets -^^^^^^^^^^^^^^^^ - -This module defines the following :prop_tgt:`IMPORTED` targets: - -``gtest`` - The Google Test ``gtest`` library, if found; adds Thread::Thread - automatically -``gtest_main`` - The Google Test ``gtest_main`` library, if found -``GMock::GMock`` - The Google Mock ``gmock`` library, if found - - -Result variables -^^^^^^^^^^^^^^^^ - -This module will set the following variables in your project: - -``GTEST_FOUND`` - Found the Google Testing framework -``GTEST_INCLUDE_DIRS`` - the directory containing the Google Test headers - -The library variables below are set as normal variables. These -contain debug/optimized keywords when a debugging library is found. - -``GTEST_LIBRARIES`` - The Google Test ``gtest`` library; note it also requires linking - with an appropriate thread library -``GTEST_MAIN_LIBRARIES`` - The Google Test ``gtest_main`` library -``GTEST_BOTH_LIBRARIES`` - Both ``gtest`` and ``gtest_main`` - -Cache variables -^^^^^^^^^^^^^^^ - -The following cache variables may also be set: - -``GTEST_ROOT`` - The root directory of the Google Test installation (may also be - set as an environment variable) -``GTEST_MSVC_SEARCH`` - If compiling with MSVC, this variable can be set to ``MT`` or - ``MD`` (the default) to enable searching a GTest build tree - - -Example usage -^^^^^^^^^^^^^ - -:: - - enable_testing() - find_package(GTest REQUIRED) - - add_executable(foo foo.cc) - target_link_libraries(foo gtest gtest_main) - - add_test(AllTestsInFoo foo) - - -Deeper integration with CTest -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -See :module:`GoogleTest` for information on the :command:`gtest_add_tests` -and :command:`gtest_discover_tests` commands. -#]=======================================================================] - -# include(${CMAKE_CURRENT_LIST_DIR}/GoogleTest.cmake) - -function(__gtest_append_debugs _endvar _library) - if(${_library} AND ${_library}_DEBUG) - set(_output optimized ${${_library}} debug ${${_library}_DEBUG}) - else() - set(_output ${${_library}}) - endif() - set(${_endvar} ${_output} PARENT_SCOPE) -endfunction() - -function(__gtest_find_library _name) - find_library(${_name} - NAMES ${ARGN} - HINTS - ENV GTEST_ROOT - ${GTEST_ROOT} - PATH_SUFFIXES ${_gtest_libpath_suffixes} - ) - mark_as_advanced(${_name}) -endfunction() - -macro(__gtest_determine_windows_library_type _var) - if(EXISTS "${${_var}}") - file(TO_NATIVE_PATH "${${_var}}" _lib_path) - get_filename_component(_name "${${_var}}" NAME_WE) - file(STRINGS "${${_var}}" _match REGEX "${_name}\\.dll" LIMIT_COUNT 1) - if(NOT _match STREQUAL "") - set(${_var}_TYPE SHARED PARENT_SCOPE) - else() - set(${_var}_TYPE UNKNOWN PARENT_SCOPE) - endif() - return() - endif() -endmacro() - -function(__gtest_determine_library_type _var) - if(WIN32) - # For now, at least, only Windows really needs to know the library type - __gtest_determine_windows_library_type(${_var}) - __gtest_determine_windows_library_type(${_var}_RELEASE) - __gtest_determine_windows_library_type(${_var}_DEBUG) - endif() - # If we get here, no determination was made from the above checks - set(${_var}_TYPE UNKNOWN PARENT_SCOPE) -endfunction() - -function(__gtest_import_library _target _var _config) - if(_config) - set(_config_suffix "_${_config}") - else() - set(_config_suffix "") - endif() - - set(_lib "${${_var}${_config_suffix}}") - if(EXISTS "${_lib}") - if(_config) - set_property(TARGET ${_target} APPEND PROPERTY - IMPORTED_CONFIGURATIONS ${_config}) - endif() - set_target_properties(${_target} PROPERTIES - IMPORTED_LINK_INTERFACE_LANGUAGES${_config_suffix} "CXX") - if(WIN32 AND ${_var}_TYPE STREQUAL SHARED) - set_target_properties(${_target} PROPERTIES - IMPORTED_IMPLIB${_config_suffix} "${_lib}") - else() - set_target_properties(${_target} PROPERTIES - IMPORTED_LOCATION${_config_suffix} "${_lib}") - endif() - endif() -endfunction() - -# - -if(NOT DEFINED GTEST_MSVC_SEARCH) - set(GTEST_MSVC_SEARCH MD) -endif() - -set(_gtest_libpath_suffixes lib) -if(MSVC) - if(GTEST_MSVC_SEARCH STREQUAL "MD") - list(APPEND _gtest_libpath_suffixes - msvc/gtest-md/Debug - msvc/gtest-md/Release - msvc/x64/Debug - msvc/x64/Release - ) - elseif(GTEST_MSVC_SEARCH STREQUAL "MT") - list(APPEND _gtest_libpath_suffixes - msvc/gtest/Debug - msvc/gtest/Release - msvc/x64/Debug - msvc/x64/Release - ) - endif() -endif() - - -find_path(GTEST_INCLUDE_DIR gtest/gtest.h - HINTS - $ENV{GTEST_ROOT}/include - ${GTEST_ROOT}/include - PATH_SUFFIXES ${LIB_PATH_SUFFIXES} -) -mark_as_advanced(GTEST_INCLUDE_DIR) - -if(MSVC AND GTEST_MSVC_SEARCH STREQUAL "MD") - # The provided /MD project files for Google Test add -md suffixes to the - # library names. - __gtest_find_library(GTEST_LIBRARY gtest-md gtest) - __gtest_find_library(GTEST_LIBRARY_DEBUG gtest-mdd gtestd) - __gtest_find_library(GTEST_MAIN_LIBRARY gtest_main-md gtest_main) - __gtest_find_library(GTEST_MAIN_LIBRARY_DEBUG gtest_main-mdd gtest_maind) - __gtest_find_library(GMOCK_LIBRARY gmock-md gmock) - __gtest_find_library(GMOCK_LIBRARY_DEBUG gmock-mdd gmockd) -else() - __gtest_find_library(GTEST_LIBRARY gtest) - __gtest_find_library(GTEST_LIBRARY_DEBUG gtestd) - __gtest_find_library(GTEST_MAIN_LIBRARY gtest_main) - __gtest_find_library(GTEST_MAIN_LIBRARY_DEBUG gtest_maind) - __gtest_find_library(GMOCK_LIBRARY gmock) - __gtest_find_library(GMOCK_LIBRARY_DEBUG gtestd) -endif() - -include(FindPackageHandleStandardArgs) -FIND_PACKAGE_HANDLE_STANDARD_ARGS(GTest DEFAULT_MSG GTEST_LIBRARY GTEST_INCLUDE_DIR GTEST_MAIN_LIBRARY GMOCK_LIBRARY) - -if(GTEST_FOUND) - set(GTEST_INCLUDE_DIRS ${GTEST_INCLUDE_DIR}) - __gtest_append_debugs(GTEST_LIBRARIES GTEST_LIBRARY) - __gtest_append_debugs(GTEST_MAIN_LIBRARIES GTEST_MAIN_LIBRARY) - __gtest_append_debugs(GMOCK_LIBRARIES GMOCK_LIBRARY) - set(GTEST_BOTH_LIBRARIES ${GTEST_LIBRARIES} ${GTEST_MAIN_LIBRARIES}) - - find_package(Threads QUIET) - - if(NOT TARGET gtest) - __gtest_determine_library_type(GTEST_LIBRARY) - add_library(gtest ${GTEST_LIBRARY_TYPE} IMPORTED) - if(TARGET Threads::Threads) - set_target_properties(gtest PROPERTIES - INTERFACE_LINK_LIBRARIES Threads::Threads) - endif() - if(GTEST_LIBRARY_TYPE STREQUAL "SHARED") - set_target_properties(gtest PROPERTIES - INTERFACE_COMPILE_DEFINITIONS "GTEST_LINKED_AS_SHARED_LIBRARY=1") - endif() - if(GTEST_INCLUDE_DIRS) - set_target_properties(gtest PROPERTIES - INTERFACE_INCLUDE_DIRECTORIES "${GTEST_INCLUDE_DIRS}") - endif() - __gtest_import_library(gtest GTEST_LIBRARY "") - __gtest_import_library(gtest GTEST_LIBRARY "RELEASE") - __gtest_import_library(gtest GTEST_LIBRARY "DEBUG") - endif() - if(NOT TARGET gtest_main) - __gtest_determine_library_type(GTEST_MAIN_LIBRARY) - add_library(gtest_main ${GTEST_MAIN_LIBRARY_TYPE} IMPORTED) - set_target_properties(gtest_main PROPERTIES - INTERFACE_LINK_LIBRARIES "gtest") - __gtest_import_library(gtest_main GTEST_MAIN_LIBRARY "") - __gtest_import_library(gtest_main GTEST_MAIN_LIBRARY "RELEASE") - __gtest_import_library(gtest_main GTEST_MAIN_LIBRARY "DEBUG") - endif() - if(NOT TARGET gmock) - __gtest_determine_library_type(GMOCK_LIBRARY) - add_library(gmock ${GMOCK_LIBRARY_TYPE} IMPORTED) - if(TARGET Threads::Threads) - set_target_properties(gmock PROPERTIES - INTERFACE_LINK_LIBRARIES Threads::Threads) - endif() - if(GMOCK_LIBRARY_TYPE STREQUAL "SHARED") - set_target_properties(gmock PROPERTIES - INTERFACE_COMPILE_DEFINITIONS "GTEST_LINKED_AS_SHARED_LIBRARY=1") - endif() - if(GTEST_INCLUDE_DIRS) - set_target_properties(gmock PROPERTIES - INTERFACE_INCLUDE_DIRECTORIES "${GTEST_INCLUDE_DIRS}") - endif() - __gtest_import_library(gmock GMOCK_LIBRARY "") - __gtest_import_library(gmock GMOCK_LIBRARY "RELEASE") - __gtest_import_library(gmock GMOCK_LIBRARY "DEBUG") - endif() -endif() From 0d254639b0bf5647217506e323d8e131aaef2c86 Mon Sep 17 00:00:00 2001 From: "yudong.cai" Date: Tue, 12 Nov 2019 18:16:58 +0800 Subject: [PATCH 073/103] #89 update faiss benchmark to support IDMap --- .../faiss_benchmark/faiss_benchmark_test.cpp | 95 +++++++++++-------- 1 file changed, 54 insertions(+), 41 deletions(-) diff --git a/core/src/index/unittest/faiss_benchmark/faiss_benchmark_test.cpp b/core/src/index/unittest/faiss_benchmark/faiss_benchmark_test.cpp index 780e620e6a..5e9d45f75b 100644 --- a/core/src/index/unittest/faiss_benchmark/faiss_benchmark_test.cpp +++ b/core/src/index/unittest/faiss_benchmark/faiss_benchmark_test.cpp @@ -202,14 +202,14 @@ parse_ann_test_name(const std::string& ann_test_name, size_t& dim, faiss::Metric int32_t GetResultHitCount(const faiss::Index::idx_t* ground_index, const faiss::Index::idx_t* index, size_t ground_k, size_t k, size_t nq, int32_t index_add_loops) { - assert(ground_k <= k); + size_t min_k = std::min(ground_k, k); int hit = 0; for (int i = 0; i < nq; i++) { // count the num of results exist in ground truth result set // each result replicates INDEX_ADD_LOOPS times for (int j_c = 0; j_c < k; j_c++) { int r_c = index[i * k + j_c]; - for (int j_g = 0; j_g < ground_k / index_add_loops; j_g++) { + for (int j_g = 0; j_g < min_k / index_add_loops; j_g++) { if (ground_index[i * ground_k + j_g] == r_c) { hit++; continue; @@ -283,7 +283,11 @@ load_base_data(faiss::Index*& index, const std::string& ann_test_name, const std // add index multiple times to get ~1G data set for (int i = 0; i < index_add_loops; i++) { printf("[%.3f s] No.%d Indexing database, size %ld*%ld\n", elapsed() - t0, i, nb, d); - gpu_index->add(nb, xb); + std::vector xids(nb); + for (int t = 0; t < nb; t++) { + xids[t] = i * nb + t; + } + gpu_index->add_with_ids(nb, xb, xids.data()); } printf("[%.3f s] Coping GPU index to CPU\n", elapsed() - t0); @@ -359,13 +363,14 @@ test_with_nprobes(const std::string& ann_test_name, const std::string& index_key const int32_t search_loops) { double t0 = elapsed(); - const size_t NQ = 1000, NQ_START = 10, NQ_STEP = 10; - const size_t K = 1000, K_START = 100, K_STEP = 10; + const std::vector NQ = {10, 100}; + const std::vector K = {10, 100, 1000}; const size_t GK = 100; // topk of ground truth std::unordered_map mode_str_map = { {MODE_CPU, "MODE_CPU"}, {MODE_MIX, "MODE_MIX"}, {MODE_GPU, "MODE_GPU"}}; + double copy_time = 0.0; faiss::Index *gpu_index, *index; if (query_mode != MODE_CPU) { faiss::gpu::GpuClonerOptions option; @@ -375,7 +380,6 @@ test_with_nprobes(const std::string& ann_test_name, const std::string& index_key index_composition.index = cpu_index; index_composition.quantizer = nullptr; - double copy_time; switch (query_mode) { case MODE_MIX: { index_composition.mode = 1; // 0: all data, 1: copy quantizer, 2: copy data @@ -420,34 +424,39 @@ test_with_nprobes(const std::string& ann_test_name, const std::string& index_key } for (auto nprobe : nprobes) { - switch (query_mode) { - case MODE_CPU: - case MODE_MIX: { - faiss::ParameterSpace params; - std::string nprobe_str = "nprobe=" + std::to_string(nprobe); - params.set_index_parameters(index, nprobe_str.c_str()); - break; - } - case MODE_GPU: { - faiss::gpu::GpuIndexIVF* gpu_index_ivf = dynamic_cast(index); - gpu_index_ivf->setNumProbes(nprobe); + // brute-force need not set nprobe + if (index_key.find("IDMap") == std::string::npos) { + switch (query_mode) { + case MODE_CPU: + case MODE_MIX: { + faiss::ParameterSpace params; + std::string nprobe_str = "nprobe=" + std::to_string(nprobe); + params.set_index_parameters(index, nprobe_str.c_str()); + break; + } + case MODE_GPU: { + faiss::gpu::GpuIndexIVF *gpu_index_ivf = dynamic_cast(index); + gpu_index_ivf->setNumProbes(nprobe); + } } } // output buffers - faiss::Index::idx_t* I = new faiss::Index::idx_t[NQ * K]; - faiss::Index::distance_t* D = new faiss::Index::distance_t[NQ * K]; + faiss::Index::idx_t* I = new faiss::Index::idx_t[NQ.back() * K.back()]; + faiss::Index::distance_t* D = new faiss::Index::distance_t[NQ.back() * K.back()]; printf("\n%s | %s - %s | nprobe=%lu\n", ann_test_name.c_str(), index_key.c_str(), mode_str_map[query_mode].c_str(), nprobe); printf("======================================================================================\n"); - for (size_t t_nq = NQ_START; t_nq <= NQ; t_nq *= NQ_STEP) { // nq = {10, 100, 1000} - for (size_t t_k = K_START; t_k <= K; t_k *= K_STEP) { // k = {100, 1000} + for (size_t j = 0; j < K.size(); j++) { + size_t t_k = K[j]; + for (size_t i = 0; i < NQ.size(); i++) { + size_t t_nq = NQ[i]; faiss::indexIVF_stats.quantization_time = 0.0; faiss::indexIVF_stats.search_time = 0.0; double t_start = elapsed(), t_end; - for (int i = 0; i < search_loops; i++) { + for (int s = 0; s < search_loops; s++) { index->search(t_nq, xq, t_k, D, I); } t_end = elapsed(); @@ -466,7 +475,7 @@ test_with_nprobes(const std::string& ann_test_name, const std::string& index_key printf("nq = %4ld, k = %4ld, elapse = %.4fs (quant = %.4fs, search = %.4fs), R@ = %.4f\n", t_nq, t_k, (t_end - t_start) / search_loops, faiss::indexIVF_stats.quantization_time / 1000 / search_loops, faiss::indexIVF_stats.search_time / 1000 / search_loops, - (hit / float(t_nq * GK / index_add_loops))); + (hit / float(t_nq * std::min(GK, t_k) / index_add_loops))); } } printf("======================================================================================\n"); @@ -479,8 +488,9 @@ test_with_nprobes(const std::string& ann_test_name, const std::string& index_key } void -test_ann_hdf5(const std::string& ann_test_name, const std::string& index_type, const QueryMode query_mode, - int32_t index_add_loops, const std::vector& nprobes, int32_t search_loops) { +test_ann_hdf5(const std::string& ann_test_name, const std::string& cluster_type, const std::string& index_type, + const QueryMode query_mode, int32_t index_add_loops, const std::vector& nprobes, + int32_t search_loops) { double t0 = elapsed(); faiss::gpu::StandardGpuResources res; @@ -493,7 +503,7 @@ test_ann_hdf5(const std::string& ann_test_name, const std::string& index_type, c return; } - std::string index_key = "IVF16384," + index_type; + std::string index_key = cluster_type + "," + index_type; if (!parse_ann_test_name(ann_test_name, dim, metric_type)) { printf("Invalid ann test name: %s\n", ann_test_name.c_str()); @@ -526,7 +536,7 @@ test_ann_hdf5(const std::string& ann_test_name, const std::string& index_type, c * * Dataset Dimensions Train_size Test_size Neighbors Distance Download * Fashion- - * MNIST 784 60,000 10,000 100 Euclidean HDF5 (217MB) + MNIST 784 60,000 10,000 100 Euclidean HDF5 (217MB) * GIST 960 1,000,000 1,000 100 Euclidean HDF5 (3.6GB) * GloVe 100 1,183,514 10,000 100 Angular HDF5 (463MB) * GloVe 200 1,183,514 10,000 100 Angular HDF5 (918MB) @@ -542,30 +552,33 @@ TEST(FAISSTEST, BENCHMARK) { /////////////////////////////////////////////////////////////////////////////////////////////////////////////////// const int32_t SIFT_INSERT_LOOPS = 2; // insert twice to get ~1G data set - test_ann_hdf5("sift-128-euclidean", "Flat", MODE_CPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); - test_ann_hdf5("sift-128-euclidean", "Flat", MODE_GPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("sift-128-euclidean", "IDMap", "Flat", MODE_CPU,SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("sift-128-euclidean", "IDMap", "Flat", MODE_GPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); - test_ann_hdf5("sift-128-euclidean", "SQ8", MODE_CPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); - test_ann_hdf5("sift-128-euclidean", "SQ8", MODE_GPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("sift-128-euclidean", "IVF16384", "Flat", MODE_CPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("sift-128-euclidean", "IVF16384", "Flat", MODE_GPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + + test_ann_hdf5("sift-128-euclidean", "IVF16384", "SQ8", MODE_CPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("sift-128-euclidean", "IVF16384", "SQ8", MODE_GPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); #ifdef CUSTOMIZATION - test_ann_hdf5("sift-128-euclidean", "SQ8Hybrid", MODE_CPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); - test_ann_hdf5("sift-128-euclidean", "SQ8Hybrid", MODE_MIX, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); - test_ann_hdf5("sift-128-euclidean", "SQ8Hybrid", MODE_GPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("sift-128-euclidean", "IVF16384", "SQ8Hybrid", MODE_CPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("sift-128-euclidean", "IVF16384", "SQ8Hybrid", MODE_MIX, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("sift-128-euclidean", "IVF16384", "SQ8Hybrid", MODE_GPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); #endif /////////////////////////////////////////////////////////////////////////////////////////////////////////////////// const int32_t GLOVE_INSERT_LOOPS = 1; - test_ann_hdf5("glove-200-angular", "Flat", MODE_CPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); - test_ann_hdf5("glove-200-angular", "Flat", MODE_GPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("glove-200-angular", "IVF16384", "Flat", MODE_CPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("glove-200-angular", "IVF16384", "Flat", MODE_GPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); - test_ann_hdf5("glove-200-angular", "SQ8", MODE_CPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); - test_ann_hdf5("glove-200-angular", "SQ8", MODE_GPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("glove-200-angular", "IVF16384", "SQ8", MODE_CPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("glove-200-angular", "IVF16384", "SQ8", MODE_GPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); #ifdef CUSTOMIZATION - test_ann_hdf5("glove-200-angular", "SQ8Hybrid", MODE_CPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); - test_ann_hdf5("glove-200-angular", "SQ8Hybrid", MODE_MIX, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); - test_ann_hdf5("glove-200-angular", "SQ8Hybrid", MODE_GPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("glove-200-angular", "IVF16384", "SQ8Hybrid", MODE_CPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("glove-200-angular", "IVF16384", "SQ8Hybrid", MODE_MIX, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("glove-200-angular", "IVF16384", "SQ8Hybrid", MODE_GPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); #endif } From 4493a6248dc2f648687db94ea728d56499617e1e Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Tue, 12 Nov 2019 18:55:32 +0800 Subject: [PATCH 074/103] [skip ci] Create Mishards README --- shards/README.md | 264 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 264 insertions(+) create mode 100644 shards/README.md diff --git a/shards/README.md b/shards/README.md new file mode 100644 index 0000000000..dba6f785aa --- /dev/null +++ b/shards/README.md @@ -0,0 +1,264 @@ +# Mishards - An Experimental Sharding Middleware + +Milvus aims to achieve efficient similarity search and analytics for massive-scale vectors. A standalone Milvus instance can easily handle vector search among billion-scale vectors. However, for 10 billion, 100 billion or even larger datasets, a Milvus cluster is needed. + +Ideally, this cluster can be accessed and used just as the standalone instance, meanwhile it satisfies the business requirements such as low latency and high concurrency. + +This page meant to demonstrates how to use Mishards, an experimental sharding middleware for Milvus, to establish an orchestrated cluster. + +## What is Mishards + +Mishards is a middleware that is developed using Python. It provides unlimited extension of memory and computation capacity through request forwarding, read/write splitting, horizontal scalability and dynamic extension. It works as the proxy of the Milvus system. + +Using Mishards in Milvus cluster deployment is an experimental feature available for user test and feedback. + +## How Mishards works + +Mishards splits the upstream requests to sub-requests and forwards them to Milvus servers. When the search computation is completed, all results are collected by Mishards and sent back to the client. + +Below graph is a demonstration of the process: + +![mishards](https://raw.githubusercontent.com/milvus-io/docs/master/assets/mishards.png) + +## Mishards example codes + +Below examples codes demonstrate how to build from source code a Milvus server with Mishards on a standalone machine, as well as how to use Kubernetes to establish Milvus cluster with Mishards. + +Before executing these examples, make sure you meet the prerequisites of [Milvus installation](https://www.milvus.io/docs/en/userguide/install_milvus/). + +### Build from source code + +#### Prequisites + +Make sure Python 3.6 or higher is installed. + +#### Start Milvus and Mishards from source code + +Follow below steps to start a standalone Milvus instance with Mishards from source code: + +1. Clone milvus repository. + + ```shell + git clone + ``` + +2. Install Mishards dependencies. + + ```shell + $ cd milvus/shards + $ pip install -r requirements.txt + ``` + +3. Start Milvus server. + + ```shell + $ sudo nvidia-docker run --rm -d -p 19530:19530 -v /tmp/milvus/db:/opt/milvus/db milvusdb/milvus:0.5.0-d102119-ede20b + ``` + +4. Update path permissions. + + ```shell + $ sudo chown -R $USER:$USER /tmp/milvus + ``` + +5. Configure Mishards environmental variables. + + ```shell + $ cp mishards/.env.example mishards/.env + ``` + +6. Start Mishards server. + + ```shell + $ python mishards/main.py + ``` + +### Docker example + +The `all_in_one` example shows how to use Docker container to start 2 Milvus instances, 1 Mishards instance and 1 Jaeger instance. + + 1. Install [Docker Compose](https://docs.docker.com/compose/install/). + + 2. Build docker images for these instances. + + ```shell + $ make build + ``` + + 3. Start all instances. + + ```shell + $ make deploy + ``` + + 4. Confirm instance status. + + ```shell + $ make probe_deploy + Pass ==> Pass: Connected + Fail ==> Error: Fail connecting to server on 127.0.0.1:19530. Timeout + ``` + +To check the service tracing, open the [Jaeger page](http://127.0.0.1:16686/) on your browser. + +![jaegerui](https://raw.githubusercontent.com/milvus-io/docs/master/assets/jaegerui.png) + +![jaegertraces](https://raw.githubusercontent.com/milvus-io/docs/master/assets/jaegertraces.png) + +To stop all instances, use the following command: + +```shell +$ make clean_deploy +``` + +### Kubernetes example + +Using Kubernetes to deploy Milvus cluster requires that the developers have a basic understanding of [general concepts](https://kubernetes.io/docs/concepts/) of Kubernetes. + +This example mainly demonstrates how to use Kubernetes to establish a Milvus cluster containing 2 Milvus instances（1 read instance and 1 write instance), 1 MySQL instance and 1 Mishards instance. + +This example does not include tasks such as setting up Kubernetes cluster, [installing shared storage](https://kubernetes.io/docs/concepts/storage/volumes/) and using command tools such as [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/). + +Below is the architecture of Milvus cluster built upon Kubernetes: + +![k8s_arch](https://raw.githubusercontent.com/milvus-io/docs/master/assets/k8s_arch.png) + +#### Prerequisites + +- A Kubernetes cluster is already established. +- [nvidia-docker 2.0](https://github.com/nvidia/nvidia-docker/wiki/Installation-(version-2.0)) is already installed. +- Shared storage is already installed. +- kubectl is installed and can access the Kubernetes cluster. + +#### Use Kubernetes to build a Milvus cluster + +1. Start Milvus cluster + + ```shell + $ make cluster + ``` + +2. Confirm that Mishards is connected to Milvus. + + ```shell + $ make probe_cluster + Pass ==> Pass: Connected + ``` + +To check cluster status: + +```shell +$ make cluster_status +``` + +To delete the cluster: + +```shell +$ make clean_cluster +``` + +To add a read instance: + +```shell +$ cd kubernetes_demo +$ ./start.sh scale-ro-server 2 +``` + +To add a proxy instance: + +```shell +$ cd kubernetes_demo +$ ./start.sh scale-proxy 2 +``` + +To check cluster logs: + +```shell +$ kubectl logs -f --tail=1000 -n milvus milvus-ro-servers-0 +``` + +## Mishards Unit test + +**Unit test** + +```shell +$ cd milvus/shards +$ make test +``` + +**Code coverage test** + +```shell +$ cd milvus/shards +$ make coverage +``` + +**Code format check** + +```shell +$ cd milvus/shards +$ make style +``` + +## Mishards configuration + +### Overall configuration + +| Name | Required | Type | Default | Description | +| ------------- | -------- | ------- | ------- | ------------------------------------------------------------ | +| `Debug` | No | boolean | `True` | Choose if to enable `Debug` work mode. | +| `TIMEZONE` | No | string | `UTC` | Timezone | +| `MAX_RETRY` | No | integer | `3` | The maximum retry times allowed to connect to Milvus. | +| `SERVER_PORT` | No | integer | `19530` | Define the server port of Mishards. | +| `WOSERVER` | **Yes** | string | ` ` | Define the address of Milvus write instance. Currently, only static settings are supported. Format for reference: `tcp://127.0.0.1:19530`. | + +### Metadata + +| Name | Required | Type | Default | Description | +| ------------------------------ | -------- | ------- | ------- | ------------------------------------------------------------ | +| `SQLALCHEMY_DATABASE_URI` | **Yes** | string | ` ` | Define the database address for metadata storage. Format standard: RFC-738-style. For example: `mysql+pymysql://root:root@127.0.0.1:3306/milvus?charset=utf8mb4`. | +| `SQL_ECHO` | No | boolean | `False` | Choose if to print SQL statements. | +| `SQLALCHEMY_DATABASE_TEST_URI` | No | string | ` ` | Define the database address of metadata storage in test environment. | +| `SQL_TEST_ECHO` | No | boolean | `False` | Choose if to print SQL statements in test environment. | + +### Service discovery + +| Name | Required | Type | Default | Description | +| ------------------------------------- | -------- | ------- | ------------- | ------------------------------------------------------------ | +| `DISCOVERY_PLUGIN_PATH` | No | string | ` ` | Define the search path to locate the plug-in. The default path is used if the value is not set. | +| `DISCOVERY_CLASS_NAME` | No | string | `static` | Under the plug-in search path, search the class based on the class name, and instantiate it. Currently, the system provides 2 classes: `static` and `kubernetes`. | +| `DISCOVERY_STATIC_HOSTS` | No | list | `[]` | When `DISCOVERY_CLASS_NAME` is `static` , define a comma-separated service address list, for example`192.168.1.188,192.168.1.190`. | +| `DISCOVERY_STATIC_PORT` | No | integer | `19530` | When `DISCOVERY_CLASS_NAME` is `static`, define the server port. | +| `DISCOVERY_KUBERNETES_NAMESPACE` | No | string | ` ` | When `DISCOVERY_CLASS_NAME` is `kubernetes`, define the namespace of Milvus cluster. | +| `DISCOVERY_KUBERNETES_IN_CLUSTER` | No | boolean | `False` | When `DISCOVERY_CLASS_NAME` is `kubernetes` , choose if to run the server in Kubernetes. | +| `DISCOVERY_KUBERNETES_POLL_INTERVAL` | No | integer | `5` (Seconds) | When `DISCOVERY_CLASS_NAME` is `kubernetes` , define the listening cycle of the server. | +| `DISCOVERY_KUBERNETES_POD_PATT` | No | string | ` ` | When `DISCOVERY_CLASS_NAME` is `kubernetes` , map the regular expression of Milvus Pod. | +| `DISCOVERY_KUBERNETES_LABEL_SELECTOR` | No | string | ` ` | When `SD_PROVIDER` is `kubernetes`, map the label of Milvus Pod. For example: `tier=ro-servers`. | + +### Tracing + +| Name | Required | Type | Default | Description | +| ----------------------- | -------- | ------- | ---------- | ------------------------------------------------------------ | +| `TRACER_PLUGIN_PATH` | No | string | ` ` | Define the search path to locate the tracing plug-in. The default path is used if the value is not set. | +| `TRACER_CLASS_NAME` | No | string | ` ` | Under the plug-in search path, search the class based on the class name, and instantiate it. Currently, only `Jaeger` is supported. | +| `TRACING_SERVICE_NAME` | No | string | `mishards` | When `TRACING_CLASS_NAME` is [`Jaeger`](https://www.jaegertracing.io/docs/1.14/), the name of the tracing service. | +| `TRACING_SAMPLER_TYPE` | No | string | `const` | When `TRACING_CLASS_NAME` is [`Jaeger`](https://www.jaegertracing.io/docs/1.14/), the [sampling type](https://www.jaegertracing.io/docs/1.14/sampling/) of the tracing service. | +| `TRACING_SAMPLER_PARAM` | No | integer | `1` | When `TRACING_CLASS_NAME` is [`Jaeger`](https://www.jaegertracing.io/docs/1.14/), the [sampling frequency](https://www.jaegertracing.io/docs/1.14/sampling/) of the tracing service. | +| `TRACING_LOG_PAYLOAD` | No | boolean | `False` | When `TRACING_CLASS_NAME` is [`Jaeger`](https://www.jaegertracing.io/docs/1.14/), choose if to sample Payload. | + +### Logging + +| Name | Required | Type | Default | Description | +| ----------- | -------- | ------ | --------------- | ------------------------------------------------------------ | +| `LOG_LEVEL` | No | string | `DEBUG` | Log recording levels. Currently supports `DEBUG` ,`INFO` ,`WARNING` and `ERROR`. | +| `LOG_PATH` | No | string | `/tmp/mishards` | Log recording path. | +| `LOG_NAME` | No | string | `logfile` | Log recording name. | + +### Routing + +| Name | Required | Type | Default | Description | +| ------------------------ | -------- | ------ | ------------------------- | ------------------------------------------------------------ | +| `ROUTER_PLUGIN_PATH` | No | string | ` ` | Define the search path to locate the routing plug-in. The default path is used if the value is not set. | +| `ROUTER_CLASS_NAME` | No | string | `FileBasedHashRingRouter` | Under the plug-in search path, search the class based on the class name, and instantiate it. Currently, only `FileBasedHashRingRouter` is supported. | +| `ROUTER_CLASS_TEST_NAME` | No | string | `FileBasedHashRingRouter` | Under the plug-in search path, search the class based on the class name, and instantiate it. Currently, `FileBasedHashRingRouter` is supported for test environment only. | + From eedb67270ddf7fe094689bb41751d2f5b429ffeb Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Tue, 12 Nov 2019 19:04:35 +0800 Subject: [PATCH 075/103] [skip ci] Update README_CN --- shards/README_CN.md | 261 ++++++++++++++++++++++++++++++++++++++++++ shards/Tutorial_CN.md | 147 ------------------------ 2 files changed, 261 insertions(+), 147 deletions(-) create mode 100644 shards/README_CN.md delete mode 100644 shards/Tutorial_CN.md diff --git a/shards/README_CN.md b/shards/README_CN.md new file mode 100644 index 0000000000..24e019d001 --- /dev/null +++ b/shards/README_CN.md @@ -0,0 +1,261 @@ +# Mishards - Milvus 集群分片中间件 + +Milvus 旨在帮助用户实现海量非结构化数据的近似检索和分析。单个 Milvus 实例可处理十亿级数据规模，而对于百亿或者千亿级数据，则需要一个 Milvus 集群实例。该实例对于上层应用可以像单机实例一样使用，同时满足海量数据低延迟、高并发业务需求。 + +本文主要展示如何使用 Mishards 分片中间件来搭建 Milvus 集群。 + +## Mishards 是什么 + +Mishards 是一个用 Python 开发的 Milvus 集群分片中间件，其内部处理请求转发、读写分离、水平扩展、动态扩容，为用户提供内存和算力可以无限扩容的 Milvus 实例。 + +Mishards 的设计尚未完成，属于试用功能，希望大家多多测试、提供反馈。 + +## Mishards 如何工作 + +Mishards 负责将上游请求拆分，并路由到内部各细分子服务，最后将子服务结果汇总，返回给上游。 + +![mishards](https://raw.githubusercontent.com/milvus-io/docs/master/assets/mishards.png) + +## Mishards 相关示例 + +以下分别向您展示如何使用源代码在单机上启动 Mishards 和 Milvus 服务，以及如何使用 Kubernetes 启动 Milvus 集群和 Mishards。 + +Milvus 启动的前提条件请参考 [Milvus 安装](https://www.milvus.io/docs/zh-CN/userguide/install_milvus/)。 + +### 源代码启动示例 + +#### 前提条件 + +Python 版本为3.6及以上。 + +#### 源代码启动 Milvus 和 Mishards 实例 + +请按照以下步骤在单机上启动单个 Milvus 实例和 Mishards 服务： + +1. 将 milvus repository 复制到本地。 + + ```shell + git clone + ``` + +2. 安装 Mishards 的依赖库。 + + ```shell + $ cd milvus/shards + $ pip install -r requirements.txt + ``` + +3. 启动 Milvus 服务。 + + ```shell + $ sudo nvidia-docker run --rm -d -p 19530:19530 -v /tmp/milvus/db:/opt/milvus/db milvusdb/milvus:0.5.0-d102119-ede20b + ``` + +4. 更改目录权限。 + + ```shell + $ sudo chown -R $USER:$USER /tmp/milvus + ``` + +5. 配置 Mishards 环境变量 + + ```shell + $ cp mishards/.env.example mishards/.env + ``` + +6. 启动 Mishards 服务 + + ```shell + $ python mishards/main.py + ``` + +### Docker 示例 + +`all_in_one` 使用 Docker 容器启动2个 Milvus 实例，1个 Mishards 中间件实例，和1个 Jaeger 链路追踪实例。 + + 1. 安装 [Docker Compose](https://docs.docker.com/compose/install/)。 + + 2. 制作实例镜像。 + + ```shell + $ make build + ``` + + 3. 启动所有服务。 + + ```shell + $ make deploy + ``` + + 4. 检查确认服务状态。 + + ```shell + $ make probe_deploy + Pass ==> Pass: Connected + Fail ==> Error: Fail connecting to server on 127.0.0.1:19530. Timeout + ``` + +若要查看服务踪迹，使用浏览器打开 [Jaeger 页面](http://127.0.0.1:16686/)。 + +![jaegerui](https://github.com/milvus-io/docs/blob/master/assets/jaegerui.png) + +![jaegertraces](https://github.com/milvus-io/docs/blob/master/assets/jaegertraces.png) + +若要清理所有服务，请使用如下命令： + +```shell +$ make clean_deploy +``` + +### Kubernetes 示例 + +使用 Kubernetes 部署 Milvus 分布式集群要求开发人员对 Kubernetes 的[基本概念](https://kubernetes.io/docs/concepts/)和操作有基本了解。 + +本示例主要展示如何使用 Kubernetes 搭建 Milvus 集群，包含2个 Milvus 实例（1个可读实例，1个可写实例）、1个 MySQL 实例和1个 Mishards 实例。 + +本示例不包括如何搭建 Kubernetes 集群，如何安装[共享存储](https://kubernetes.io/docs/concepts/storage/volumes/)和如何安装 [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) 命令行工具等。 + +以下是 Kubernetes 示例架构图： + +![k8s_arch](https://github.com/milvus-io/docs/blob/master/assets/k8s_arch.png) + +#### 前提条件 + +使用 Kubernetes 启动多个 Milvus 实例之前，请确保您已满足以下条件： + +- 已创建 Kubernetes 集群 +- 已安装 [nvidia-docker 2.0](https://github.com/nvidia/nvidia-docker/wiki/Installation-(version-2.0)) +- 已安装共享存储 +- 已安装 kubectl，且能访问集群 + +#### Kubernetes 启动集群 + +1. 启动 Milvus 集群。 + + ```shell + $ make cluster + ``` + +2. 确认 Mishards 是否可用。 + + ```shell + $ make probe_cluster + Pass ==> Pass: Connected + ``` + +查看集群状态： + +```shell +$ make cluster_status +``` + +删除 Milvus 集群： + +```shell +$ make clean_cluster +``` + +扩容 Milvus 可读实例到2个： + +```shell +$ cd kubernetes_demo +$ ./start.sh scale-ro-server 2 +``` + +扩容 Mishards（代理）实例到2个： + +```shell +$ cd kubernetes_demo +$ ./start.sh scale-proxy 2 +``` + +查看计算节点 `milvus-ro-servers-0` 日志： + +```shell +$ kubectl logs -f --tail=1000 -n milvus milvus-ro-servers-0 +``` + +## 单元测试 + +**单元测试** + +```shell +$ cd milvus/shards +$ make test +``` + +**代码覆盖率测试** + +```shell +$ cd milvus/shards +$ make coverage +``` + +**代码格式检查** + +```shell +$ cd milvus/shards +$ make style +``` + +## Mishards 配置 + +### 全局配置 + +| 参数 | 是否必填 | 类型 | 默认值 | 说明 | +| ------------- | -------- | ------- | ------- | ------------------------------------------------------------ | +| `Debug` | No | boolean | `True` | 选择是否启用 `Debug` 工作模式。 | +| `TIMEZONE` | No | string | `UTC` | 时区 | +| `MAX_RETRY` | No | integer | `3` | Mishards 连接 Milvus 的最大重试次数。 | +| `SERVER_PORT` | No | integer | `19530` | 定义 Mishards 的服务端口。 | +| `WOSERVER` | **Yes** | string | ` ` | 定义 Milvus 可写实例的地址，目前只支持静态设置。参考格式： `tcp://127.0.0.1:19530`。 | + +### 元数据 + +| 参数 | 是否必填 | 类型 | 默认值 | 说明 | +| ------------------------------ | -------- | ------- | ------- | ------------------------------------------------------------ | +| `SQLALCHEMY_DATABASE_URI` | **Yes** | string | ` ` | 定义元数据存储的数据库地址，格式标准为 RFC-738-style。例如：`mysql+pymysql://root:root@127.0.0.1:3306/milvus?charset=utf8mb4`。 | +| `SQL_ECHO` | No | boolean | `False` | 选择是否打印 SQL 详细语句。 | +| `SQLALCHEMY_DATABASE_TEST_URI` | No | string | ` ` | 定义测试环境下元数据存储的数据库地址。 | +| `SQL_TEST_ECHO` | No | boolean | `False` | 选择测试环境下是否打印 SQL 详细语句。 | + +### 服务发现 + +| 参数 | 是否必填 | 类型 | 默认值 | 说明 | +| ------------------------------------- | -------- | ------- | -------- | ------------------------------------------------------------ | +| `DISCOVERY_PLUGIN_PATH` | No | string | ` ` | 用户自定义服务发现插件的搜索路径，默认使用系统搜索路径。 | +| `DISCOVERY_CLASS_NAME` | No | string | `static` | 在插件搜索路径下，根据类名搜索类，并将其实例化。目前系统提供 `static` 和 `kubernetes` 两种类，默认使用 `static`。 | +| `DISCOVERY_STATIC_HOSTS` | No | list | `[]` | `DISCOVERY_CLASS_NAME`为 `static` 时，定义服务地址列表，地址之间以逗号隔开，例如 `192.168.1.188,192.168.1.190`。 | +| `DISCOVERY_STATIC_PORT` | No | integer | `19530` | `DISCOVERY_CLASS_NAME` 为 `static` 时，定义服务地址监听端口。 | +| `DISCOVERY_KUBERNETES_NAMESPACE` | No | string | ` ` | `DISCOVERY_CLASS_NAME` 为 `kubernetes`时，定义 Milvus 集群的namespace。 | +| `DISCOVERY_KUBERNETES_IN_CLUSTER` | No | boolean | `False` | `DISCOVERY_CLASS_NAME` 为 `kubernetes` 时，选择服务发现是否在集群中运行。 | +| `DISCOVERY_KUBERNETES_POLL_INTERVAL` | No | integer | `5` | `DISCOVERY_CLASS_NAME` 为 `kubernetes` 时，定义服务发现监听周期，单位：second。 | +| `DISCOVERY_KUBERNETES_POD_PATT` | No | string | ` ` | `DISCOVERY_CLASS_NAME` 为 `kubernetes` 时，匹配 Milvus Pod 名字的正则表达式。 | +| `DISCOVERY_KUBERNETES_LABEL_SELECTOR` | No | string | ` ` | `SD_PROVIDER`为`kubernetes`时，匹配 Milvus Pod 的标签。例如：`tier=ro-servers`。 | + +### 链路追踪 + +| 参数 | 是否必填 | 类型 | 默认值 | 说明 | +| ----------------------- | -------- | ------- | ---------- | ------------------------------------------------------------ | +| `TRACER_PLUGIN_PATH` | No | string | ` ` | 用户自定义链路追踪插件的搜索路径，默认使用系统搜索路径。 | +| `TRACER_CLASS_NAME` | No | string | ` ` | 在插件搜索路径下，根据类名搜索类，并将其实例化。目前只支持 `Jaeger`, 默认不使用。 | +| `TRACING_SERVICE_NAME` | No | string | `mishards` | `TRACING_CLASS_NAME` 为 [`Jaeger`](https://www.jaegertracing.io/docs/1.14/)时，链路追踪的 service。 | +| `TRACING_SAMPLER_TYPE` | No | string | `const` | `TRACING_CLASS_NAME`为 `Jaeger` 时，链路追踪的[采样类型](https://www.jaegertracing.io/docs/1.14/sampling/)。 | +| `TRACING_SAMPLER_PARAM` | No | integer | `1` | `TRACING_CLASS_NAME` 为 `Jaeger`时，链路追踪的[采样频率](https://www.jaegertracing.io/docs/1.14/sampling/)。 | +| `TRACING_LOG_PAYLOAD` | No | boolean | `False` | `TRACING_CLASS_NAME`为 `Jaeger`时，链路追踪是否采集 Payload。 | + +### 日志 + +| 参数 | 是否必填 | 类型 | 默认值 | 说明 | +| ----------- | -------- | ------ | --------------- | ------------------------------------------------------------ | +| `LOG_LEVEL` | No | string | `DEBUG` | 日志记录级别，目前支持 `DEBUG` 、`INFO` 、`WARNING` 和`ERROR`。 | +| `LOG_PATH` | No | string | `/tmp/mishards` | 日志记录路径。 | +| `LOG_NAME` | No | string | `logfile` | 日志记录名。 | + +### 路由 + +| 参数 | 是否必填 | 类型 | 默认值 | 说明 | +| ------------------------ | -------- | ------ | ------------------------- | ------------------------------------------------------------ | +| `ROUTER_PLUGIN_PATH` | No | string | ` ` | 用户自定义路由插件的搜索路径，默认使用系统搜索路径。 | +| `ROUTER_CLASS_NAME` | No | string | `FileBasedHashRingRouter` | 在插件搜索路径下，根据类名搜索路由的类，并将其实例化。目前系统只提供了 `FileBasedHashRingRouter`。 | +| `ROUTER_CLASS_TEST_NAME` | No | string | `FileBasedHashRingRouter` | 在插件搜索路径下，根据类名搜索路由的类，并将其实例化。目前系统只提供了 `FileBasedHashRingRouter`，仅限测试环境下使用。 | diff --git a/shards/Tutorial_CN.md b/shards/Tutorial_CN.md deleted file mode 100644 index 192a0fd285..0000000000 --- a/shards/Tutorial_CN.md +++ /dev/null @@ -1,147 +0,0 @@ -# Mishards使用文档 ---- -Milvus 旨在帮助用户实现海量非结构化数据的近似检索和分析。单个 Milvus 实例可处理十亿级数据规模，而对于百亿或者千亿规模数据的需求，则需要一个 Milvus 集群实例，该实例对于上层应用可以像单机实例一样使用，同时满足海量数据低延迟，高并发业务需求。mishards就是一个集群中间件，其内部处理请求转发，读写分离，水平扩展，动态扩容，为用户提供内存和算力可以无限扩容的 Milvus 实例。 - -## 运行环境 ---- - -### 单机快速启动实例 -**`python >= 3.4`环境** - -``` -1. cd milvus/shards -2. pip install -r requirements.txt -3. nvidia-docker run --rm -d -p 19530:19530 -v /tmp/milvus/db:/opt/milvus/db milvusdb/milvus:0.5.0-d102119-ede20b -4. sudo chown -R $USER:$USER /tmp/milvus -5. cp mishards/.env.example mishards/.env -6. 在python mishards/main.py #.env配置mishards监听19532端口 -7. make probe port=19532 #健康检查 -``` - -### 容器启动实例 -`all_in_one`会在服务器上开启两个milvus实例，一个mishards实例，一个jaeger链路追踪实例 - -**启动** -``` -cd milvus/shards -1. 安装docker-compose -2. make build -3. make deploy #监听19531端口 -4. make clean_deploy #清理服务 -5. make probe_deplopy #健康检查 -``` - -**打开Jaeger UI** -``` -浏览器打开 "http://127.0.0.1:16686/" -``` - -### kubernetes中快速启动 -**准备** -``` -- kubernetes集群 -- 安装nvidia-docker -- 共享存储 -- 安装kubectl并能访问集群 -``` - -**步骤** -``` -cd milvus/shards -1. make deploy_cluster #启动集群 -2. make probe_cluster #健康检查 -3. make clean_cluster #关闭集群 -``` - -**扩容计算实例** -``` -cd milvus/shards/kubernetes_demo/ -./start.sh scale-ro-server 2 扩容计算实例到2 -``` - -**扩容代理器实例** -``` -cd milvus/shards/kubernetes_demo/ -./start.sh scale-proxy 2 扩容代理服务器实例到2 -``` - -**查看日志** -``` -kubectl logs -f --tail=1000 -n milvus milvus-ro-servers-0 查看计算节点milvus-ro-servers-0日志 -``` - -## 测试 - -**启动单元测试** -``` -1. cd milvus/shards -2. make test -``` - -**单元测试覆盖率** -``` -1. cd milvus/shards -2. make coverage -``` - -**代码风格检查** -``` -1. cd milvus/shards -2. make style -``` - -## mishards配置详解 - -### 全局 -| Name | Required | Type | Default Value | Explanation | -| --------------------------- | -------- | -------- | ------------- | ------------- | -| Debug | No | bool | True | 是否Debug工作模式 | -| TIMEZONE | No | string | "UTC" | 时区 | -| MAX_RETRY | No | int | 3 | 最大连接重试次数 | -| SERVER_PORT | No | int | 19530 | 配置服务端口 | -| WOSERVER | **Yes** | str | - | 配置后台可写Milvus实例地址。目前只支持静态设置,例"tcp://127.0.0.1:19530" | - -### 元数据 -| Name | Required | Type | Default Value | Explanation | -| --------------------------- | -------- | -------- | ------------- | ------------- | -| SQLALCHEMY_DATABASE_URI | **Yes** | string | - | 配置元数据存储数据库地址 | -| SQL_ECHO | No | bool | False | 是否打印Sql详细语句 | -| SQLALCHEMY_DATABASE_TEST_URI | No | string | - | 配置测试环境下元数据存储数据库地址 | -| SQL_TEST_ECHO | No | bool | False | 配置测试环境下是否打印Sql详细语句 | - -### 服务发现 -| Name | Required | Type | Default Value | Explanation | -| --------------------------- | -------- | -------- | ------------- | ------------- | -| DISCOVERY_PLUGIN_PATH | No | string | - | 用户自定义服务发现插件搜索路径，默认使用系统搜索路径| -| DISCOVERY_CLASS_NAME | No | string | static | 在服务发现插件搜索路径下搜索类并实例化。目前系统提供 **static** 和 **kubernetes** 两种类，默认使用 **static** | -| DISCOVERY_STATIC_HOSTS | No | list | [] | **DISCOVERY_CLASS_NAME** 为 **static** 时，配置服务地址列表，例"192.168.1.188,192.168.1.190"| -| DISCOVERY_STATIC_PORT | No | int | 19530 | **DISCOVERY_CLASS_NAME** 为 **static** 时，配置 Hosts 监听端口 | -| DISCOVERY_KUBERNETES_NAMESPACE | No | string | - | **DISCOVERY_CLASS_NAME** 为 **kubernetes** 时，配置集群 namespace | -| DISCOVERY_KUBERNETES_IN_CLUSTER | No | bool | False | **DISCOVERY_CLASS_NAME** 为 **kubernetes** 时，标明服务发现是否在集群中运行 | -| DISCOVERY_KUBERNETES_POLL_INTERVAL | No | int | 5 | **DISCOVERY_CLASS_NAME** 为 **kubernetes** 时，标明服务发现监听服务列表频率,单位 Second | -| DISCOVERY_KUBERNETES_POD_PATT | No | string | - | **DISCOVERY_CLASS_NAME** 为 **kubernetes** 时，匹配可读 Milvus 实例的正则表达式 | -| DISCOVERY_KUBERNETES_LABEL_SELECTOR | No | string | - | **SD_PROVIDER** 为**Kubernetes**时，匹配可读Milvus实例的标签选择 | - -### 链路追踪 -| Name | Required | Type | Default Value | Explanation | -| --------------------------- | -------- | -------- | ------------- | ------------- | -| TRACER_PLUGIN_PATH | No | string | - | 用户自定义链路追踪插件搜索路径，默认使用系统搜索路径| -| TRACER_CLASS_NAME | No | string | "" | 链路追踪方案选择，目前只实现 **Jaeger**, 默认不使用| -| TRACING_SERVICE_NAME | No | string | "mishards" | **TRACING_TYPE** 为 **Jaeger** 时，链路追踪服务名 | -| TRACING_SAMPLER_TYPE | No | string | "const" | **TRACING_TYPE** 为 **Jaeger** 时，链路追踪采样类型 | -| TRACING_SAMPLER_PARAM | No | int | 1 | **TRACING_TYPE** 为 **Jaeger** 时，链路追踪采样频率 | -| TRACING_LOG_PAYLOAD | No | bool | False | **TRACING_TYPE** 为 **Jaeger** 时，链路追踪是否采集 Payload | - -### 日志 -| Name | Required | Type | Default Value | Explanation | -| --------------------------- | -------- | -------- | ------------- | ------------- | -| LOG_LEVEL | No | string | "DEBUG" if Debug is ON else "INFO" | 日志记录级别 | -| LOG_PATH | No | string | "/tmp/mishards" | 日志记录路径 | -| LOG_NAME | No | string | "logfile" | 日志记录名 | - -### 路由 -| Name | Required | Type | Default Value | Explanation | -| --------------------------- | -------- | -------- | ------------- | ------------- | -| ROUTER_PLUGIN_PATH | No | string | - | 用户自定义路由插件搜索路径，默认使用系统搜索路径| -| ROUTER_CLASS_NAME | No | string | FileBasedHashRingRouter | 处理请求路由类名, 可注册自定义类。目前系统只提供了类 **FileBasedHashRingRouter** | -| ROUTER_CLASS_TEST_NAME | No | string | FileBasedHashRingRouter | 测试环境下处理请求路由类名, 可注册自定义类 | From 285788eba8c47087a0965b4e21bc6b24bc577db1 Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Tue, 12 Nov 2019 19:06:51 +0800 Subject: [PATCH 076/103] [skip ci] Add link to CN README --- shards/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/shards/README.md b/shards/README.md index dba6f785aa..f59eca0460 100644 --- a/shards/README.md +++ b/shards/README.md @@ -1,5 +1,7 @@ # Mishards - An Experimental Sharding Middleware +[中文版](README_CN.md) + Milvus aims to achieve efficient similarity search and analytics for massive-scale vectors. A standalone Milvus instance can easily handle vector search among billion-scale vectors. However, for 10 billion, 100 billion or even larger datasets, a Milvus cluster is needed. Ideally, this cluster can be accessed and used just as the standalone instance, meanwhile it satisfies the business requirements such as low latency and high concurrency. From dfcce896e2cd7ab6910730e2e384762095ad4cad Mon Sep 17 00:00:00 2001 From: groot Date: Tue, 12 Nov 2019 19:11:07 +0800 Subject: [PATCH 077/103] #260 C++ SDK README --- CHANGELOG.md | 1 + core/src/sdk/README.md | 93 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+) create mode 100644 core/src/sdk/README.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 363c2cc385..02fb05d9cc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ Please mark all change in change log and use the ticket from JIRA. ## Improvement - \#284 - Change C++ SDK to shared library +- \#260 - C++ SDK README ## Task diff --git a/core/src/sdk/README.md b/core/src/sdk/README.md new file mode 100644 index 0000000000..5dc5c733e0 --- /dev/null +++ b/core/src/sdk/README.md @@ -0,0 +1,93 @@ +### Build C++ sdk + +The C++ sdk source code is under milvus/core/src/sdk. Build entire milvus project will also build the sdk project. +If you don't want to build entire milvus project, you can do the following steps: +```shell + # generate make files + $ cd [Milvus root path]/core + $ ./build.sh -l + + # build C++ SDK project + $ cd [Milvus root path]/core/cmake_build + $ make -C src/sdk +``` + +### Try C++ example + +Firstly you need to launch a milvus server. +If you build whole milvus project, just run: +```shell + # start milvus server + $ cd [Milvus root path]/core + $ ./start_server.sh +``` +You also can pull milvus release docker image to launch milvus server. +```shell + # pull milvus docker image and start milvus server + $ docker pull milvusdb/milvus:latest + $ docker run --runtime=nvidia -p 19530:19530 -d milvusdb/milvus:latest +``` + +To run C++ example, use below command: + +```shell + # run milvus C++ example + $ cd [Milvus root path]/core/cmake_build/src/sdk/examples/simple + $ ./sdk_simple +``` + +### Make your own C++ client project + +Firstly create a project folder. And copy C++ sdk header and library into the folder. +```shell + # create project folder + $ mkdir MyMilvusClient + $ cd MyMilvusClient + + # copy necessary files + $ cp [Milvus root path]/core/cmake_build/src/sdk/libmilvus_sdk.so . + $ cp [Milvus root path]/core/src/sdk/include/MilvusApi.h . + $ cp [Milvus root path]/core/src/sdk/include/Status.h . +``` + +Create a main.cpp under the project folder, and include C++ sdk headers: +```shell +#include "./MilvusApi.h" +#include "./Status.h" + +int main() { + // connect to milvus server + std::shared_ptr conn = milvus::Connection::Create(); + milvus::ConnectParam param = {"127.0.0.1", "19530"}; + conn->Connect(param); + + // put your client code here + + milvus::Connection::Destroy(conn); + return 0; +} +``` + +Create a CMakeList.txt under the project folder, and paste the follow code into the file: +```shell + cmake_minimum_required(VERSION 3.14) + project(test) + set(CMAKE_CXX_STANDARD 14) + + add_executable(milvus_client main.cpp) + target_link_libraries(milvus_client + ${PROJECT_SOURCE_DIR}/libmilvus_sdk.so) +``` + +Build the client project: +```shell + $ mkdir cmake_build + $ cd cmake_build + $ cmake .. + $ make +``` + +Run your client program: +```shell + $ ./milvus_client +``` \ No newline at end of file From 06ae349fe249aa7f7b7868c6ab33879c1e6d4366 Mon Sep 17 00:00:00 2001 From: groot Date: Tue, 12 Nov 2019 19:14:20 +0800 Subject: [PATCH 078/103] #260 C++ SDK README --- CHANGELOG.md | 1 + core/src/sdk/README.md | 93 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+) create mode 100644 core/src/sdk/README.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 92167c3922..3f78e72192 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,7 @@ Please mark all change in change log and use the ticket from JIRA. - \#207 - Add more unittest for config set/get - \#208 - optimize unittest to support run single test more easily - \#284 - Change C++ SDK to shared library +- \#260 - C++ SDK README ## Task diff --git a/core/src/sdk/README.md b/core/src/sdk/README.md new file mode 100644 index 0000000000..5dc5c733e0 --- /dev/null +++ b/core/src/sdk/README.md @@ -0,0 +1,93 @@ +### Build C++ sdk + +The C++ sdk source code is under milvus/core/src/sdk. Build entire milvus project will also build the sdk project. +If you don't want to build entire milvus project, you can do the following steps: +```shell + # generate make files + $ cd [Milvus root path]/core + $ ./build.sh -l + + # build C++ SDK project + $ cd [Milvus root path]/core/cmake_build + $ make -C src/sdk +``` + +### Try C++ example + +Firstly you need to launch a milvus server. +If you build whole milvus project, just run: +```shell + # start milvus server + $ cd [Milvus root path]/core + $ ./start_server.sh +``` +You also can pull milvus release docker image to launch milvus server. +```shell + # pull milvus docker image and start milvus server + $ docker pull milvusdb/milvus:latest + $ docker run --runtime=nvidia -p 19530:19530 -d milvusdb/milvus:latest +``` + +To run C++ example, use below command: + +```shell + # run milvus C++ example + $ cd [Milvus root path]/core/cmake_build/src/sdk/examples/simple + $ ./sdk_simple +``` + +### Make your own C++ client project + +Firstly create a project folder. And copy C++ sdk header and library into the folder. +```shell + # create project folder + $ mkdir MyMilvusClient + $ cd MyMilvusClient + + # copy necessary files + $ cp [Milvus root path]/core/cmake_build/src/sdk/libmilvus_sdk.so . + $ cp [Milvus root path]/core/src/sdk/include/MilvusApi.h . + $ cp [Milvus root path]/core/src/sdk/include/Status.h . +``` + +Create a main.cpp under the project folder, and include C++ sdk headers: +```shell +#include "./MilvusApi.h" +#include "./Status.h" + +int main() { + // connect to milvus server + std::shared_ptr conn = milvus::Connection::Create(); + milvus::ConnectParam param = {"127.0.0.1", "19530"}; + conn->Connect(param); + + // put your client code here + + milvus::Connection::Destroy(conn); + return 0; +} +``` + +Create a CMakeList.txt under the project folder, and paste the follow code into the file: +```shell + cmake_minimum_required(VERSION 3.14) + project(test) + set(CMAKE_CXX_STANDARD 14) + + add_executable(milvus_client main.cpp) + target_link_libraries(milvus_client + ${PROJECT_SOURCE_DIR}/libmilvus_sdk.so) +``` + +Build the client project: +```shell + $ mkdir cmake_build + $ cd cmake_build + $ cmake .. + $ make +``` + +Run your client program: +```shell + $ ./milvus_client +``` \ No newline at end of file From 0442ec7900d1dd62a203b91819be20b92fa6aff1 Mon Sep 17 00:00:00 2001 From: groot Date: Tue, 12 Nov 2019 19:22:27 +0800 Subject: [PATCH 079/103] #260 C++ SDK README --- core/src/config.h | 20 -------------------- 1 file changed, 20 deletions(-) delete mode 100644 core/src/config.h diff --git a/core/src/config.h b/core/src/config.h deleted file mode 100644 index d935c36faf..0000000000 --- a/core/src/config.h +++ /dev/null @@ -1,20 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -#define MILVUS_VERSION "0.6.0" -#define BUILD_TYPE "Debug" -#define BUILD_TIME "2019-11-12 14:06.18" From 5a74558f0d674a9e815faa33113ed2348a7083b5 Mon Sep 17 00:00:00 2001 From: groot Date: Tue, 12 Nov 2019 19:45:21 +0800 Subject: [PATCH 080/103] #260 C++ SDK README --- core/src/sdk/README.md | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/core/src/sdk/README.md b/core/src/sdk/README.md index 5dc5c733e0..48a047df1f 100644 --- a/core/src/sdk/README.md +++ b/core/src/sdk/README.md @@ -7,7 +7,7 @@ If you don't want to build entire milvus project, you can do the following steps $ cd [Milvus root path]/core $ ./build.sh -l - # build C++ SDK project + # build C++ sdk project $ cd [Milvus root path]/core/cmake_build $ make -C src/sdk ``` @@ -15,20 +15,20 @@ If you don't want to build entire milvus project, you can do the following steps ### Try C++ example Firstly you need to launch a milvus server. -If you build whole milvus project, just run: +If you already build entire milvus project, just run: ```shell # start milvus server $ cd [Milvus root path]/core $ ./start_server.sh ``` -You also can pull milvus release docker image to launch milvus server. +You also can pull milvus release docker image to launch milvus server: ```shell # pull milvus docker image and start milvus server $ docker pull milvusdb/milvus:latest $ docker run --runtime=nvidia -p 19530:19530 -d milvusdb/milvus:latest ``` -To run C++ example, use below command: +Run C++ example: ```shell # run milvus C++ example @@ -38,7 +38,7 @@ To run C++ example, use below command: ### Make your own C++ client project -Firstly create a project folder. And copy C++ sdk header and library into the folder. +Firstly create a project folder. And copy C++ sdk header and library files into the folder. ```shell # create project folder $ mkdir MyMilvusClient @@ -50,7 +50,7 @@ Firstly create a project folder. And copy C++ sdk header and library into the fo $ cp [Milvus root path]/core/src/sdk/include/Status.h . ``` -Create a main.cpp under the project folder, and include C++ sdk headers: +Create main.cpp under the project folder, and paste the following code into the file: ```shell #include "./MilvusApi.h" #include "./Status.h" @@ -68,7 +68,7 @@ int main() { } ``` -Create a CMakeList.txt under the project folder, and paste the follow code into the file: +Create CMakeList.txt under the project folder, and paste the following code into the file: ```shell cmake_minimum_required(VERSION 3.14) project(test) @@ -79,7 +79,17 @@ Create a CMakeList.txt under the project folder, and paste the follow code into ${PROJECT_SOURCE_DIR}/libmilvus_sdk.so) ``` -Build the client project: +Now there are 5 files in your project: +```shell +MyMilvusClient + |-CMakeList.txt + |-main.cpp + |-libmilvus_sdk.so + |-MilvusApi.h + |-Status.h + ``` + +Build the project: ```shell $ mkdir cmake_build $ cd cmake_build From 919846ea6d329a995c2d4febb678c9f3261a72e1 Mon Sep 17 00:00:00 2001 From: groot Date: Tue, 12 Nov 2019 19:47:37 +0800 Subject: [PATCH 081/103] #260 C++ SDK README --- core/src/sdk/README.md | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/core/src/sdk/README.md b/core/src/sdk/README.md index 5dc5c733e0..48a047df1f 100644 --- a/core/src/sdk/README.md +++ b/core/src/sdk/README.md @@ -7,7 +7,7 @@ If you don't want to build entire milvus project, you can do the following steps $ cd [Milvus root path]/core $ ./build.sh -l - # build C++ SDK project + # build C++ sdk project $ cd [Milvus root path]/core/cmake_build $ make -C src/sdk ``` @@ -15,20 +15,20 @@ If you don't want to build entire milvus project, you can do the following steps ### Try C++ example Firstly you need to launch a milvus server. -If you build whole milvus project, just run: +If you already build entire milvus project, just run: ```shell # start milvus server $ cd [Milvus root path]/core $ ./start_server.sh ``` -You also can pull milvus release docker image to launch milvus server. +You also can pull milvus release docker image to launch milvus server: ```shell # pull milvus docker image and start milvus server $ docker pull milvusdb/milvus:latest $ docker run --runtime=nvidia -p 19530:19530 -d milvusdb/milvus:latest ``` -To run C++ example, use below command: +Run C++ example: ```shell # run milvus C++ example @@ -38,7 +38,7 @@ To run C++ example, use below command: ### Make your own C++ client project -Firstly create a project folder. And copy C++ sdk header and library into the folder. +Firstly create a project folder. And copy C++ sdk header and library files into the folder. ```shell # create project folder $ mkdir MyMilvusClient @@ -50,7 +50,7 @@ Firstly create a project folder. And copy C++ sdk header and library into the fo $ cp [Milvus root path]/core/src/sdk/include/Status.h . ``` -Create a main.cpp under the project folder, and include C++ sdk headers: +Create main.cpp under the project folder, and paste the following code into the file: ```shell #include "./MilvusApi.h" #include "./Status.h" @@ -68,7 +68,7 @@ int main() { } ``` -Create a CMakeList.txt under the project folder, and paste the follow code into the file: +Create CMakeList.txt under the project folder, and paste the following code into the file: ```shell cmake_minimum_required(VERSION 3.14) project(test) @@ -79,7 +79,17 @@ Create a CMakeList.txt under the project folder, and paste the follow code into ${PROJECT_SOURCE_DIR}/libmilvus_sdk.so) ``` -Build the client project: +Now there are 5 files in your project: +```shell +MyMilvusClient + |-CMakeList.txt + |-main.cpp + |-libmilvus_sdk.so + |-MilvusApi.h + |-Status.h + ``` + +Build the project: ```shell $ mkdir cmake_build $ cd cmake_build From 48650e04e2e9772a0984fca3092cb89de956155b Mon Sep 17 00:00:00 2001 From: "yudong.cai" Date: Wed, 13 Nov 2019 09:24:27 +0800 Subject: [PATCH 082/103] #89 fix clang-format --- .../faiss_benchmark/faiss_benchmark_test.cpp | 22 ++++++++++++------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/core/src/index/unittest/faiss_benchmark/faiss_benchmark_test.cpp b/core/src/index/unittest/faiss_benchmark/faiss_benchmark_test.cpp index 5e9d45f75b..ce5aa4bafa 100644 --- a/core/src/index/unittest/faiss_benchmark/faiss_benchmark_test.cpp +++ b/core/src/index/unittest/faiss_benchmark/faiss_benchmark_test.cpp @@ -435,7 +435,7 @@ test_with_nprobes(const std::string& ann_test_name, const std::string& index_key break; } case MODE_GPU: { - faiss::gpu::GpuIndexIVF *gpu_index_ivf = dynamic_cast(index); + faiss::gpu::GpuIndexIVF* gpu_index_ivf = dynamic_cast(index); gpu_index_ivf->setNumProbes(nprobe); } } @@ -552,7 +552,7 @@ TEST(FAISSTEST, BENCHMARK) { /////////////////////////////////////////////////////////////////////////////////////////////////////////////////// const int32_t SIFT_INSERT_LOOPS = 2; // insert twice to get ~1G data set - test_ann_hdf5("sift-128-euclidean", "IDMap", "Flat", MODE_CPU,SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("sift-128-euclidean", "IDMap", "Flat", MODE_CPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); test_ann_hdf5("sift-128-euclidean", "IDMap", "Flat", MODE_GPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); test_ann_hdf5("sift-128-euclidean", "IVF16384", "Flat", MODE_CPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); @@ -562,9 +562,12 @@ TEST(FAISSTEST, BENCHMARK) { test_ann_hdf5("sift-128-euclidean", "IVF16384", "SQ8", MODE_GPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); #ifdef CUSTOMIZATION - test_ann_hdf5("sift-128-euclidean", "IVF16384", "SQ8Hybrid", MODE_CPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); - test_ann_hdf5("sift-128-euclidean", "IVF16384", "SQ8Hybrid", MODE_MIX, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); - test_ann_hdf5("sift-128-euclidean", "IVF16384", "SQ8Hybrid", MODE_GPU, SIFT_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("sift-128-euclidean", "IVF16384", "SQ8Hybrid", MODE_CPU, SIFT_INSERT_LOOPS, param_nprobes, + SEARCH_LOOPS); + test_ann_hdf5("sift-128-euclidean", "IVF16384", "SQ8Hybrid", MODE_MIX, SIFT_INSERT_LOOPS, param_nprobes, + SEARCH_LOOPS); + test_ann_hdf5("sift-128-euclidean", "IVF16384", "SQ8Hybrid", MODE_GPU, SIFT_INSERT_LOOPS, param_nprobes, + SEARCH_LOOPS); #endif /////////////////////////////////////////////////////////////////////////////////////////////////////////////////// @@ -577,8 +580,11 @@ TEST(FAISSTEST, BENCHMARK) { test_ann_hdf5("glove-200-angular", "IVF16384", "SQ8", MODE_GPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); #ifdef CUSTOMIZATION - test_ann_hdf5("glove-200-angular", "IVF16384", "SQ8Hybrid", MODE_CPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); - test_ann_hdf5("glove-200-angular", "IVF16384", "SQ8Hybrid", MODE_MIX, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); - test_ann_hdf5("glove-200-angular", "IVF16384", "SQ8Hybrid", MODE_GPU, GLOVE_INSERT_LOOPS, param_nprobes, SEARCH_LOOPS); + test_ann_hdf5("glove-200-angular", "IVF16384", "SQ8Hybrid", MODE_CPU, GLOVE_INSERT_LOOPS, param_nprobes, + SEARCH_LOOPS); + test_ann_hdf5("glove-200-angular", "IVF16384", "SQ8Hybrid", MODE_MIX, GLOVE_INSERT_LOOPS, param_nprobes, + SEARCH_LOOPS); + test_ann_hdf5("glove-200-angular", "IVF16384", "SQ8Hybrid", MODE_GPU, GLOVE_INSERT_LOOPS, param_nprobes, + SEARCH_LOOPS); #endif } From c467ed9d07de4943930ec582af4f1e094e33b6b5 Mon Sep 17 00:00:00 2001 From: ZhifengZhang-CN Date: Wed, 13 Nov 2019 11:15:03 +0800 Subject: [PATCH 083/103] add server_gpu_config.template --- core/CMakeLists.txt | 8 ++++- core/conf/server_gpu_config.template | 45 ++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+), 1 deletion(-) create mode 100644 core/conf/server_gpu_config.template diff --git a/core/CMakeLists.txt b/core/CMakeLists.txt index 65094b19a3..bacab79612 100644 --- a/core/CMakeLists.txt +++ b/core/CMakeLists.txt @@ -174,7 +174,13 @@ add_custom_target(Clean-All COMMAND ${CMAKE_BUILD_TOOL} clean) if ("${MILVUS_DB_PATH}" STREQUAL "") set(MILVUS_DB_PATH "/tmp/milvus") endif () -configure_file(${CMAKE_CURRENT_SOURCE_DIR}/conf/server_config.template ${CMAKE_CURRENT_SOURCE_DIR}/conf/server_config.yaml) + +if (MILVUS_GPU_VERSION) + configure_file(${CMAKE_CURRENT_SOURCE_DIR}/conf/server_gpu_config.template ${CMAKE_CURRENT_SOURCE_DIR}/conf/server_config.yaml) +else() + configure_file(${CMAKE_CURRENT_SOURCE_DIR}/conf/server_config.template ${CMAKE_CURRENT_SOURCE_DIR}/conf/server_config.yaml) +endif() + configure_file(${CMAKE_CURRENT_SOURCE_DIR}/conf/log_config.template ${CMAKE_CURRENT_SOURCE_DIR}/conf/log_config.conf) install(DIRECTORY scripts/ diff --git a/core/conf/server_gpu_config.template b/core/conf/server_gpu_config.template new file mode 100644 index 0000000000..db172f4f1d --- /dev/null +++ b/core/conf/server_gpu_config.template @@ -0,0 +1,45 @@ +# Default values are used when you make no changes to the following parameters. + +server_config: + address: 0.0.0.0 # milvus server ip address (IPv4) + port: 19530 # milvus server port, must in range [1025, 65534] + deploy_mode: single # deployment type: single, cluster_readonly, cluster_writable + time_zone: UTC+8 # time zone, must be in format: UTC+X + +db_config: + primary_path: @MILVUS_DB_PATH@ # path used to store data and meta + secondary_path: # path used to store data only, split by semicolon + + backend_url: sqlite://:@:/ # URI format: dialect://username:password@host:port/database + # Keep 'dialect://:@:/', and replace other texts with real values + # Replace 'dialect' with 'mysql' or 'sqlite' + + insert_buffer_size: 4 # GB, maximum insert buffer size allowed, must be a positive integer + # sum of insert_buffer_size and cpu_cache_capacity cannot exceed total memory + + preload_table: # preload data at startup, '*' means load all tables, empty value means no preload + # you can specify preload tables like this: table1,table2,table3 + +metric_config: + enable_monitor: false # enable monitoring or not, must be a boolean + collector: prometheus # prometheus + prometheus_config: + port: 8080 # port prometheus uses to fetch metrics, must in range [1025, 65534] + +cache_config: + cpu_cache_capacity: 16 # GB, CPU memory used for cache, must be a positive integer + cpu_cache_threshold: 0.85 # percentage of data that will be kept when cache cleanup is triggered, must be in range (0.0, 1.0] + gpu_cache_capacity: 4 # GB, GPU memory used for cache, must be a positive integer + gpu_cache_threshold: 0.85 # percentage of data that will be kept when cache cleanup is triggered, must be in range (0.0, 1.0] + cache_insert_data: false # whether to load inserted data into cache, must be a boolean + +engine_config: + use_blas_threshold: 1100 # if nq < use_blas_threshold, use SSE, faster with fluctuated response times + # if nq >= use_blas_threshold, use OpenBlas, slower with stable response times + gpu_search_threshold: 1000 # threshold beyond which the search computation is executed on GPUs only + +resource_config: + search_resources: # define the devices used for search computation, must be in format: cpu or gpux + - cpu + - gpu0 + index_build_device: gpu0 # GPU used for building index, must be in format: gpux From 5772c3b40058eaeddc6cb515cab995c88a8d2bd9 Mon Sep 17 00:00:00 2001 From: groot Date: Wed, 13 Nov 2019 11:42:20 +0800 Subject: [PATCH 084/103] #260 C++ SDK README --- core/src/sdk/README.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/core/src/sdk/README.md b/core/src/sdk/README.md index 48a047df1f..0f4e20d0e7 100644 --- a/core/src/sdk/README.md +++ b/core/src/sdk/README.md @@ -1,27 +1,27 @@ -### Build C++ sdk +### Build C++ SDK -The C++ sdk source code is under milvus/core/src/sdk. Build entire milvus project will also build the sdk project. -If you don't want to build entire milvus project, you can do the following steps: +The C++ SDK source code is under milvus/core/src/sdk. Build entire milvus project will also build the C++ SDK project. +If you don't want to build the entire milvus project, follow below steps: ```shell # generate make files $ cd [Milvus root path]/core $ ./build.sh -l - # build C++ sdk project + # build C++ SDK project $ cd [Milvus root path]/core/cmake_build $ make -C src/sdk ``` ### Try C++ example -Firstly you need to launch a milvus server. -If you already build entire milvus project, just run: +Firstly, you need to start a Milvus server. +If you've already built the entire milvus project, just start Milvus server with the following command: ```shell # start milvus server $ cd [Milvus root path]/core $ ./start_server.sh ``` -You also can pull milvus release docker image to launch milvus server: +You can also use Docker to start Milvus server: ```shell # pull milvus docker image and start milvus server $ docker pull milvusdb/milvus:latest @@ -38,7 +38,7 @@ Run C++ example: ### Make your own C++ client project -Firstly create a project folder. And copy C++ sdk header and library files into the folder. +Create a folder for the project, and copy C++ SDK header and library files into it. ```shell # create project folder $ mkdir MyMilvusClient @@ -50,7 +50,7 @@ Firstly create a project folder. And copy C++ sdk header and library files into $ cp [Milvus root path]/core/src/sdk/include/Status.h . ``` -Create main.cpp under the project folder, and paste the following code into the file: +Create file main.cpp in the project folder, and copy the following code into it: ```shell #include "./MilvusApi.h" #include "./Status.h" @@ -68,7 +68,7 @@ int main() { } ``` -Create CMakeList.txt under the project folder, and paste the following code into the file: +Create file CMakeList.txt in the project folder, and copy the following code into it: ```shell cmake_minimum_required(VERSION 3.14) project(test) From be733f1e5d493d41ff967d07a2f2ab0577c62044 Mon Sep 17 00:00:00 2001 From: groot Date: Wed, 13 Nov 2019 11:43:23 +0800 Subject: [PATCH 085/103] #260 C++ SDK README --- core/src/sdk/README.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/core/src/sdk/README.md b/core/src/sdk/README.md index 48a047df1f..0f4e20d0e7 100644 --- a/core/src/sdk/README.md +++ b/core/src/sdk/README.md @@ -1,27 +1,27 @@ -### Build C++ sdk +### Build C++ SDK -The C++ sdk source code is under milvus/core/src/sdk. Build entire milvus project will also build the sdk project. -If you don't want to build entire milvus project, you can do the following steps: +The C++ SDK source code is under milvus/core/src/sdk. Build entire milvus project will also build the C++ SDK project. +If you don't want to build the entire milvus project, follow below steps: ```shell # generate make files $ cd [Milvus root path]/core $ ./build.sh -l - # build C++ sdk project + # build C++ SDK project $ cd [Milvus root path]/core/cmake_build $ make -C src/sdk ``` ### Try C++ example -Firstly you need to launch a milvus server. -If you already build entire milvus project, just run: +Firstly, you need to start a Milvus server. +If you've already built the entire milvus project, just start Milvus server with the following command: ```shell # start milvus server $ cd [Milvus root path]/core $ ./start_server.sh ``` -You also can pull milvus release docker image to launch milvus server: +You can also use Docker to start Milvus server: ```shell # pull milvus docker image and start milvus server $ docker pull milvusdb/milvus:latest @@ -38,7 +38,7 @@ Run C++ example: ### Make your own C++ client project -Firstly create a project folder. And copy C++ sdk header and library files into the folder. +Create a folder for the project, and copy C++ SDK header and library files into it. ```shell # create project folder $ mkdir MyMilvusClient @@ -50,7 +50,7 @@ Firstly create a project folder. And copy C++ sdk header and library files into $ cp [Milvus root path]/core/src/sdk/include/Status.h . ``` -Create main.cpp under the project folder, and paste the following code into the file: +Create file main.cpp in the project folder, and copy the following code into it: ```shell #include "./MilvusApi.h" #include "./Status.h" @@ -68,7 +68,7 @@ int main() { } ``` -Create CMakeList.txt under the project folder, and paste the following code into the file: +Create file CMakeList.txt in the project folder, and copy the following code into it: ```shell cmake_minimum_required(VERSION 3.14) project(test) From dfbd0ec63ae1b7f6c383ce4d1f7e1aab6db31b43 Mon Sep 17 00:00:00 2001 From: ZhifengZhang-CN Date: Wed, 13 Nov 2019 13:38:59 +0800 Subject: [PATCH 086/103] update Jenkins CI --- ci/jenkins/scripts/build.sh | 142 --------------------------- ci/jenkins/step/build.groovy | 5 +- ci/jenkins/step/coverage.groovy | 2 +- ci/scripts/build.sh | 36 ++++++- ci/{jenkins => }/scripts/coverage.sh | 19 ++-- 5 files changed, 48 insertions(+), 156 deletions(-) delete mode 100755 ci/jenkins/scripts/build.sh rename ci/{jenkins => }/scripts/coverage.sh (86%) diff --git a/ci/jenkins/scripts/build.sh b/ci/jenkins/scripts/build.sh deleted file mode 100755 index 2ccdf4a618..0000000000 --- a/ci/jenkins/scripts/build.sh +++ /dev/null @@ -1,142 +0,0 @@ -#!/bin/bash - -SOURCE="${BASH_SOURCE[0]}" -while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink - DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - SOURCE="$(readlink "$SOURCE")" - [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located -done -SCRIPTS_DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - -CMAKE_BUILD_DIR="${SCRIPTS_DIR}/../../../core/cmake_build" -BUILD_TYPE="Debug" -BUILD_UNITTEST="OFF" -INSTALL_PREFIX="/opt/milvus" -BUILD_COVERAGE="OFF" -DB_PATH="/opt/milvus" -PROFILING="OFF" -USE_JFROG_CACHE="OFF" -RUN_CPPLINT="OFF" -CUSTOMIZATION="OFF" # default use ori faiss -CUDA_COMPILER=/usr/local/cuda/bin/nvcc - -CUSTOMIZED_FAISS_URL="${FAISS_URL:-NONE}" -wget -q --method HEAD ${CUSTOMIZED_FAISS_URL} -if [ $? -eq 0 ]; then - CUSTOMIZATION="ON" -else - CUSTOMIZATION="OFF" -fi - -while getopts "o:d:t:ulcgjhx" arg -do - case $arg in - o) - INSTALL_PREFIX=$OPTARG - ;; - d) - DB_PATH=$OPTARG - ;; - t) - BUILD_TYPE=$OPTARG # BUILD_TYPE - ;; - u) - echo "Build and run unittest cases" ; - BUILD_UNITTEST="ON"; - ;; - l) - RUN_CPPLINT="ON" - ;; - c) - BUILD_COVERAGE="ON" - ;; - g) - PROFILING="ON" - ;; - j) - USE_JFROG_CACHE="ON" - ;; - x) - CUSTOMIZATION="OFF" # force use ori faiss - ;; - h) # help - echo " - -parameter: --o: install prefix(default: /opt/milvus) --d: db data path(default: /opt/milvus) --t: build type(default: Debug) --u: building unit test options(default: OFF) --l: run cpplint, clang-format and clang-tidy(default: OFF) --c: code coverage(default: OFF) --g: profiling(default: OFF) --j: use jfrog cache build directory(default: OFF) --h: help - -usage: -./build.sh -p \${INSTALL_PREFIX} -t \${BUILD_TYPE} [-u] [-l] [-r] [-c] [-g] [-j] [-h] - " - exit 0 - ;; - ?) - echo "ERROR! unknown argument" - exit 1 - ;; - esac -done - -if [[ ! -d ${CMAKE_BUILD_DIR} ]]; then - mkdir ${CMAKE_BUILD_DIR} -fi - -cd ${CMAKE_BUILD_DIR} - -# remove make cache since build.sh -l use default variables -# force update the variables each time -make rebuild_cache - -CMAKE_CMD="cmake \ --DBUILD_UNIT_TEST=${BUILD_UNITTEST} \ --DCMAKE_INSTALL_PREFIX=${INSTALL_PREFIX} --DCMAKE_BUILD_TYPE=${BUILD_TYPE} \ --DCMAKE_CUDA_COMPILER=${CUDA_COMPILER} \ --DBUILD_COVERAGE=${BUILD_COVERAGE} \ --DMILVUS_DB_PATH=${DB_PATH} \ --DMILVUS_ENABLE_PROFILING=${PROFILING} \ --DUSE_JFROG_CACHE=${USE_JFROG_CACHE} \ --DCUSTOMIZATION=${CUSTOMIZATION} \ --DFAISS_URL=${CUSTOMIZED_FAISS_URL} \ -.." -echo ${CMAKE_CMD} -${CMAKE_CMD} - -if [[ ${RUN_CPPLINT} == "ON" ]]; then - # cpplint check - make lint - if [ $? -ne 0 ]; then - echo "ERROR! cpplint check failed" - exit 1 - fi - echo "cpplint check passed!" - - # clang-format check - make check-clang-format - if [ $? -ne 0 ]; then - echo "ERROR! clang-format check failed" - exit 1 - fi - echo "clang-format check passed!" - -# # clang-tidy check -# make check-clang-tidy -# if [ $? -ne 0 ]; then -# echo "ERROR! clang-tidy check failed" -# rm -f CMakeCache.txt -# exit 1 -# fi -# echo "clang-tidy check passed!" -else - # compile and build - make -j8 || exit 1 - make install || exit 1 -fi diff --git a/ci/jenkins/step/build.groovy b/ci/jenkins/step/build.groovy index 14d0414f4f..5b3a46818d 100644 --- a/ci/jenkins/step/build.groovy +++ b/ci/jenkins/step/build.groovy @@ -1,8 +1,7 @@ timeout(time: 60, unit: 'MINUTES') { - dir ("ci/jenkins/scripts") { - sh "./build.sh -l" + dir ("ci/scripts") { withCredentials([usernamePassword(credentialsId: "${params.JFROG_CREDENTIALS_ID}", usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) { - sh "export JFROG_ARTFACTORY_URL='${params.JFROG_ARTFACTORY_URL}' && export JFROG_USER_NAME='${USERNAME}' && export JFROG_PASSWORD='${PASSWORD}' && ./build.sh -t ${params.BUILD_TYPE} -o /opt/milvus -d /opt/milvus -j -u -c" + sh "export JFROG_ARTFACTORY_URL='${params.JFROG_ARTFACTORY_URL}' && export JFROG_USER_NAME='${USERNAME}' && export JFROG_PASSWORD='${PASSWORD}' && ./build.sh -t ${params.BUILD_TYPE} -o /opt/milvus -d /opt/milvus -l -j -u -c" } } } diff --git a/ci/jenkins/step/coverage.groovy b/ci/jenkins/step/coverage.groovy index ff2e3e6fa2..75ac7b48b5 100644 --- a/ci/jenkins/step/coverage.groovy +++ b/ci/jenkins/step/coverage.groovy @@ -1,5 +1,5 @@ timeout(time: 30, unit: 'MINUTES') { - dir ("ci/jenkins/scripts") { + dir ("ci/scripts") { sh "./coverage.sh -o /opt/milvus -u root -p 123456 -t \$POD_IP" // Set some env variables so codecov detection script works correctly withCredentials([[$class: 'StringBinding', credentialsId: "${env.PIPELINE_NAME}-codecov-token", variable: 'CODECOV_TOKEN']]) { diff --git a/ci/scripts/build.sh b/ci/scripts/build.sh index a8bd339ff5..444537fe6b 100755 --- a/ci/scripts/build.sh +++ b/ci/scripts/build.sh @@ -18,7 +18,7 @@ INSTALL_PREFIX="/opt/milvus" BUILD_COVERAGE="OFF" USE_JFROG_CACHE="OFF" RUN_CPPLINT="OFF" -CPU_VERSION="ON" +GPU_VERSION="OFF" WITH_MKL="OFF" CUDA_COMPILER=/usr/local/cuda/bin/nvcc @@ -35,7 +35,7 @@ do CORE_BUILD_DIR=$OPTARG # CORE_BUILD_DIR ;; g) - CPU_VERSION="OFF"; + GPU_VERSION="ON"; ;; u) echo "Build and run unittest cases" ; @@ -84,13 +84,13 @@ if [[ ! -d ${CORE_BUILD_DIR} ]]; then mkdir ${CORE_BUILD_DIR} fi -pushd ${CORE_BUILD_DIR} +cd ${CORE_BUILD_DIR} CMAKE_CMD="cmake \ -DCMAKE_INSTALL_PREFIX=${INSTALL_PREFIX} -DCMAKE_BUILD_TYPE=${BUILD_TYPE} \ -DCMAKE_CUDA_COMPILER=${CUDA_COMPILER} \ --DMILVUS_CPU_VERSION=${CPU_VERSION} \ +-DMILVUS_GPU_VERSION=${GPU_VERSION} \ -DBUILD_UNIT_TEST=${BUILD_UNITTEST} \ -DBUILD_COVERAGE=${BUILD_COVERAGE} \ -DUSE_JFROG_CACHE=${USE_JFROG_CACHE} \ @@ -100,6 +100,34 @@ ${MILVUS_CORE_DIR}" echo ${CMAKE_CMD} ${CMAKE_CMD} + +if [[ ${RUN_CPPLINT} == "ON" ]]; then + # cpplint check + make lint + if [ $? -ne 0 ]; then + echo "ERROR! cpplint check failed" + exit 1 + fi + echo "cpplint check passed!" + + # clang-format check + make check-clang-format + if [ $? -ne 0 ]; then + echo "ERROR! clang-format check failed" + exit 1 + fi + echo "clang-format check passed!" + +# # clang-tidy check +# make check-clang-tidy +# if [ $? -ne 0 ]; then +# echo "ERROR! clang-tidy check failed" +# rm -f CMakeCache.txt +# exit 1 +# fi +# echo "clang-tidy check passed!" +fi + # compile and build make -j8 || exit 1 make install || exit 1 diff --git a/ci/jenkins/scripts/coverage.sh b/ci/scripts/coverage.sh similarity index 86% rename from ci/jenkins/scripts/coverage.sh rename to ci/scripts/coverage.sh index 5c9d010d46..17bc08ad00 100755 --- a/ci/jenkins/scripts/coverage.sh +++ b/ci/scripts/coverage.sh @@ -9,18 +9,22 @@ done SCRIPTS_DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" INSTALL_PREFIX="/opt/milvus" -CMAKE_BUILD_DIR="${SCRIPTS_DIR}/../../../core/cmake_build" +MILVUS_CORE_DIR="${SCRIPTS_DIR}/../../core" +CORE_BUILD_DIR="${MILVUS_CORE_DIR}/cmake_build" MYSQL_USER_NAME=root MYSQL_PASSWORD=123456 MYSQL_HOST='127.0.0.1' MYSQL_PORT='3306' -while getopts "o:u:p:t:h" arg +while getopts "o:b:u:p:t:h" arg do case $arg in o) INSTALL_PREFIX=$OPTARG ;; + b) + CORE_BUILD_DIR=$OPTARG # CORE_BUILD_DIR + ;; u) MYSQL_USER_NAME=$OPTARG ;; @@ -35,13 +39,14 @@ do parameter: -o: milvus install prefix(default: /opt/milvus) +-b: core code build directory -u: mysql account -p: mysql password -t: mysql host -h: help usage: -./coverage.sh -o \${INSTALL_PREFIX} -u \${MYSQL_USER} -p \${MYSQL_PASSWORD} -t \${MYSQL_HOST} [-h] +./coverage.sh -o \${INSTALL_PREFIX} -b \$${CORE_BUILD_DIR} -u \${MYSQL_USER} -p \${MYSQL_PASSWORD} -t \${MYSQL_HOST} [-h] " exit 0 ;; @@ -63,12 +68,14 @@ FILE_INFO_OUTPUT="output.info" FILE_INFO_OUTPUT_NEW="output_new.info" DIR_LCOV_OUTPUT="lcov_out" -DIR_GCNO="${CMAKE_BUILD_DIR}" +DIR_GCNO="${CORE_BUILD_DIR}" DIR_UNITTEST="${INSTALL_PREFIX}/unittest" +cd ${SCRIPTS_DIR} + # delete old code coverage info files -rm -rf lcov_out -rm -f FILE_INFO_BASE FILE_INFO_MILVUS FILE_INFO_OUTPUT FILE_INFO_OUTPUT_NEW +rm -rf ${DIR_LCOV_OUTPUT} +rm -f ${FILE_INFO_BASE} ${FILE_INFO_MILVUS} ${FILE_INFO_OUTPUT} ${FILE_INFO_OUTPUT_NEW} MYSQL_DB_NAME=milvus_`date +%s%N` From 613c74afcf40c163734da5bcdbfeed137daab649 Mon Sep 17 00:00:00 2001 From: ZhifengZhang-CN Date: Wed, 13 Nov 2019 13:43:44 +0800 Subject: [PATCH 087/103] rename server_config.template to server_cpu_config.template --- core/conf/{server_config.template => server_cpu_config.template} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename core/conf/{server_config.template => server_cpu_config.template} (100%) diff --git a/core/conf/server_config.template b/core/conf/server_cpu_config.template similarity index 100% rename from core/conf/server_config.template rename to core/conf/server_cpu_config.template From 03f297fff4a81c6efc05d42a690dcf40a42b8246 Mon Sep 17 00:00:00 2001 From: ZhifengZhang-CN Date: Wed, 13 Nov 2019 13:52:16 +0800 Subject: [PATCH 088/103] format server_cpu_config.template and server_gpu_config.template --- ci/jenkins/step/build.groovy | 2 +- core/conf/server_cpu_config.template | 8 ++------ core/conf/server_gpu_config.template | 2 +- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/ci/jenkins/step/build.groovy b/ci/jenkins/step/build.groovy index 5b3a46818d..ee0ca020a6 100644 --- a/ci/jenkins/step/build.groovy +++ b/ci/jenkins/step/build.groovy @@ -1,7 +1,7 @@ timeout(time: 60, unit: 'MINUTES') { dir ("ci/scripts") { withCredentials([usernamePassword(credentialsId: "${params.JFROG_CREDENTIALS_ID}", usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) { - sh "export JFROG_ARTFACTORY_URL='${params.JFROG_ARTFACTORY_URL}' && export JFROG_USER_NAME='${USERNAME}' && export JFROG_PASSWORD='${PASSWORD}' && ./build.sh -t ${params.BUILD_TYPE} -o /opt/milvus -d /opt/milvus -l -j -u -c" + sh "export JFROG_ARTFACTORY_URL='${params.JFROG_ARTFACTORY_URL}' && export JFROG_USER_NAME='${USERNAME}' && export JFROG_PASSWORD='${PASSWORD}' && ./build.sh -t ${params.BUILD_TYPE} -o /opt/milvus -l -j -u -c" } } } diff --git a/core/conf/server_cpu_config.template b/core/conf/server_cpu_config.template index 8fc31366e3..6c95126390 100644 --- a/core/conf/server_cpu_config.template +++ b/core/conf/server_cpu_config.template @@ -32,16 +32,12 @@ cache_config: cpu_cache_threshold: 0.85 # percentage of data that will be kept when cache cleanup is triggered, must be in range (0.0, 1.0] cache_insert_data: false # whether to load inserted data into cache, must be a boolean -#Uncomment the following config if you are using GPU version -# gpu_cache_capacity: 4 # GB, GPU memory used for cache, must be a positive integer -# gpu_cache_threshold: 0.85 # percentage of data that will be kept when cache cleanup is triggered, must be in range (0.0, 1.0] - engine_config: use_blas_threshold: 1100 # if nq < use_blas_threshold, use SSE, faster with fluctuated response times # if nq >= use_blas_threshold, use OpenBlas, slower with stable response times gpu_search_threshold: 1000 # threshold beyond which the search computation is executed on GPUs only resource_config: - search_resources: # define the devices used for search computation, must be in format: cpu or gpux + search_resources: # define the device used for search computation - cpu - index_build_device: cpu # CPU / GPU used for building index, must be in format: cpu / gpux + index_build_device: cpu # CPU used for building index diff --git a/core/conf/server_gpu_config.template b/core/conf/server_gpu_config.template index db172f4f1d..154db5d134 100644 --- a/core/conf/server_gpu_config.template +++ b/core/conf/server_gpu_config.template @@ -42,4 +42,4 @@ resource_config: search_resources: # define the devices used for search computation, must be in format: cpu or gpux - cpu - gpu0 - index_build_device: gpu0 # GPU used for building index, must be in format: gpux + index_build_device: gpu0 # CPU / GPU used for building index, must be in format: cpu or gpux From 37487a8fee0ddf404a48119933bb47cbb0949703 Mon Sep 17 00:00:00 2001 From: ZhifengZhang-CN Date: Wed, 13 Nov 2019 13:55:46 +0800 Subject: [PATCH 089/103] fix make error --- ci/jenkins/step/build.groovy | 2 +- core/CMakeLists.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/ci/jenkins/step/build.groovy b/ci/jenkins/step/build.groovy index ee0ca020a6..bae4259a6f 100644 --- a/ci/jenkins/step/build.groovy +++ b/ci/jenkins/step/build.groovy @@ -1,7 +1,7 @@ timeout(time: 60, unit: 'MINUTES') { dir ("ci/scripts") { withCredentials([usernamePassword(credentialsId: "${params.JFROG_CREDENTIALS_ID}", usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) { - sh "export JFROG_ARTFACTORY_URL='${params.JFROG_ARTFACTORY_URL}' && export JFROG_USER_NAME='${USERNAME}' && export JFROG_PASSWORD='${PASSWORD}' && ./build.sh -t ${params.BUILD_TYPE} -o /opt/milvus -l -j -u -c" + sh "export JFROG_ARTFACTORY_URL='${params.JFROG_ARTFACTORY_URL}' && export JFROG_USER_NAME='${USERNAME}' && export JFROG_PASSWORD='${PASSWORD}' && ./build.sh -t ${params.BUILD_TYPE} -o /opt/milvus -l -g -j -u -c" } } } diff --git a/core/CMakeLists.txt b/core/CMakeLists.txt index bacab79612..d2ba516fec 100644 --- a/core/CMakeLists.txt +++ b/core/CMakeLists.txt @@ -178,7 +178,7 @@ endif () if (MILVUS_GPU_VERSION) configure_file(${CMAKE_CURRENT_SOURCE_DIR}/conf/server_gpu_config.template ${CMAKE_CURRENT_SOURCE_DIR}/conf/server_config.yaml) else() - configure_file(${CMAKE_CURRENT_SOURCE_DIR}/conf/server_config.template ${CMAKE_CURRENT_SOURCE_DIR}/conf/server_config.yaml) + configure_file(${CMAKE_CURRENT_SOURCE_DIR}/conf/server_cpu_config.template ${CMAKE_CURRENT_SOURCE_DIR}/conf/server_config.yaml) endif() configure_file(${CMAKE_CURRENT_SOURCE_DIR}/conf/log_config.template ${CMAKE_CURRENT_SOURCE_DIR}/conf/log_config.conf) From 46cce2cd8eded0b6731f3673f4e6caeffd495c94 Mon Sep 17 00:00:00 2001 From: ZhifengZhang-CN Date: Wed, 13 Nov 2019 14:08:14 +0800 Subject: [PATCH 090/103] format ci/scripts/coverage.sh --- ci/scripts/coverage.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/scripts/coverage.sh b/ci/scripts/coverage.sh index 17bc08ad00..1590e445ba 100755 --- a/ci/scripts/coverage.sh +++ b/ci/scripts/coverage.sh @@ -46,7 +46,7 @@ parameter: -h: help usage: -./coverage.sh -o \${INSTALL_PREFIX} -b \$${CORE_BUILD_DIR} -u \${MYSQL_USER} -p \${MYSQL_PASSWORD} -t \${MYSQL_HOST} [-h] +./coverage.sh -o \${INSTALL_PREFIX} -b \${CORE_BUILD_DIR} -u \${MYSQL_USER} -p \${MYSQL_PASSWORD} -t \${MYSQL_HOST} [-h] " exit 0 ;; From afbe9152e90e76f8a9de6716c14fdf7fe65c9b1c Mon Sep 17 00:00:00 2001 From: "yudong.cai" Date: Wed, 13 Nov 2019 14:34:07 +0800 Subject: [PATCH 091/103] #275 rename c++ sdk IndexType --- CHANGELOG.md | 1 + .../sdk/examples/partition/src/ClientTest.cpp | 2 +- .../src/sdk/examples/simple/src/ClientTest.cpp | 2 +- core/src/sdk/examples/utils/Utils.cpp | 18 ++++++++++-------- core/src/sdk/include/MilvusApi.h | 12 ++++++------ 5 files changed, 19 insertions(+), 16 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 02fb05d9cc..ce1af3a29b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,7 @@ Please mark all change in change log and use the ticket from JIRA. - \#226 - Experimental shards middleware for Milvus ## Improvement +- \#275 - Rename C++ SDK IndexType - \#284 - Change C++ SDK to shared library - \#260 - C++ SDK README diff --git a/core/src/sdk/examples/partition/src/ClientTest.cpp b/core/src/sdk/examples/partition/src/ClientTest.cpp index 6e4a7d1826..e58de0ce27 100644 --- a/core/src/sdk/examples/partition/src/ClientTest.cpp +++ b/core/src/sdk/examples/partition/src/ClientTest.cpp @@ -41,7 +41,7 @@ constexpr int64_t NQ = 5; constexpr int64_t TOP_K = 10; constexpr int64_t NPROBE = 32; constexpr int64_t SEARCH_TARGET = 5000; // change this value, result is different -constexpr milvus::IndexType INDEX_TYPE = milvus::IndexType::gpu_ivfsq8; +constexpr milvus::IndexType INDEX_TYPE = milvus::IndexType::IVFSQ8; constexpr int32_t N_LIST = 15000; constexpr int32_t PARTITION_COUNT = 5; constexpr int32_t TARGET_PARTITION = 3; diff --git a/core/src/sdk/examples/simple/src/ClientTest.cpp b/core/src/sdk/examples/simple/src/ClientTest.cpp index 9045168f2a..9b217a1ed0 100644 --- a/core/src/sdk/examples/simple/src/ClientTest.cpp +++ b/core/src/sdk/examples/simple/src/ClientTest.cpp @@ -41,7 +41,7 @@ constexpr int64_t TOP_K = 10; constexpr int64_t NPROBE = 32; constexpr int64_t SEARCH_TARGET = 5000; // change this value, result is different constexpr int64_t ADD_VECTOR_LOOP = 5; -constexpr milvus::IndexType INDEX_TYPE = milvus::IndexType::gpu_ivfsq8; +constexpr milvus::IndexType INDEX_TYPE = milvus::IndexType::IVFSQ8; constexpr int32_t N_LIST = 15000; milvus::TableSchema diff --git a/core/src/sdk/examples/utils/Utils.cpp b/core/src/sdk/examples/utils/Utils.cpp index c527cf47e1..3f96a5b6df 100644 --- a/core/src/sdk/examples/utils/Utils.cpp +++ b/core/src/sdk/examples/utils/Utils.cpp @@ -89,14 +89,16 @@ Utils::MetricTypeName(const milvus::MetricType& metric_type) { std::string Utils::IndexTypeName(const milvus::IndexType& index_type) { switch (index_type) { - case milvus::IndexType::cpu_idmap: - return "cpu idmap"; - case milvus::IndexType::gpu_ivfflat: - return "gpu ivflat"; - case milvus::IndexType::gpu_ivfsq8: - return "gpu ivfsq8"; - case milvus::IndexType::mix_nsg: - return "mix nsg"; + case milvus::IndexType::FLAT: + return "FLAT"; + case milvus::IndexType::IVFFLAT: + return "IVFFLAT"; + case milvus::IndexType::IVFSQ8: + return "IVFSQ8"; + case milvus::IndexType::NSG: + return "NSG"; + case milvus::IndexType::IVFSQ8H: + return "IVFSQ8H"; default: return "Unknown index type"; } diff --git a/core/src/sdk/include/MilvusApi.h b/core/src/sdk/include/MilvusApi.h index 8c92375649..ea1dbf6d75 100644 --- a/core/src/sdk/include/MilvusApi.h +++ b/core/src/sdk/include/MilvusApi.h @@ -31,12 +31,12 @@ namespace milvus { * @brief Index Type */ enum class IndexType { - invalid = 0, - cpu_idmap, - gpu_ivfflat, - gpu_ivfsq8, - mix_nsg, - ivfsq8h, + INVALID = 0, + FLAT = 1, + IVFFLAT = 2, + IVFSQ8 = 3, + NSG = 4, + IVFSQ8H = 5, }; enum class MetricType { From bf69d10f86b821a9456bdc062213e174b6b515bb Mon Sep 17 00:00:00 2001 From: Zhiru Zhu Date: Wed, 13 Nov 2019 14:53:37 +0800 Subject: [PATCH 092/103] add find faiss --- core/build.sh | 2 +- core/src/CMakeLists.txt | 2 +- core/src/index/build.sh | 6 +- core/src/index/cmake/DefineOptionsCore.cmake | 2 +- core/src/index/cmake/FindFAISS.cmake | 44 +++++++++ .../index/cmake/ThirdPartyPackagesCore.cmake | 96 ++++++++++--------- core/src/index/knowhere/CMakeLists.txt | 2 +- core/src/index/unittest/CMakeLists.txt | 2 +- .../unittest/faiss_benchmark/CMakeLists.txt | 2 +- .../index/unittest/faiss_ori/CMakeLists.txt | 2 +- .../index/unittest/test_nsg/CMakeLists.txt | 2 +- 11 files changed, 105 insertions(+), 57 deletions(-) create mode 100644 core/src/index/cmake/FindFAISS.cmake diff --git a/core/build.sh b/core/build.sh index e844528ad3..5abdaf175a 100755 --- a/core/build.sh +++ b/core/build.sh @@ -117,7 +117,7 @@ CMAKE_CMD="cmake \ -DCUSTOMIZATION=${CUSTOMIZATION} \ -DFAISS_URL=${CUSTOMIZED_FAISS_URL} \ -DMILVUS_GPU_VERSION=${GPU_VERSION} \ --DBUILD_FAISS_WITH_MKL=${WITH_MKL} \ +-DFAISS_WITH_MKL=${WITH_MKL} \ ../" echo ${CMAKE_CMD} ${CMAKE_CMD} diff --git a/core/src/CMakeLists.txt b/core/src/CMakeLists.txt index 79b5e0f1da..4d55fbee18 100644 --- a/core/src/CMakeLists.txt +++ b/core/src/CMakeLists.txt @@ -24,7 +24,7 @@ include_directories(${MILVUS_ENGINE_SRC}/grpc/gen-status) include_directories(${MILVUS_ENGINE_SRC}/grpc/gen-milvus) add_subdirectory(index) -if (BUILD_FAISS_WITH_MKL) +if (FAISS_WITH_MKL) add_compile_definitions("WITH_MKL") endif () diff --git a/core/src/index/build.sh b/core/src/index/build.sh index a77a16cf0b..357ac5693a 100644 --- a/core/src/index/build.sh +++ b/core/src/index/build.sh @@ -5,7 +5,7 @@ BUILD_UNITTEST="OFF" INSTALL_PREFIX=$(pwd)/cmake_build MAKE_CLEAN="OFF" PROFILING="OFF" -BUILD_FAISS_WITH_MKL="OFF" +FAISS_WITH_MKL="OFF" USE_JFROG_CACHE="OFF" while getopts "p:d:t:uhrcgmj" arg @@ -31,7 +31,7 @@ do PROFILING="ON" ;; m) - BUILD_FAISS_WITH_MKL="ON" + FAISS_WITH_MKL="ON" ;; j) USE_JFROG_CACHE="ON" @@ -74,7 +74,7 @@ if [[ ${MAKE_CLEAN} == "ON" ]]; then -DCMAKE_BUILD_TYPE=${BUILD_TYPE} \ -DCMAKE_CUDA_COMPILER=${CUDA_COMPILER} \ -DMILVUS_ENABLE_PROFILING=${PROFILING} \ - -DBUILD_FAISS_WITH_MKL=${BUILD_FAISS_WITH_MKL} \ + -DFAISS_WITH_MKL=${FAISS_WITH_MKL} \ -DUSE_JFROG_CACHE=${USE_JFROG_CACHE} \ ../" echo ${CMAKE_CMD} diff --git a/core/src/index/cmake/DefineOptionsCore.cmake b/core/src/index/cmake/DefineOptionsCore.cmake index 2f050cdf4e..787a9c484f 100644 --- a/core/src/index/cmake/DefineOptionsCore.cmake +++ b/core/src/index/cmake/DefineOptionsCore.cmake @@ -79,7 +79,7 @@ define_option(KNOWHERE_WITH_FAISS "Build with FAISS library" ON) define_option(KNOWHERE_WITH_FAISS_GPU_VERSION "Build with FAISS GPU version" ON) -define_option(BUILD_FAISS_WITH_MKL "Build FAISS with MKL" OFF) +define_option(FAISS_WITH_MKL "Build FAISS with MKL" OFF) #---------------------------------------------------------------------- set_option_category("Test and benchmark") diff --git a/core/src/index/cmake/FindFAISS.cmake b/core/src/index/cmake/FindFAISS.cmake new file mode 100644 index 0000000000..e0e6337bce --- /dev/null +++ b/core/src/index/cmake/FindFAISS.cmake @@ -0,0 +1,44 @@ +set(FAISS_STATIC_LIB_NAME ${CMAKE_STATIC_LIBRARY_PREFIX}faiss${CMAKE_STATIC_LIBRARY_SUFFIX}) + +# First, find via if specified FAISS_ROOT +if (FAISS_ROOT) + find_library(FAISS_STATIC_LIB + NAMES ${FAISS_STATIC_LIB_NAME} + PATHS ${FAISS_ROOT} + PATH_SUFFIXES "lib" + NO_DEFAULT_PATH + ) + find_path(FAISS_INCLUDE_DIR + NAMES "faiss/Index.h" + PATHS ${FAISS_ROOT} + NO_DEFAULT_PATH + PATH_SUFFIXES "include" + ) +endif () + +find_package_handle_standard_args(FAISS REQUIRED_VARS FAISS_STATIC_LIB FAISS_INCLUDE_DIR) + +if (FAISS_FOUND) + if (NOT TARGET faiss) + add_library(faiss STATIC IMPORTED) + + set_target_properties( + faiss + PROPERTIES + IMPORTED_LOCATION "${FAISS_STATIC_LIB}" + INTERFACE_INCLUDE_DIRECTORIES "${FAISS_INCLUDE_DIR}" + ) + + if (FAISS_WITH_MKL) + set_target_properties( + faiss + PROPERTIES + INTERFACE_LINK_LIBRARIES "${MKL_LIBS}") + else () + set_target_properties( + faiss + PROPERTIES + INTERFACE_LINK_LIBRARIES "openblas;lapack") + endif () + endif () +endif () diff --git a/core/src/index/cmake/ThirdPartyPackagesCore.cmake b/core/src/index/cmake/ThirdPartyPackagesCore.cmake index 60ac4b9eac..8563e37e87 100644 --- a/core/src/index/cmake/ThirdPartyPackagesCore.cmake +++ b/core/src/index/cmake/ThirdPartyPackagesCore.cmake @@ -21,6 +21,7 @@ set(KNOWHERE_THIRDPARTY_DEPENDENCIES GTest LAPACK OpenBLAS + MKL ) message(STATUS "Using ${KNOWHERE_DEPENDENCY_SOURCE} approach to find dependencies") @@ -43,6 +44,8 @@ macro(build_dependency DEPENDENCY_NAME) build_openblas() elseif ("${DEPENDENCY_NAME}" STREQUAL "FAISS") build_faiss() + elseif ("${DEPENDENCY_NAME}" STREQUAL "MKL") + build_mkl() else () message(FATAL_ERROR "Unknown thirdparty dependency to build: ${DEPENDENCY_NAME}") endif () @@ -51,9 +54,9 @@ endmacro() macro(resolve_dependency DEPENDENCY_NAME) if (${DEPENDENCY_NAME}_SOURCE STREQUAL "AUTO") find_package(${DEPENDENCY_NAME} MODULE) - if(NOT ${${DEPENDENCY_NAME}_FOUND}) - build_dependency(${DEPENDENCY_NAME}) - endif() + if (NOT ${${DEPENDENCY_NAME}_FOUND}) + build_dependency(${DEPENDENCY_NAME}) + endif () elseif (${DEPENDENCY_NAME}_SOURCE STREQUAL "BUNDLED") build_dependency(${DEPENDENCY_NAME}) elseif (${DEPENDENCY_NAME}_SOURCE STREQUAL "SYSTEM") @@ -238,11 +241,11 @@ if (CUSTOMIZATION) # set(FAISS_MD5 "f3b2ce3364c3fa7febd3aa7fdd0fe380") # commit-id 694e03458e6b69ce8a62502f71f69a614af5af8f branch-0.3.0 # set(FAISS_MD5 "bb30722c22390ce5f6759ccb216c1b2a") # commit-id d324db297475286afe107847c7fb7a0f9dc7e90e branch-0.3.0 set(FAISS_MD5 "2293cdb209c3718e3b19f3edae8b32b3") # commit-id a13c1205dc52977a9ad3b33a14efa958604a8bff branch-0.3.0 - endif() -else() + endif () +else () set(FAISS_SOURCE_URL "https://github.com/JinHai-CN/faiss/archive/1.6.0.tar.gz") set(FAISS_MD5 "b02c1a53234f5acc9bea1b0c55524f50") -endif() +endif () message(STATUS "FAISS URL = ${FAISS_SOURCE_URL}") if (DEFINED ENV{KNOWHERE_ARROW_URL}) @@ -673,28 +676,46 @@ if (KNOWHERE_BUILD_TESTS AND NOT TARGET googletest_ep) include_directories(SYSTEM ${GTEST_INCLUDE_DIR}) endif () +# ---------------------------------------------------------------------- +# MKL + +macro(build_mkl) + + if (FAISS_WITH_MKL) + if (EXISTS "/proc/cpuinfo") + FILE(READ /proc/cpuinfo PROC_CPUINFO) + + SET(VENDOR_ID_RX "vendor_id[ \t]*:[ \t]*([a-zA-Z]+)\n") + STRING(REGEX MATCH "${VENDOR_ID_RX}" VENDOR_ID "${PROC_CPUINFO}") + STRING(REGEX REPLACE "${VENDOR_ID_RX}" "\\1" VENDOR_ID "${VENDOR_ID}") + + if (NOT ${VENDOR_ID} STREQUAL "GenuineIntel") + set(FAISS_WITH_MKL OFF) + endif () + endif () + + find_path(MKL_LIB_PATH + NAMES "libmkl_intel_ilp64.a" "libmkl_gnu_thread.a" "libmkl_core.a" + PATH_SUFFIXES "intel/compilers_and_libraries_${MKL_VERSION}/linux/mkl/lib/intel64/") + if (${MKL_LIB_PATH} STREQUAL "MKL_LIB_PATH-NOTFOUND") + message(FATAL_ERROR "Could not find MKL libraries") + endif () + message(STATUS "MKL lib path = ${MKL_LIB_PATH}") + + set(MKL_LIBS + ${MKL_LIB_PATH}/libmkl_intel_ilp64.a + ${MKL_LIB_PATH}/libmkl_gnu_thread.a + ${MKL_LIB_PATH}/libmkl_core.a + ) + endif () +endmacro() + # ---------------------------------------------------------------------- # FAISS macro(build_faiss) message(STATUS "Building FAISS-${FAISS_VERSION} from source") - if (NOT DEFINED BUILD_FAISS_WITH_MKL) - set(BUILD_FAISS_WITH_MKL OFF) - endif () - - if (EXISTS "/proc/cpuinfo") - FILE(READ /proc/cpuinfo PROC_CPUINFO) - - SET(VENDOR_ID_RX "vendor_id[ \t]*:[ \t]*([a-zA-Z]+)\n") - STRING(REGEX MATCH "${VENDOR_ID_RX}" VENDOR_ID "${PROC_CPUINFO}") - STRING(REGEX REPLACE "${VENDOR_ID_RX}" "\\1" VENDOR_ID "${VENDOR_ID}") - - if (NOT ${VENDOR_ID} STREQUAL "GenuineIntel") - set(BUILD_FAISS_WITH_MKL OFF) - endif () - endif () - set(FAISS_PREFIX "${INDEX_BINARY_DIR}/faiss_ep-prefix/src/faiss_ep") set(FAISS_INCLUDE_DIR "${FAISS_PREFIX}/include") set(FAISS_STATIC_LIB @@ -706,30 +727,11 @@ macro(build_faiss) "CXXFLAGS=${EP_CXX_FLAGS}" --without-python) - set(FAISS_CFLAGS ${EP_C_FLAGS}) - set(FAISS_CXXFLAGS ${EP_CXX_FLAGS}) - - if (BUILD_FAISS_WITH_MKL) - - find_path(MKL_LIB_PATH - NAMES "libmkl_intel_ilp64.a" "libmkl_gnu_thread.a" "libmkl_core.a" - PATH_SUFFIXES "intel/compilers_and_libraries_${MKL_VERSION}/linux/mkl/lib/intel64/") - if (${MKL_LIB_PATH} STREQUAL "MKL_LIB_PATH-NOTFOUND") - message(FATAL_ERROR "Could not find MKL libraries") - endif () - message(STATUS "Build Faiss with MKL. MKL lib path = ${MKL_LIB_PATH}") - - set(MKL_LIBS - ${MKL_LIB_PATH}/libmkl_intel_ilp64.a - ${MKL_LIB_PATH}/libmkl_gnu_thread.a - ${MKL_LIB_PATH}/libmkl_core.a - ) - + if (FAISS_WITH_MKL) set(FAISS_CONFIGURE_ARGS ${FAISS_CONFIGURE_ARGS} "CPPFLAGS=-DFINTEGER=long -DMKL_ILP64 -m64 -I${MKL_LIB_PATH}/../../include" "LDFLAGS=-L${MKL_LIB_PATH}" ) - else () message(STATUS "Build Faiss with OpenBlas/LAPACK") set(FAISS_CONFIGURE_ARGS ${FAISS_CONFIGURE_ARGS} @@ -770,7 +772,7 @@ macro(build_faiss) BUILD_BYPRODUCTS ${FAISS_STATIC_LIB}) - if (NOT BUILD_FAISS_WITH_MKL) + if (NOT FAISS_WITH_MKL) ExternalProject_Add_StepDependencies(faiss_ep build openblas_ep lapack_ep) endif () @@ -800,7 +802,7 @@ macro(build_faiss) BUILD_BYPRODUCTS ${FAISS_STATIC_LIB}) - if (NOT BUILD_FAISS_WITH_MKL) + if (NOT FAISS_WITH_MKL) ExternalProject_Add_StepDependencies(faiss_ep build openblas_ep lapack_ep) endif () @@ -815,7 +817,7 @@ macro(build_faiss) IMPORTED_LOCATION "${FAISS_STATIC_LIB}" INTERFACE_INCLUDE_DIRECTORIES "${FAISS_INCLUDE_DIR}" ) - if (BUILD_FAISS_WITH_MKL) + if (FAISS_WITH_MKL) set_target_properties( faiss PROPERTIES @@ -834,7 +836,9 @@ endmacro() if (KNOWHERE_WITH_FAISS AND NOT TARGET faiss_ep) - if (NOT BUILD_FAISS_WITH_MKL) + if (FAISS_WITH_MKL) + resolve_dependency(MKL) + else () resolve_dependency(OpenBLAS) get_target_property(OPENBLAS_INCLUDE_DIR openblas INTERFACE_INCLUDE_DIRECTORIES) include_directories(SYSTEM "${OPENBLAS_INCLUDE_DIR}") diff --git a/core/src/index/knowhere/CMakeLists.txt b/core/src/index/knowhere/CMakeLists.txt index 373965acf0..5f8e4d6970 100644 --- a/core/src/index/knowhere/CMakeLists.txt +++ b/core/src/index/knowhere/CMakeLists.txt @@ -54,7 +54,7 @@ set(depend_libs gfortran pthread ) -if (BUILD_FAISS_WITH_MKL) +if (FAISS_WITH_MKL) set(depend_libs ${depend_libs} "-Wl,--start-group \ ${MKL_LIB_PATH}/libmkl_intel_ilp64.a \ diff --git a/core/src/index/unittest/CMakeLists.txt b/core/src/index/unittest/CMakeLists.txt index 13277aa40e..71636738aa 100644 --- a/core/src/index/unittest/CMakeLists.txt +++ b/core/src/index/unittest/CMakeLists.txt @@ -8,7 +8,7 @@ set(depend_libs faiss arrow "${ARROW_LIB_DIR}/libjemalloc_pic.a" ) -if (BUILD_FAISS_WITH_MKL) +if (FAISS_WITH_MKL) set(depend_libs ${depend_libs} "-Wl,--start-group \ ${MKL_LIB_PATH}/libmkl_intel_ilp64.a \ diff --git a/core/src/index/unittest/faiss_benchmark/CMakeLists.txt b/core/src/index/unittest/faiss_benchmark/CMakeLists.txt index 6cad5abda5..e2fd7a6c85 100644 --- a/core/src/index/unittest/faiss_benchmark/CMakeLists.txt +++ b/core/src/index/unittest/faiss_benchmark/CMakeLists.txt @@ -15,7 +15,7 @@ if (KNOWHERE_GPU_VERSION) faiss hdf5 arrow ${ARROW_LIB_DIR}/libjemalloc_pic.a ) - if (BUILD_FAISS_WITH_MKL) + if (FAISS_WITH_MKL) set(depend_libs ${depend_libs} "-Wl,--start-group \ ${MKL_LIB_PATH}/libmkl_intel_ilp64.a \ diff --git a/core/src/index/unittest/faiss_ori/CMakeLists.txt b/core/src/index/unittest/faiss_ori/CMakeLists.txt index 829a27fd87..907076ca6e 100644 --- a/core/src/index/unittest/faiss_ori/CMakeLists.txt +++ b/core/src/index/unittest/faiss_ori/CMakeLists.txt @@ -10,7 +10,7 @@ if (KNOWHERE_GPU_VERSION) faiss arrow ${ARROW_LIB_DIR}/libjemalloc_pic.a ) - if (BUILD_FAISS_WITH_MKL) + if (FAISS_WITH_MKL) set(depend_libs ${depend_libs} "-Wl,--start-group \ ${MKL_LIB_PATH}/libmkl_intel_ilp64.a \ diff --git a/core/src/index/unittest/test_nsg/CMakeLists.txt b/core/src/index/unittest/test_nsg/CMakeLists.txt index 269c0d2139..82dbb5110c 100644 --- a/core/src/index/unittest/test_nsg/CMakeLists.txt +++ b/core/src/index/unittest/test_nsg/CMakeLists.txt @@ -5,7 +5,7 @@ add_definitions(-std=c++11 -O3 -lboost -march=native -Wall -DINFO) find_package(OpenMP REQUIRED) -if (OPENMP_FOUND) +if (OpenMP_FOUND) set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${OpenMP_C_FLAGS}") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS}") else () From 1f60505ef764ae3ef60f75b3dba4175b9a7fca04 Mon Sep 17 00:00:00 2001 From: Zhiru Zhu Date: Wed, 13 Nov 2019 15:17:44 +0800 Subject: [PATCH 093/103] update CHANGELOG --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index ce1af3a29b..be8bc36b3b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,7 @@ Please mark all change in change log and use the ticket from JIRA. - \#12 - Pure CPU version for Milvus - \#77 - Support table partition - \#226 - Experimental shards middleware for Milvus +- \#314 - add Find FAISS in CMake ## Improvement - \#275 - Rename C++ SDK IndexType From f20e5705b4144706c19d4e1a24dd7cd64ffc0fdd Mon Sep 17 00:00:00 2001 From: Zhiru Zhu Date: Wed, 13 Nov 2019 15:43:24 +0800 Subject: [PATCH 094/103] Add Q&A for 'protocol https not supported or disable in libcurl' issue --- CHANGELOG.md | 3 ++- install.md | 26 ++++++++++++++++++++++---- 2 files changed, 24 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index be8bc36b3b..d7b4c85ae8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,12 +11,13 @@ Please mark all change in change log and use the ticket from JIRA. - \#12 - Pure CPU version for Milvus - \#77 - Support table partition - \#226 - Experimental shards middleware for Milvus -- \#314 - add Find FAISS in CMake ## Improvement - \#275 - Rename C++ SDK IndexType - \#284 - Change C++ SDK to shared library - \#260 - C++ SDK README +- \#314 - add Find FAISS in CMake +- \#310 - Add Q&A for 'protocol https not supported or disable in libcurl' issue ## Task diff --git a/install.md b/install.md index f869520ba3..f92174ae36 100644 --- a/install.md +++ b/install.md @@ -3,7 +3,10 @@ ## Software requirements - Ubuntu 18.04 or higher -- CMake 3.14 or higher +- CMake 3.12 or higher + +##### For GPU version, you will also need: + - CUDA 10.0 or higher - NVIDIA driver 418 or higher @@ -12,17 +15,20 @@ ### Step 1 Install dependencies ```shell -$ cd [Milvus sourcecode path]/core +$ cd [Milvus root path]/core $ ./ubuntu_build_deps.sh ``` ### Step 2 Build ```shell -$ cd [Milvus sourcecode path]/core +$ cd [Milvus root path]/core $ ./build.sh -t Debug or $ ./build.sh -t Release + +For GPU version, add -g option +$ ./build.sh -g ``` When the build is completed, all the stuff that you need in order to run Milvus will be installed under `[Milvus root path]/core/milvus`. @@ -36,7 +42,7 @@ $ cd [Milvus root path]/core/milvus Add `lib/` directory to `LD_LIBRARY_PATH` ``` -$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/milvus/lib +$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:[Milvus root path]/core/milvus/lib ``` Then start Milvus server: @@ -51,3 +57,15 @@ To stop Milvus server, run: ```shell $ ./stop_server.sh ``` + +## Troubleshooting +1. If you encounter the following error when compiling: +`protocol https not supported or disabled in libcurl`. +First, make sure you have `libcurl4-openssl-dev` installed in your system. +Then try reinstall CMake from source with `--system-curl` option: +``` +$ ./bootstrap --system-curl +$ make +$ sudo make install +``` + From 12f87049a09712b30d73646adb90457d7de830ac Mon Sep 17 00:00:00 2001 From: Zhiru Zhu Date: Wed, 13 Nov 2019 15:46:45 +0800 Subject: [PATCH 095/103] update CHANGELOG --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index be8bc36b3b..fefd4e54c1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,12 +11,12 @@ Please mark all change in change log and use the ticket from JIRA. - \#12 - Pure CPU version for Milvus - \#77 - Support table partition - \#226 - Experimental shards middleware for Milvus -- \#314 - add Find FAISS in CMake ## Improvement - \#275 - Rename C++ SDK IndexType - \#284 - Change C++ SDK to shared library - \#260 - C++ SDK README +- \#314 - add Find FAISS in CMake ## Task From 1fa4c2531a2762130b9707bace08805c4601a07f Mon Sep 17 00:00:00 2001 From: Zhiru Zhu Date: Wed, 13 Nov 2019 16:01:03 +0800 Subject: [PATCH 096/103] remove FAISS_URL --- .../index/cmake/ThirdPartyPackagesCore.cmake | 22 +++---------------- 1 file changed, 3 insertions(+), 19 deletions(-) diff --git a/core/src/index/cmake/ThirdPartyPackagesCore.cmake b/core/src/index/cmake/ThirdPartyPackagesCore.cmake index 8563e37e87..854b295b1a 100644 --- a/core/src/index/cmake/ThirdPartyPackagesCore.cmake +++ b/core/src/index/cmake/ThirdPartyPackagesCore.cmake @@ -225,28 +225,12 @@ foreach (_VERSION_ENTRY ${TOOLCHAIN_VERSIONS_TXT}) set(${_LIB_NAME} "${_LIB_VERSION}") endforeach () -if (CUSTOMIZATION) - execute_process(COMMAND wget -q --method HEAD ${FAISS_URL} RESULT_VARIABLE return_code) - message(STATUS "Check the remote cache file ${FAISS_URL}. return code = ${return_code}") - if (NOT return_code EQUAL 0) - MESSAGE(FATAL_ERROR "Can't access to ${FAISS_URL}") - else () - set(FAISS_SOURCE_URL ${FAISS_URL}) - # set(FAISS_MD5 "a589663865a8558205533c8ac414278c") - # set(FAISS_MD5 "57da9c4f599cc8fa4260488b1c96e1cc") # commit-id 6dbdf75987c34a2c853bd172ea0d384feea8358c branch-0.2.0 - # set(FAISS_MD5 "21deb1c708490ca40ecb899122c01403") # commit-id 643e48f479637fd947e7b93fa4ca72b38ecc9a39 branch-0.2.0 - # set(FAISS_MD5 "072db398351cca6e88f52d743bbb9fa0") # commit-id 3a2344d04744166af41ef1a74449d68a315bfe17 branch-0.2.1 - # set(FAISS_MD5 "c89ea8e655f5cdf58f42486f13614714") # commit-id 9c28a1cbb88f41fa03b03d7204106201ad33276b branch-0.2.1 - # set(FAISS_MD5 "87fdd86351ffcaf3f80dc26ade63c44b") # commit-id 841a156e67e8e22cd8088e1b58c00afbf2efc30b branch-0.2.1 - # set(FAISS_MD5 "f3b2ce3364c3fa7febd3aa7fdd0fe380") # commit-id 694e03458e6b69ce8a62502f71f69a614af5af8f branch-0.3.0 - # set(FAISS_MD5 "bb30722c22390ce5f6759ccb216c1b2a") # commit-id d324db297475286afe107847c7fb7a0f9dc7e90e branch-0.3.0 - set(FAISS_MD5 "2293cdb209c3718e3b19f3edae8b32b3") # commit-id a13c1205dc52977a9ad3b33a14efa958604a8bff branch-0.3.0 - endif () +if (DEFINED ENV{FAISS_SOURCE_URL}) + set(FAISS_SOURCE_URL "$ENV{FAISS_SOURCE_URL}") else () - set(FAISS_SOURCE_URL "https://github.com/JinHai-CN/faiss/archive/1.6.0.tar.gz") + set(FAISS_SOURCE_URL "https://github.com/milvus-io/faiss/archive/${FAISS_VERSION}.tar.gz") set(FAISS_MD5 "b02c1a53234f5acc9bea1b0c55524f50") endif () -message(STATUS "FAISS URL = ${FAISS_SOURCE_URL}") if (DEFINED ENV{KNOWHERE_ARROW_URL}) set(ARROW_SOURCE_URL "$ENV{KNOWHERE_ARROW_URL}") From de1aa04bd0cf5666d325cb0ff08a61689bb6c8f9 Mon Sep 17 00:00:00 2001 From: Zhiru Zhu Date: Wed, 13 Nov 2019 16:11:31 +0800 Subject: [PATCH 097/103] update build.sh --- core/build.sh | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/core/build.sh b/core/build.sh index 5abdaf175a..9b690a0261 100755 --- a/core/build.sh +++ b/core/build.sh @@ -14,16 +14,10 @@ CUSTOMIZATION="OFF" # default use ori faiss CUDA_COMPILER=/usr/local/cuda/bin/nvcc GPU_VERSION="OFF" #defaults to CPU version WITH_MKL="OFF" +FAISS_ROOT="" +FAISS_SOURCE="BUNDLED" -CUSTOMIZED_FAISS_URL="${FAISS_URL:-NONE}" -wget -q --method HEAD ${CUSTOMIZED_FAISS_URL} -if [ $? -eq 0 ]; then - CUSTOMIZATION="ON" -else - CUSTOMIZATION="OFF" -fi - -while getopts "p:d:t:ulrcgjhxzm" arg +while getopts "p:d:t:f:ulrcgjhxzm" arg do case $arg in p) @@ -35,6 +29,10 @@ do t) BUILD_TYPE=$OPTARG # BUILD_TYPE ;; + f) + FAISS_ROOT=$OPTARG + FAISS_SOURCE="AUTO" + ;; u) echo "Build and run unittest cases" ; BUILD_UNITTEST="ON"; @@ -73,6 +71,7 @@ parameter: -p: install prefix(default: $(pwd)/milvus) -d: db data path(default: /tmp/milvus) -t: build type(default: Debug) +-f: faiss root path(default: empty) -u: building unit test options(default: OFF) -l: run cpplint, clang-format and clang-tidy(default: OFF) -r: remove previous build directory(default: OFF) @@ -84,7 +83,7 @@ parameter: -h: help usage: -./build.sh -p \${INSTALL_PREFIX} -t \${BUILD_TYPE} [-u] [-l] [-r] [-c] [-z] [-j] [-g] [-m] [-h] +./build.sh -p \${INSTALL_PREFIX} -t \${BUILD_TYPE} -f \${FAISS_ROOT} [-u] [-l] [-r] [-c] [-z] [-j] [-g] [-m] [-h] " exit 0 ;; @@ -109,13 +108,14 @@ CMAKE_CMD="cmake \ -DBUILD_UNIT_TEST=${BUILD_UNITTEST} \ -DCMAKE_INSTALL_PREFIX=${INSTALL_PREFIX} -DCMAKE_BUILD_TYPE=${BUILD_TYPE} \ +-DFAISS_ROOT=${FAISS_ROOT} \ +-DFAISS_SOURCE=${FAISS_SOURCE} \ -DCMAKE_CUDA_COMPILER=${CUDA_COMPILER} \ -DBUILD_COVERAGE=${BUILD_COVERAGE} \ -DMILVUS_DB_PATH=${DB_PATH} \ -DMILVUS_ENABLE_PROFILING=${PROFILING} \ -DUSE_JFROG_CACHE=${USE_JFROG_CACHE} \ -DCUSTOMIZATION=${CUSTOMIZATION} \ --DFAISS_URL=${CUSTOMIZED_FAISS_URL} \ -DMILVUS_GPU_VERSION=${GPU_VERSION} \ -DFAISS_WITH_MKL=${WITH_MKL} \ ../" From 7aef840e48f3bfc014ce2d48176a4977594afbe6 Mon Sep 17 00:00:00 2001 From: quicksilver Date: Wed, 13 Nov 2019 16:14:59 +0800 Subject: [PATCH 098/103] Fix determine Milvus version from git branch name bug --- core/CMakeLists.txt | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/core/CMakeLists.txt b/core/CMakeLists.txt index d2ba516fec..2c991a47f6 100644 --- a/core/CMakeLists.txt +++ b/core/CMakeLists.txt @@ -32,10 +32,8 @@ string(REGEX REPLACE "\n" "" BUILD_TIME ${BUILD_TIME}) message(STATUS "Build time = ${BUILD_TIME}") MACRO(GET_GIT_BRANCH_NAME GIT_BRANCH_NAME) - execute_process(COMMAND "git" rev-parse --abbrev-ref HEAD OUTPUT_VARIABLE ${GIT_BRANCH_NAME}) - if (GIT_BRANCH_NAME STREQUAL "") - execute_process(COMMAND "git" symbolic-ref --short -q HEAD OUTPUT_VARIABLE ${GIT_BRANCH_NAME}) - endif () + execute_process(COMMAND sh "-c" "git log --decorate | head -n 1 | sed 's/.*(\\(.*\\))/\\1/' | sed 's/.* \\(.*\\),.*/\\1/' | sed 's=[a-zA-Z]*\/==g'" + OUTPUT_VARIABLE ${GIT_BRANCH_NAME}) ENDMACRO(GET_GIT_BRANCH_NAME) GET_GIT_BRANCH_NAME(GIT_BRANCH_NAME) From 85e76aad8d94437e6a3e899bcf62b7332ddd747f Mon Sep 17 00:00:00 2001 From: Zhiru Zhu Date: Wed, 13 Nov 2019 16:18:18 +0800 Subject: [PATCH 099/103] minor update --- install.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/install.md b/install.md index f92174ae36..4d2088a3be 100644 --- a/install.md +++ b/install.md @@ -26,8 +26,10 @@ $ cd [Milvus root path]/core $ ./build.sh -t Debug or $ ./build.sh -t Release +``` -For GPU version, add -g option +By default, it will build CPU version. To build GPU version, add `-g` option +``` $ ./build.sh -g ``` From f251d26bbb8c2a4caef1377e78490e64cae06676 Mon Sep 17 00:00:00 2001 From: wxyu Date: Wed, 13 Nov 2019 16:35:37 +0800 Subject: [PATCH 100/103] Reside src/external in thirdparty --- CHANGELOG.md | 1 + ci/jenkins/scripts/coverage.sh | 2 +- core/CMakeLists.txt | 1 + core/build-support/lint_exclusions.txt | 2 +- core/coverage.sh | 2 +- core/src/CMakeLists.txt | 13 +++++++------ core/src/index/knowhere/knowhere/common/Log.h | 2 +- core/src/index/unittest/CMakeLists.txt | 2 +- core/src/main.cpp | 2 +- core/src/utils/Json.h | 2 +- core/src/utils/Log.h | 2 +- core/src/utils/LogUtil.h | 2 +- .../easyloggingpp/easylogging++.cc | 0 .../easyloggingpp/easylogging++.h | 0 .../external => thirdparty}/nlohmann/json.hpp | 0 core/unittest/CMakeLists.txt | 15 ++++++++------- core/unittest/main.cpp | 2 +- core/unittest/wrapper/CMakeLists.txt | 2 +- core/unittest/wrapper/test_wrapper.cpp | 2 +- 19 files changed, 29 insertions(+), 25 deletions(-) rename core/{src/external => thirdparty}/easyloggingpp/easylogging++.cc (100%) rename core/{src/external => thirdparty}/easyloggingpp/easylogging++.h (100%) rename core/{src/external => thirdparty}/nlohmann/json.hpp (100%) diff --git a/CHANGELOG.md b/CHANGELOG.md index 02fb05d9cc..6ee8bfef0d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ Please mark all change in change log and use the ticket from JIRA. ## Bug - \#246 - Exclude src/external folder from code coverage for jenkin ci +- \#248 - Reside src/external in thirdparty ## Feature - \#12 - Pure CPU version for Milvus diff --git a/ci/jenkins/scripts/coverage.sh b/ci/jenkins/scripts/coverage.sh index 5c9d010d46..fa4fd76490 100755 --- a/ci/jenkins/scripts/coverage.sh +++ b/ci/jenkins/scripts/coverage.sh @@ -132,7 +132,7 @@ ${LCOV_CMD} -r "${FILE_INFO_OUTPUT}" -o "${FILE_INFO_OUTPUT_NEW}" \ "*/src/server/Server.cpp" \ "*/src/server/DBWrapper.cpp" \ "*/src/server/grpc_impl/GrpcServer.cpp" \ - "*/src/external/*" + "*/thirdparty/*" if [ $? -ne 0 ]; then echo "gen ${FILE_INFO_OUTPUT_NEW} failed" diff --git a/core/CMakeLists.txt b/core/CMakeLists.txt index 65094b19a3..11e417397b 100644 --- a/core/CMakeLists.txt +++ b/core/CMakeLists.txt @@ -112,6 +112,7 @@ endif () set(MILVUS_SOURCE_DIR ${PROJECT_SOURCE_DIR}) set(MILVUS_BINARY_DIR ${PROJECT_BINARY_DIR}) set(MILVUS_ENGINE_SRC ${PROJECT_SOURCE_DIR}/src) +set(MILVUS_THIRDPARTY_SRC ${PROJECT_SOURCE_DIR}/thirdparty) include(ExternalProject) include(DefineOptions) diff --git a/core/build-support/lint_exclusions.txt b/core/build-support/lint_exclusions.txt index 2be060f121..34d469fccb 100644 --- a/core/build-support/lint_exclusions.txt +++ b/core/build-support/lint_exclusions.txt @@ -6,5 +6,5 @@ *easylogging++* *SqliteMetaImpl.cpp *src/grpc* -*src/external* +*thirdparty* *milvus/include* \ No newline at end of file diff --git a/core/coverage.sh b/core/coverage.sh index 9011e290e5..833b1b0050 100755 --- a/core/coverage.sh +++ b/core/coverage.sh @@ -122,7 +122,7 @@ ${LCOV_CMD} -r "${FILE_INFO_OUTPUT}" -o "${FILE_INFO_OUTPUT_NEW}" \ "*/src/server/Server.cpp" \ "*/src/server/DBWrapper.cpp" \ "*/src/server/grpc_impl/GrpcServer.cpp" \ - "*/src/external/*" + "*/thirdparty/*" if [ $? -ne 0 ]; then echo "generate ${FILE_INFO_OUTPUT_NEW} failed" diff --git a/core/src/CMakeLists.txt b/core/src/CMakeLists.txt index 79b5e0f1da..e0cc5d0855 100644 --- a/core/src/CMakeLists.txt +++ b/core/src/CMakeLists.txt @@ -19,6 +19,7 @@ include_directories(${MILVUS_SOURCE_DIR}) include_directories(${MILVUS_ENGINE_SRC}) +include_directories(${MILVUS_THIRDPARTY_SRC}) include_directories(${MILVUS_ENGINE_SRC}/grpc/gen-status) include_directories(${MILVUS_ENGINE_SRC}/grpc/gen-milvus) @@ -65,11 +66,11 @@ set(scheduler_files ${scheduler_task_files} ) -aux_source_directory(${MILVUS_ENGINE_SRC}/external/easyloggingpp external_easyloggingpp_files) -aux_source_directory(${MILVUS_ENGINE_SRC}/external/nlohmann external_nlohmann_files) -set(external_files - ${external_easyloggingpp_files} - ${external_nlohmann_files} +aux_source_directory(${MILVUS_THIRDPARTY_SRC}/easyloggingpp thirdparty_easyloggingpp_files) +aux_source_directory(${MILVUS_THIRDPARTY_SRC}/nlohmann thirdparty_nlohmann_files) +set(thirdparty_files + ${thirdparty_easyloggingpp_files} + ${thirdparty_nlohmann_files} ) aux_source_directory(${MILVUS_ENGINE_SRC}/server server_files) @@ -85,7 +86,7 @@ set(engine_files ${db_insert_files} ${db_meta_files} ${metrics_files} - ${external_files} + ${thirdparty_files} ${utils_files} ${wrapper_files} ) diff --git a/core/src/index/knowhere/knowhere/common/Log.h b/core/src/index/knowhere/knowhere/common/Log.h index 369e7143af..e50a28a15e 100644 --- a/core/src/index/knowhere/knowhere/common/Log.h +++ b/core/src/index/knowhere/knowhere/common/Log.h @@ -17,7 +17,7 @@ #pragma once -#include "external/easyloggingpp/easylogging++.h" +#include "easyloggingpp/easylogging++.h" namespace knowhere { diff --git a/core/src/index/unittest/CMakeLists.txt b/core/src/index/unittest/CMakeLists.txt index 13277aa40e..d27c7e8860 100644 --- a/core/src/index/unittest/CMakeLists.txt +++ b/core/src/index/unittest/CMakeLists.txt @@ -27,7 +27,7 @@ set(basic_libs ) set(util_srcs - ${MILVUS_ENGINE_SRC}/external/easyloggingpp/easylogging++.cc + ${MILVUS_THIRDPARTY_SRC}/easyloggingpp/easylogging++.cc ${INDEX_SOURCE_DIR}/knowhere/knowhere/index/vector_index/helpers/FaissIO.cpp ${INDEX_SOURCE_DIR}/knowhere/knowhere/index/vector_index/helpers/IndexParameter.cpp ${INDEX_SOURCE_DIR}/knowhere/knowhere/adapter/Structure.cpp diff --git a/core/src/main.cpp b/core/src/main.cpp index c5b2d2dffe..401736c34f 100644 --- a/core/src/main.cpp +++ b/core/src/main.cpp @@ -22,7 +22,7 @@ #include #include -#include "external/easyloggingpp/easylogging++.h" +#include "easyloggingpp/easylogging++.h" #include "metrics/Metrics.h" #include "server/Server.h" #include "src/config.h" diff --git a/core/src/utils/Json.h b/core/src/utils/Json.h index e5e74a3b67..235d6ac9c1 100644 --- a/core/src/utils/Json.h +++ b/core/src/utils/Json.h @@ -16,7 +16,7 @@ // under the License. #pragma once -#include "external/nlohmann/json.hpp" +#include "nlohmann/json.hpp" namespace milvus { using json = nlohmann::json; diff --git a/core/src/utils/Log.h b/core/src/utils/Log.h index 4aa3146b01..cef72a866d 100644 --- a/core/src/utils/Log.h +++ b/core/src/utils/Log.h @@ -17,7 +17,7 @@ #pragma once -#include "external/easyloggingpp/easylogging++.h" +#include "easyloggingpp/easylogging++.h" namespace milvus { diff --git a/core/src/utils/LogUtil.h b/core/src/utils/LogUtil.h index 7e5afd087a..69607c4631 100644 --- a/core/src/utils/LogUtil.h +++ b/core/src/utils/LogUtil.h @@ -17,7 +17,7 @@ #pragma once -#include "external/easyloggingpp/easylogging++.h" +#include "easyloggingpp/easylogging++.h" #include "utils/Status.h" #include diff --git a/core/src/external/easyloggingpp/easylogging++.cc b/core/thirdparty/easyloggingpp/easylogging++.cc similarity index 100% rename from core/src/external/easyloggingpp/easylogging++.cc rename to core/thirdparty/easyloggingpp/easylogging++.cc diff --git a/core/src/external/easyloggingpp/easylogging++.h b/core/thirdparty/easyloggingpp/easylogging++.h similarity index 100% rename from core/src/external/easyloggingpp/easylogging++.h rename to core/thirdparty/easyloggingpp/easylogging++.h diff --git a/core/src/external/nlohmann/json.hpp b/core/thirdparty/nlohmann/json.hpp similarity index 100% rename from core/src/external/nlohmann/json.hpp rename to core/thirdparty/nlohmann/json.hpp diff --git a/core/unittest/CMakeLists.txt b/core/unittest/CMakeLists.txt index e485bd729a..10ab362e77 100644 --- a/core/unittest/CMakeLists.txt +++ b/core/unittest/CMakeLists.txt @@ -23,6 +23,7 @@ endforeach () include_directories(${MILVUS_SOURCE_DIR}) include_directories(${MILVUS_ENGINE_SRC}) +include_directories(${MILVUS_THIRDPARTY_SRC}) include_directories(${CMAKE_CURRENT_SOURCE_DIR}) aux_source_directory(${MILVUS_ENGINE_SRC}/cache cache_files) @@ -57,11 +58,11 @@ set(scheduler_files ${scheduler_optimizer_files} ) -aux_source_directory(${MILVUS_ENGINE_SRC}/external/easyloggingpp external_easyloggingpp_files) -aux_source_directory(${MILVUS_ENGINE_SRC}/external/nlohmann external_nlohmann_files) -set(external_files - ${external_easyloggingpp_files} - ${external_nlohmann_files} +aux_source_directory(${MILVUS_THIRDPARTY_SRC}/easyloggingpp thirdparty_easyloggingpp_files) +aux_source_directory(${MILVUS_THIRDPARTY_SRC}/nlohmann thirdparty_nlohmann_files) +set(thirdparty_files + ${thirdparty_easyloggingpp_files} + ${thirdparty_nlohmann_files} ) aux_source_directory(${MILVUS_ENGINE_SRC}/server server_files) @@ -79,7 +80,7 @@ set(helper_files ${MILVUS_ENGINE_SRC}/utils/Status.cpp ${MILVUS_ENGINE_SRC}/utils/StringHelpFunctions.cpp ${MILVUS_ENGINE_SRC}/utils/ValidationUtil.cpp - ${MILVUS_ENGINE_SRC}/external/easyloggingpp/easylogging++.cc + ${MILVUS_THIRDPARTY_SRC}/easyloggingpp/easylogging++.cc ) set(common_files @@ -90,7 +91,7 @@ set(common_files ${db_insert_files} ${db_meta_files} ${metrics_files} - ${external_files} + ${thirdparty_files} ${scheduler_files} ${wrapper_files} ${helper_files} diff --git a/core/unittest/main.cpp b/core/unittest/main.cpp index ff70f3f2cf..7452b95821 100644 --- a/core/unittest/main.cpp +++ b/core/unittest/main.cpp @@ -18,7 +18,7 @@ #include #include -#include "external/easyloggingpp/easylogging++.h" +#include "easyloggingpp/easylogging++.h" INITIALIZE_EASYLOGGINGPP diff --git a/core/unittest/wrapper/CMakeLists.txt b/core/unittest/wrapper/CMakeLists.txt index 232abf6e1a..a320ef723d 100644 --- a/core/unittest/wrapper/CMakeLists.txt +++ b/core/unittest/wrapper/CMakeLists.txt @@ -32,7 +32,7 @@ set(wrapper_files set(util_files utils.cpp - ${MILVUS_ENGINE_SRC}/external/easyloggingpp/easylogging++.cc + ${MILVUS_THIRDPARTY_SRC}/easyloggingpp/easylogging++.cc ${MILVUS_ENGINE_SRC}/utils/Status.cpp ) diff --git a/core/unittest/wrapper/test_wrapper.cpp b/core/unittest/wrapper/test_wrapper.cpp index 097cb69075..ddfdfa2fad 100644 --- a/core/unittest/wrapper/test_wrapper.cpp +++ b/core/unittest/wrapper/test_wrapper.cpp @@ -15,7 +15,7 @@ // specific language governing permissions and limitations // under the License. -#include "external/easyloggingpp/easylogging++.h" +#include "easyloggingpp/easylogging++.h" #include "wrapper/VecIndex.h" #ifdef MILVUS_GPU_VERSION From 5f4866616d0f6d33fa6a9e596a5979a1f1ff4cd7 Mon Sep 17 00:00:00 2001 From: Zhiru Zhu Date: Wed, 13 Nov 2019 16:36:40 +0800 Subject: [PATCH 101/103] update build.sh in ci --- ci/scripts/build.sh | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/ci/scripts/build.sh b/ci/scripts/build.sh index 444537fe6b..27962ccb54 100755 --- a/ci/scripts/build.sh +++ b/ci/scripts/build.sh @@ -15,6 +15,7 @@ CORE_BUILD_DIR="${MILVUS_CORE_DIR}/cmake_build" BUILD_TYPE="Debug" BUILD_UNITTEST="OFF" INSTALL_PREFIX="/opt/milvus" +FAISS_ROOT="" BUILD_COVERAGE="OFF" USE_JFROG_CACHE="OFF" RUN_CPPLINT="OFF" @@ -22,7 +23,7 @@ GPU_VERSION="OFF" WITH_MKL="OFF" CUDA_COMPILER=/usr/local/cuda/bin/nvcc -while getopts "o:t:b:gulcjmh" arg +while getopts "o:t:b:f:gulcjmh" arg do case $arg in o) @@ -34,6 +35,9 @@ do b) CORE_BUILD_DIR=$OPTARG # CORE_BUILD_DIR ;; + f) + FAISS_ROOT=$OPTARG # FAISS ROOT PATH + ;; g) GPU_VERSION="ON"; ;; @@ -60,6 +64,7 @@ parameter: -o: install prefix(default: /opt/milvus) -t: build type(default: Debug) -b: core code build directory +-f: faiss root path -g: gpu version -u: building unit test options(default: OFF) -l: run cpplint, clang-format and clang-tidy(default: OFF) @@ -69,7 +74,7 @@ parameter: -h: help usage: -./build.sh -o \${INSTALL_PREFIX} -t \${BUILD_TYPE} -b \${CORE_BUILD_DIR} [-u] [-l] [-c] [-j] [-m] [-h] +./build.sh -o \${INSTALL_PREFIX} -t \${BUILD_TYPE} -b \${CORE_BUILD_DIR} -f \${FAISS_ROOT} [-u] [-l] [-c] [-j] [-m] [-h] " exit 0 ;; @@ -94,8 +99,10 @@ CMAKE_CMD="cmake \ -DBUILD_UNIT_TEST=${BUILD_UNITTEST} \ -DBUILD_COVERAGE=${BUILD_COVERAGE} \ -DUSE_JFROG_CACHE=${USE_JFROG_CACHE} \ --DBUILD_FAISS_WITH_MKL=${WITH_MKL} \ +-DFAISS_ROOT=${FAISS_ROOT} \ +-DFAISS_WITH_MKL=${WITH_MKL} \ -DArrow_SOURCE=AUTO \ +-DFAISS_SOURCE=AUTO \ ${MILVUS_CORE_DIR}" echo ${CMAKE_CMD} ${CMAKE_CMD} From ff7452a21fc9f8ef80aa0162c1d9f250d8aec6c4 Mon Sep 17 00:00:00 2001 From: Zhiru Zhu Date: Wed, 13 Nov 2019 16:53:23 +0800 Subject: [PATCH 102/103] fix faiss source url --- core/src/index/cmake/ThirdPartyPackagesCore.cmake | 2 +- core/src/index/thirdparty/versions.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/core/src/index/cmake/ThirdPartyPackagesCore.cmake b/core/src/index/cmake/ThirdPartyPackagesCore.cmake index 854b295b1a..c046bc3a56 100644 --- a/core/src/index/cmake/ThirdPartyPackagesCore.cmake +++ b/core/src/index/cmake/ThirdPartyPackagesCore.cmake @@ -228,7 +228,7 @@ endforeach () if (DEFINED ENV{FAISS_SOURCE_URL}) set(FAISS_SOURCE_URL "$ENV{FAISS_SOURCE_URL}") else () - set(FAISS_SOURCE_URL "https://github.com/milvus-io/faiss/archive/${FAISS_VERSION}.tar.gz") + set(FAISS_SOURCE_URL "https://github.com/JinHai-CN/faiss/archive/${FAISS_VERSION}.tar.gz") set(FAISS_MD5 "b02c1a53234f5acc9bea1b0c55524f50") endif () diff --git a/core/src/index/thirdparty/versions.txt b/core/src/index/thirdparty/versions.txt index 380c9dedad..efcef26fa9 100644 --- a/core/src/index/thirdparty/versions.txt +++ b/core/src/index/thirdparty/versions.txt @@ -3,5 +3,5 @@ BOOST_VERSION=1.70.0 GTEST_VERSION=1.8.1 LAPACK_VERSION=v3.8.0 OPENBLAS_VERSION=v0.3.6 -FAISS_VERSION=branch-0.3.0 +FAISS_VERSION=1.6.0 MKL_VERSION=2019.5.281 From d02fee0da9318833f9c13fe0384c1ad7133208cf Mon Sep 17 00:00:00 2001 From: JinHai-CN Date: Wed, 13 Nov 2019 18:41:49 +0800 Subject: [PATCH 103/103] Update CHANGELOG for release --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3f78e72192..a0667d8a32 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,7 +1,7 @@ # Changelog Please mark all change in change log and use the ticket from JIRA. -# Milvus 0.5.3 (TODO) +# Milvus 0.5.3 (2019-11-13) ## Bug - \#258 - Bytes type in proto cause big-endian/little-endian problem