diff --git a/shards/README.md b/shards/README.md new file mode 100644 index 0000000000..dba6f785aa --- /dev/null +++ b/shards/README.md @@ -0,0 +1,264 @@ +# Mishards - An Experimental Sharding Middleware + +Milvus aims to achieve efficient similarity search and analytics for massive-scale vectors. A standalone Milvus instance can easily handle vector search among billion-scale vectors. However, for 10 billion, 100 billion or even larger datasets, a Milvus cluster is needed. + +Ideally, this cluster can be accessed and used just as the standalone instance, meanwhile it satisfies the business requirements such as low latency and high concurrency. + +This page meant to demonstrates how to use Mishards, an experimental sharding middleware for Milvus, to establish an orchestrated cluster. + +## What is Mishards + +Mishards is a middleware that is developed using Python. It provides unlimited extension of memory and computation capacity through request forwarding, read/write splitting, horizontal scalability and dynamic extension. It works as the proxy of the Milvus system. + +Using Mishards in Milvus cluster deployment is an experimental feature available for user test and feedback. + +## How Mishards works + +Mishards splits the upstream requests to sub-requests and forwards them to Milvus servers. When the search computation is completed, all results are collected by Mishards and sent back to the client. + +Below graph is a demonstration of the process: + +![mishards](https://raw.githubusercontent.com/milvus-io/docs/master/assets/mishards.png) + +## Mishards example codes + +Below examples codes demonstrate how to build from source code a Milvus server with Mishards on a standalone machine, as well as how to use Kubernetes to establish Milvus cluster with Mishards. + +Before executing these examples, make sure you meet the prerequisites of [Milvus installation](https://www.milvus.io/docs/en/userguide/install_milvus/). + +### Build from source code + +#### Prequisites + +Make sure Python 3.6 or higher is installed. + +#### Start Milvus and Mishards from source code + +Follow below steps to start a standalone Milvus instance with Mishards from source code: + +1. Clone milvus repository. + + ```shell + git clone + ``` + +2. Install Mishards dependencies. + + ```shell + $ cd milvus/shards + $ pip install -r requirements.txt + ``` + +3. Start Milvus server. + + ```shell + $ sudo nvidia-docker run --rm -d -p 19530:19530 -v /tmp/milvus/db:/opt/milvus/db milvusdb/milvus:0.5.0-d102119-ede20b + ``` + +4. Update path permissions. + + ```shell + $ sudo chown -R $USER:$USER /tmp/milvus + ``` + +5. Configure Mishards environmental variables. + + ```shell + $ cp mishards/.env.example mishards/.env + ``` + +6. Start Mishards server. + + ```shell + $ python mishards/main.py + ``` + +### Docker example + +The `all_in_one` example shows how to use Docker container to start 2 Milvus instances, 1 Mishards instance and 1 Jaeger instance. + + 1. Install [Docker Compose](https://docs.docker.com/compose/install/). + + 2. Build docker images for these instances. + + ```shell + $ make build + ``` + + 3. Start all instances. + + ```shell + $ make deploy + ``` + + 4. Confirm instance status. + + ```shell + $ make probe_deploy + Pass ==> Pass: Connected + Fail ==> Error: Fail connecting to server on 127.0.0.1:19530. Timeout + ``` + +To check the service tracing, open the [Jaeger page](http://127.0.0.1:16686/) on your browser. + +![jaegerui](https://raw.githubusercontent.com/milvus-io/docs/master/assets/jaegerui.png) + +![jaegertraces](https://raw.githubusercontent.com/milvus-io/docs/master/assets/jaegertraces.png) + +To stop all instances, use the following command: + +```shell +$ make clean_deploy +``` + +### Kubernetes example + +Using Kubernetes to deploy Milvus cluster requires that the developers have a basic understanding of [general concepts](https://kubernetes.io/docs/concepts/) of Kubernetes. + +This example mainly demonstrates how to use Kubernetes to establish a Milvus cluster containing 2 Milvus instances（1 read instance and 1 write instance), 1 MySQL instance and 1 Mishards instance. + +This example does not include tasks such as setting up Kubernetes cluster, [installing shared storage](https://kubernetes.io/docs/concepts/storage/volumes/) and using command tools such as [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/). + +Below is the architecture of Milvus cluster built upon Kubernetes: + +![k8s_arch](https://raw.githubusercontent.com/milvus-io/docs/master/assets/k8s_arch.png) + +#### Prerequisites + +- A Kubernetes cluster is already established. +- [nvidia-docker 2.0](https://github.com/nvidia/nvidia-docker/wiki/Installation-(version-2.0)) is already installed. +- Shared storage is already installed. +- kubectl is installed and can access the Kubernetes cluster. + +#### Use Kubernetes to build a Milvus cluster + +1. Start Milvus cluster + + ```shell + $ make cluster + ``` + +2. Confirm that Mishards is connected to Milvus. + + ```shell + $ make probe_cluster + Pass ==> Pass: Connected + ``` + +To check cluster status: + +```shell +$ make cluster_status +``` + +To delete the cluster: + +```shell +$ make clean_cluster +``` + +To add a read instance: + +```shell +$ cd kubernetes_demo +$ ./start.sh scale-ro-server 2 +``` + +To add a proxy instance: + +```shell +$ cd kubernetes_demo +$ ./start.sh scale-proxy 2 +``` + +To check cluster logs: + +```shell +$ kubectl logs -f --tail=1000 -n milvus milvus-ro-servers-0 +``` + +## Mishards Unit test + +**Unit test** + +```shell +$ cd milvus/shards +$ make test +``` + +**Code coverage test** + +```shell +$ cd milvus/shards +$ make coverage +``` + +**Code format check** + +```shell +$ cd milvus/shards +$ make style +``` + +## Mishards configuration + +### Overall configuration + +| Name | Required | Type | Default | Description | +| ------------- | -------- | ------- | ------- | ------------------------------------------------------------ | +| `Debug` | No | boolean | `True` | Choose if to enable `Debug` work mode. | +| `TIMEZONE` | No | string | `UTC` | Timezone | +| `MAX_RETRY` | No | integer | `3` | The maximum retry times allowed to connect to Milvus. | +| `SERVER_PORT` | No | integer | `19530` | Define the server port of Mishards. | +| `WOSERVER` | **Yes** | string | ` ` | Define the address of Milvus write instance. Currently, only static settings are supported. Format for reference: `tcp://127.0.0.1:19530`. | + +### Metadata + +| Name | Required | Type | Default | Description | +| ------------------------------ | -------- | ------- | ------- | ------------------------------------------------------------ | +| `SQLALCHEMY_DATABASE_URI` | **Yes** | string | ` ` | Define the database address for metadata storage. Format standard: RFC-738-style. For example: `mysql+pymysql://root:root@127.0.0.1:3306/milvus?charset=utf8mb4`. | +| `SQL_ECHO` | No | boolean | `False` | Choose if to print SQL statements. | +| `SQLALCHEMY_DATABASE_TEST_URI` | No | string | ` ` | Define the database address of metadata storage in test environment. | +| `SQL_TEST_ECHO` | No | boolean | `False` | Choose if to print SQL statements in test environment. | + +### Service discovery + +| Name | Required | Type | Default | Description | +| ------------------------------------- | -------- | ------- | ------------- | ------------------------------------------------------------ | +| `DISCOVERY_PLUGIN_PATH` | No | string | ` ` | Define the search path to locate the plug-in. The default path is used if the value is not set. | +| `DISCOVERY_CLASS_NAME` | No | string | `static` | Under the plug-in search path, search the class based on the class name, and instantiate it. Currently, the system provides 2 classes: `static` and `kubernetes`. | +| `DISCOVERY_STATIC_HOSTS` | No | list | `[]` | When `DISCOVERY_CLASS_NAME` is `static` , define a comma-separated service address list, for example`192.168.1.188,192.168.1.190`. | +| `DISCOVERY_STATIC_PORT` | No | integer | `19530` | When `DISCOVERY_CLASS_NAME` is `static`, define the server port. | +| `DISCOVERY_KUBERNETES_NAMESPACE` | No | string | ` ` | When `DISCOVERY_CLASS_NAME` is `kubernetes`, define the namespace of Milvus cluster. | +| `DISCOVERY_KUBERNETES_IN_CLUSTER` | No | boolean | `False` | When `DISCOVERY_CLASS_NAME` is `kubernetes` , choose if to run the server in Kubernetes. | +| `DISCOVERY_KUBERNETES_POLL_INTERVAL` | No | integer | `5` (Seconds) | When `DISCOVERY_CLASS_NAME` is `kubernetes` , define the listening cycle of the server. | +| `DISCOVERY_KUBERNETES_POD_PATT` | No | string | ` ` | When `DISCOVERY_CLASS_NAME` is `kubernetes` , map the regular expression of Milvus Pod. | +| `DISCOVERY_KUBERNETES_LABEL_SELECTOR` | No | string | ` ` | When `SD_PROVIDER` is `kubernetes`, map the label of Milvus Pod. For example: `tier=ro-servers`. | + +### Tracing + +| Name | Required | Type | Default | Description | +| ----------------------- | -------- | ------- | ---------- | ------------------------------------------------------------ | +| `TRACER_PLUGIN_PATH` | No | string | ` ` | Define the search path to locate the tracing plug-in. The default path is used if the value is not set. | +| `TRACER_CLASS_NAME` | No | string | ` ` | Under the plug-in search path, search the class based on the class name, and instantiate it. Currently, only `Jaeger` is supported. | +| `TRACING_SERVICE_NAME` | No | string | `mishards` | When `TRACING_CLASS_NAME` is [`Jaeger`](https://www.jaegertracing.io/docs/1.14/), the name of the tracing service. | +| `TRACING_SAMPLER_TYPE` | No | string | `const` | When `TRACING_CLASS_NAME` is [`Jaeger`](https://www.jaegertracing.io/docs/1.14/), the [sampling type](https://www.jaegertracing.io/docs/1.14/sampling/) of the tracing service. | +| `TRACING_SAMPLER_PARAM` | No | integer | `1` | When `TRACING_CLASS_NAME` is [`Jaeger`](https://www.jaegertracing.io/docs/1.14/), the [sampling frequency](https://www.jaegertracing.io/docs/1.14/sampling/) of the tracing service. | +| `TRACING_LOG_PAYLOAD` | No | boolean | `False` | When `TRACING_CLASS_NAME` is [`Jaeger`](https://www.jaegertracing.io/docs/1.14/), choose if to sample Payload. | + +### Logging + +| Name | Required | Type | Default | Description | +| ----------- | -------- | ------ | --------------- | ------------------------------------------------------------ | +| `LOG_LEVEL` | No | string | `DEBUG` | Log recording levels. Currently supports `DEBUG` ,`INFO` ,`WARNING` and `ERROR`. | +| `LOG_PATH` | No | string | `/tmp/mishards` | Log recording path. | +| `LOG_NAME` | No | string | `logfile` | Log recording name. | + +### Routing + +| Name | Required | Type | Default | Description | +| ------------------------ | -------- | ------ | ------------------------- | ------------------------------------------------------------ | +| `ROUTER_PLUGIN_PATH` | No | string | ` ` | Define the search path to locate the routing plug-in. The default path is used if the value is not set. | +| `ROUTER_CLASS_NAME` | No | string | `FileBasedHashRingRouter` | Under the plug-in search path, search the class based on the class name, and instantiate it. Currently, only `FileBasedHashRingRouter` is supported. | +| `ROUTER_CLASS_TEST_NAME` | No | string | `FileBasedHashRingRouter` | Under the plug-in search path, search the class based on the class name, and instantiate it. Currently, `FileBasedHashRingRouter` is supported for test environment only. | +