admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
milvus/tests
History
wei liu c6a1c49e02
enhance: Use Blocked Bloom Filter instead of basic bloom fitler impl. (#33405) 
issue: #32995
To speed up the construction and querying of Bloom filters, we chose a
blocked Bloom filter instead of a basic Bloom filter implementation.

WARN: This PR is compatible with old version bf impl, but if fall back
to old milvus version, it may causes bloom filter deserialize failed.

In single Bloom filter test cases with a capacity of 1,000,000 and a
false positive rate (FPR) of 0.001, the blocked Bloom filter is 5 times
faster than the basic Bloom filter in both querying and construction, at
the cost of a 30% increase in memory usage.

- Block BF construct time	{"time": "54.128131ms"}
- Block BF size	                {"size": 3021578}
- Block BF Test cost	        {"time": "55.407352ms"}
- Basic BF construct time	{"time": "210.262183ms"}
- Basic BF size	                {"size": 2396308}
- Basic BF Test cost	        {"time": "192.596229ms"}

In multi Bloom filter test cases with a capacity of 100,000, an FPR of
0.001, and 100 Bloom filters, we reuse the primary key locations for all
Bloom filters to avoid repeated hash computations. As a result, the
blocked Bloom filter is also 5 times faster than the basic Bloom filter
in querying.

- Block BF TestLocation cost    {"time": "529.97183ms"}
- Basic BF TestLocation cost	{"time": "3.197430181s"}

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-31 17:49:45 +08:00
..
benchmark
test: fix vulnerability of python lib (#32109)
2024-04-10 17:53:18 +08:00
docker
[automated] Update Pytest image changes (#33126)
2024-05-20 11:43:37 +08:00
go_client
…
integration
enhance: Use Blocked Bloom Filter instead of basic bloom fitler impl. (#33405)
2024-05-31 17:49:45 +08:00
java_client
…
python_client
test: add channel exclusive balance test and resource group test (#33093)
2024-05-31 13:55:52 +08:00
restful_client
test:fix retry for network error (#32804)
2024-05-07 14:01:29 +08:00
restful_client_v2
test: add different nq for restful v2 test (#33456)
2024-05-31 13:57:45 +08:00
scripts
test: update nightly ci (#33430)
2024-05-31 13:54:07 +08:00
OWNERS
[skip ci]Update OWNERS files (#11898)
2021-11-16 15:41:11 +08:00
README_CN.md
[skip ci]Specify simd type description (#11588)
2021-11-10 20:30:47 +08:00
README.md
Update doc for remove Centos7 (#26476)
2023-08-21 09:54:20 +08:00

README.md

Tests

E2E Test

Configuration Requirements

Operating System
Operating System Version
Amazon Linux 2023 or above
Ubuntu 20.04 or above
Mac 10.14 or above
Hardware
Hardware Type Recommended Configuration
CPU x86_64 architecture
Intel CPU Sandy Bridge or above
CPU Instruction Set
- SSE4_2
- AVX
- AVX2
- AVX512 or arm64 Linux/MacOS
Memory 16 GB or more
Software
Software Name Version
Docker 19.05 or above
Docker Compose 1.25.5 or above
jq 1.3 or above
kubectl 1.14 or above
helm 3.0 or above
kind 0.10.0 or above

Installing Dependencies

Troubleshooting Docker and Docker Compose
  1. Confirm that Docker Daemon is running：
$ docker info

  • Ensure that Docker is installed. Refer to the official installation instructions for Docker CE/EE.

  • Start the Docker Daemon if it is not already started.

  • To run Docker without root privileges, create a user group labeled docker, then add a user to the group with sudo usermod -aG docker $USER. Log out and log back into the terminal for the changes to take effect. For more information, see the official Docker documentation for Managing Docker as a Non-Root User.

  1. Check the version of Docker-Compose
$ docker-compose version

docker-compose version 1.25.5, build 8a1c60f6
docker-py version: 4.1.0
CPython version: 3.7.5
OpenSSL version: OpenSSL 1.1.1f  31 Mar 2020
Install jq
Install kubectl
Install helm
Install kind

Run E2E Tests

$ cd tests/scripts
$ ./e2e-k8s.sh

Getting help

You can get help with the following command:

$ ./e2e-k8s.sh --help