mirror of
https://gitee.com/milvus-io/milvus.git
synced 2025-12-28 14:35:27 +08:00
e4b0f48bc0
### **User description** Issue: #46504 test: create e2e test case for highlighter On branch feature/highlighter Changes to be committed: new file: milvus_client/test_milvus_client_highlighter.py ___ ### **PR Type** Tests ___ ### **Description** - Add comprehensive e2e test suite for LexicalHighlighter functionality - Test highlighter initialization with collection setup and data insertion - Validate highlighter with various parameters (tags, fragments, offsets) - Test edge cases including Chinese characters, long text, and invalid inputs - Verify error handling for invalid fragment sizes, offsets, and configurations ___ ### Diagram Walkthrough ```mermaid flowchart LR A["Test Suite Setup"] --> B["Highlighter Init Tests"] B --> C["Valid Test Cases"] C --> D["Fragment Parameters"] C --> E["Search Variations"] C --> F["Language Support"] B --> G["Invalid Test Cases"] G --> H["Parameter Validation"] G --> I["Error Handling"] ``` <details><summary><h3>File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><table> <tr> <td> <details> <summary><strong>test_milvus_client_highlighter.py</strong><dd><code>Add comprehensive LexicalHighlighter e2e test suite</code> </dd></summary> <hr> tests/python_client/milvus_client/test_milvus_client_highlighter.py <ul><li>Create new test file with 1163 lines of comprehensive highlighter test <br>cases<br> <li> Implement <code>TestMilvusClientHighlighterInit</code> class to initialize <br>collection with pre-defined test data including English, Chinese, and <br>long text samples<br> <li> Implement <code>TestMilvusClientHighlighterValid</code> class with 15+ test methods <br>covering basic usage, multiple tags, fragment parameters, offsets, <br>numbers, sentences, and language support<br> <li> Implement <code>TestMilvusClientHighlighterInvalid</code> class with 8+ test <br>methods validating error handling for invalid parameters and <br>configurations<br> <li> Test highlighter with BM25 search, text matching, and various analyzer <br>configurations</ul> </details> </td> <td><a href="https://github.com/milvus-io/milvus/pull/46505/files#diff-443e3fefb65fbdb088d5920083306ecfe3605745b1e2714198c6566ca67b3736">+1163/-0</a></td> </tr> </table></td></tr></tbody></table> </details> ___ <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added a comprehensive highlighter test suite covering: - Core highlighting with single and multi-analyzer setups and multi-tag variations - Fragment parameter behaviors and edge cases (size, offset, count) - Text-match and query-based highlighting, including BM25 and vector interactions - Sub-word, long-text/tag, case sensitivity, Chinese/multi-language scenarios - Error handling for invalid parameters, no-match cases, and other edge conditions - Module-scoped fixture preparing multilingual, long-form test data and teardown <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub> <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Eric Hou <eric.hou@zilliz.com> Co-authored-by: Eric Hou <eric.hou@zilliz.com>
Tests
E2E Test
Configuration Requirements
Operating System
| Operating System | Version |
|---|---|
| Amazon Linux | 2023 or above |
| Ubuntu | 20.04 or above |
| Mac | 10.14 or above |
Hardware
| Hardware Type | Recommended Configuration |
|---|---|
| CPU | x86_64 architecture Intel CPU Sandy Bridge or above CPU Instruction Set - SSE4_2 - AVX - AVX2 - AVX512 or arm64 Linux/MacOS |
| Memory | 16 GB or more |
Software
| Software Name | Version |
|---|---|
| Docker | 19.05 or above |
| Docker Compose | 1.25.5 or above |
| jq | 1.3 or above |
| kubectl | 1.14 or above |
| helm | 3.0 or above |
| kind | 0.10.0 or above |
Installing Dependencies
Troubleshooting Docker and Docker Compose
- Confirm that Docker Daemon is running：
$ docker info
-
Ensure that Docker is installed. Refer to the official installation instructions for Docker CE/EE.
-
Start the Docker Daemon if it is not already started.
-
To run Docker without
rootprivileges, create a user group labeleddocker, then add a user to the group withsudo usermod -aG docker $USER. Log out and log back into the terminal for the changes to take effect. For more information, see the official Docker documentation for Managing Docker as a Non-Root User.
- Check the version of Docker-Compose
$ docker compose version
docker compose version 1.25.5, build 8a1c60f6
docker-py version: 4.1.0
CPython version: 3.7.5
OpenSSL version: OpenSSL 1.1.1f 31 Mar 2020
- To install Docker-Compose, see Install Docker Compose
Install jq
Install kubectl
Install helm
- Refer to https://helm.sh/docs/intro/install/
Install kind
Run E2E Tests
$ cd tests/scripts
$ ./e2e-k8s.sh
Getting help
You can get help with the following command:
$ ./e2e-k8s.sh --help