Spade A 6077178553
enhance: enable STL_SORT to support VARCHAR (#44401)
issue: https://github.com/milvus-io/milvus/issues/44399

This PR implements STL_SORT for VARCHAR data type for both RAM and MMAP
mode.
The general idea is that we deduplicate field values and maintains a
posting list for each unique value.

The serialization format of the index is:
```
[unique_count][string_offsets][string_data][post_list_offsets][post_list_data][magic_code]
string_offsets: array of offsets into string_data section
string_data: str_len1, str1, str_len2, str2, ...
post_list_offsets: array of offsets into post_list_data section
post_list_data: post_list_len1, row_id1, row_id2, ..., post_list_len2, row_id1, row_id2, ...
```

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-10-23 11:00:05 +08:00
..
2025-10-20 10:44:03 +08:00
2021-11-16 15:41:11 +08:00

Tests

E2E Test

Configuration Requirements

Operating System
Operating System Version
Amazon Linux 2023 or above
Ubuntu 20.04 or above
Mac 10.14 or above
Hardware
Hardware Type Recommended Configuration
CPU x86_64 architecture
Intel CPU Sandy Bridge or above
CPU Instruction Set
- SSE4_2
- AVX
- AVX2
- AVX512 or arm64 Linux/MacOS
Memory 16 GB or more
Software
Software Name Version
Docker 19.05 or above
Docker Compose 1.25.5 or above
jq 1.3 or above
kubectl 1.14 or above
helm 3.0 or above
kind 0.10.0 or above

Installing Dependencies

Troubleshooting Docker and Docker Compose
  1. Confirm that Docker Daemon is running
$ docker info
  • Ensure that Docker is installed. Refer to the official installation instructions for Docker CE/EE.

  • Start the Docker Daemon if it is not already started.

  • To run Docker without root privileges, create a user group labeled docker, then add a user to the group with sudo usermod -aG docker $USER. Log out and log back into the terminal for the changes to take effect. For more information, see the official Docker documentation for Managing Docker as a Non-Root User.

  1. Check the version of Docker-Compose
$ docker compose version

docker compose version 1.25.5, build 8a1c60f6
docker-py version: 4.1.0
CPython version: 3.7.5
OpenSSL version: OpenSSL 1.1.1f  31 Mar 2020
Install jq
Install kubectl
Install helm
Install kind

Run E2E Tests

$ cd tests/scripts
$ ./e2e-k8s.sh

Getting help

You can get help with the following command:

$ ./e2e-k8s.sh --help