mirror of
https://gitee.com/milvus-io/milvus.git
synced 2025-12-28 22:45:26 +08:00
ca7ec23198
When delete by partition_key, Milvus will generates L0 segments globally. During L0 Compaction, those L0 segments will touch all partitions collection wise. Due to the false-positive rate of segment bloomfilters, L0 compactions will append false deltalogs to completed irrelevant partitions, which causes *partition deletion amplification. This PR uses partition_key to set targeted partitionID when producing deleteMsgs into MsgStreams. This'll narrow down L0 segments scope to partition level, and remove the false-positive influence collection-wise. However, due to DeleteMsg structure, we can only label one partition to one deleteMsg, so this enhancement fails if user wants to delete over 2 partition_keys in one deletion. See also: #34665 Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Integration test
This folder contains the integration test for Milvus components.
How to run integration test locally
Integration test still need some thirdparty components to start:
cd [milvus-folder]/deployments/docker/dev && docker compose up -d
Run following script to start the full integration test:
cd [milvus-folder]
make milvus # milvus needs to be compiled to make cpp build ready
./scripts/run_intergration_test.sh
If you want to run single test case, you could execute command like this example
# mq, etcd, minio ready before
cd [milvus-folder]
source scripts/setenv.sh
cd tests/integration/[testcase-folder]/
go test -run "$testCaseName^" -testify.m "$subTestifyCaseName^" -race -v
Recommended coding style for add new cases
Using suite
MiniClusterandMiniClusterSuite` provides lots of comment preset tool function to execute intergration test.
It is recommend to add a new test with testify/suite
import (
// ...
"github.com/milvus-io/milvus/tests/integration"
)
type NewSuite struct {
integration.MiniClusterSuite
}
// Setups and teardowns, optional if no custom logic needed
// example to suite setup & teardown, same logic applies to test setup&teardown
func (s *NewSuite) SetupSuite() {
s.MiniClusterSuite.SetupSuite()
// customized setup
}
func (s *NewSuite) TearDownSuite() {
s.MiniClusterSuite.TearDownSuite()
// customized teardown
}
New folder for each new scenario
It's a known issue that integration test cases run in same process might affect due to some singleton component not fully cleaned.
As a temp solution, test cases are separated into different packages to run independently.