related issue: #40698
1. use vector datat types instead of hard code datatpe names
2. update search pagination tests
3. remove checking distances in search results checking, for knowhere
customize the distances for different metrics and indexes. Now only
assert the distances are sorted correct.
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
Related to #41460
This PR looses insert data check based on schema. These check shall
actually happen at milvus server side.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
json stats add map null check before insert into tantivity. Json stats
index may fail if there is no data
issue:https://github.com/milvus-io/milvus/issues/41494
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
This PR fixes#41384 .
When using milvus client and compile on windows, the compilation failed
with the undefined RSS error.
On windows, the way to get memory used is the same as on darwin.
Signed-off-by: Julien Salleyron <julien.salleyron@gmail.com>
Related to #39718
This PR:
- Use WAL broadcast timestamp as Collection update timestamp
- Remove request_fields size assertion
- Remove proxy schema cache loaded field check & skip related cases
- other minor issues
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Optimized JSON filter execution by introducing
ProcessJsonStatsChunkPos() for unified position calculation and
GetNextBatchSize() for better batch processing.
Improved JSON key generation by replacing manual path joining with
milvus::Json::pointer() and adjusted slot size calculation for JSON key
index jobs.
Updated the task slot calculation logic in calculateStatsTaskSlot() to
handle the increased resource needs of JSON key index jobs.
issue: https://github.com/milvus-io/milvus/issues/41378https://github.com/milvus-io/milvus/issues/41218
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
Issue: #41261
`milvusclient.NewHybridSearchOption` receives a variadic `annRequests`
parameter. However, since `milvusclient.annRequest` is private, there is
no way to declare a slice, therefore there is no way to make it fully
generic (as in, create a slice of `milvusclient.annRequest`s and pass
them to `NewHybridSearchOption`. This PR renames
`milvusclient.annRequest` to `milvusclient.AnnRequest` to export it.
This is an API change since it's renaming a struct. However, since the
struct was previously private no external code depends on it, unless
it's doing nasty things with reflection (in which case it should not
depend on the name).
Signed-off-by: Mario Camou <mcamou@users.noreply.github.com>
Signed-off-by: mcamou <mcamou@users.noreply.github.com>
issue: #40243
This helps identify the running OS version during the build process,
ensuring better logging.
Signed-off-by: Shubhendra Kushwaha <shubhendrakushwaha94@gmail.com>
Related to #39718
Use schema update ts when it's greater than calculated guarantee
timestamp to make sure that all read request using updated schema shall
wait all schema change event processed.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
feat: Add support for modifying max capacity of array fields
This commit adds support for modifying the max capacity of array fields
in the `alterCollectionFieldTask` function. It checks if the field is an
array type and then validates and updates the max capacity value. This
change improves the flexibility of array fields in the collection.
Issue: https://github.com/milvus-io/milvus/issues/41363
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
issue: #40942
Add simde package, which can make porting SIMD code to other
architectures much easier.
Signed-off-by: Shawn Wang <shawn.wang@zilliz.com>
fix: #39755
The following shows a simple benchmark where insert 1M docs where all
rows are "hello", the latency is segcore level, CPU is 9900K:
master: 2.62ms
this PR: 2.11ms
bench mark code:
```
TEST(TextMatch, TestPerf) {
auto schema = GenTestSchema({}, true);
auto seg = CreateSealedSegment(schema, empty_index_meta);
int64_t N = 1000000;
uint64_t seed = 19190504;
auto raw_data = DataGen(schema, N, seed);
auto str_col = raw_data.raw_->mutable_fields_data()
->at(1)
.mutable_scalars()
->mutable_string_data()
->mutable_data();
for (int64_t i = 0; i < N - 1; i++) {
str_col->at(i) = "hello";
}
SealedLoadFieldData(raw_data, *seg);
seg->CreateTextIndex(FieldId(101));
auto now = std::chrono::high_resolution_clock::now();
auto expr = GetMatchExpr(schema, "hello", OpType::TextMatch);
auto final = ExecuteQueryExpr(expr, seg.get(), N, MAX_TIMESTAMP);
auto end = std::chrono::high_resolution_clock::now();
auto duration =
std::chrono::duration_cast<std::chrono::microseconds>(end - now);
std::cout << "TextMatch query time: " << duration.count() << "ms"
<< std::endl;
}
```
---------
Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
feat: Ignore reporting index metrics for non-existent indexes
Remove the reporting of index metrics for non-existent indexes in the
`getCollectionMetrics` function. This change improves the code by
skipping unnecessary operations and reduces log noise.
issue: https://github.com/milvus-io/milvus/issues/41280
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
