patch search cache param from index configs when index meta could not
get the search cache size key
issue: #30113
pr: #30119
Signed-off-by: xianliang <xianliang.li@zilliz.com>
cherry pick from master
pr: #29810#30055
`SampleFraction` shall be float and all `C.CString` shall be freed
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
When the TimeTravel functionality was previously removed, it
inadvertently affected the MVCC functionality within the system. This PR
aims to reintroduce the internal MVCC functionality as follows:
1. Add MvccTimestamp to the requests of Search/Query and the results of
Search internally.
2. When the delegator receives a Query/Search request and there is no
MVCC timestamp set in the request, set the delegator's current tsafe as
the MVCC timestamp of the request. If the request already has an MVCC
timestamp, do not modify it.
3. When the Proxy handles Search and triggers the second phase ReQuery,
divide the ReQuery into different shards and pass the MVCC timestamp to
the corresponding Query requests.
issue: #29656
pr: #29749
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
issue: #29494
pr: #29496
1. link with install path's libblob-chunk-manager
2. performance of `ShouldBindWith` is better than `ShouldBindBodyWith`
3. the middleware shouldn't read the unrefreshed parameter repeatly
4. diff from old restful, named HandlersV1
Signed-off-by: PowderLi <min.li@zilliz.com>
issue: #29672
master pr: #29703
the storage account need privileges of actions
`Microsoft.Storage/storageAccounts/blobServices/containers/blobs/*` at
least
Signed-off-by: PowderLi <min.li@zilliz.com>
Cherry-pick from master
pr: #28952
See also #28795
Orignal `C.NewSegment` may panic if some condition is not met, this pr
changes response struct to `CNewSegmentResult`, which contains
`C.CStatus` and may return catched exception
---------
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
add all loading data into a buffer and then copy them into the a
fit-in-size memory
pr: https://github.com/milvus-io/milvus/pull/29387
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
issue: [milvus-proto
#212](https://github.com/milvus-io/milvus-proto/issues/212)
master pr: #28961
milvus can't use partition related privileges until upgrade
milvus-proto, even if them were added to milvus-proto
Signed-off-by: PowderLi <min.li@zilliz.com>
issue: #28781#28329
master pr: #28782
1. There is no need to call `DescribeCollection`, if the collection's
schema is found in the globalMetaCache
2. did `GetProperties` to check the access to Azure Blob Service
Signed-off-by: PowderLi <min.li@zilliz.com>
issue: #28683 master pr: #28505
issue: #28686 master pr: #28703
1. update the base image: milvusdb/milvus-env (#28505) to avoid
downloading installation packages in CI workload install vcpkg and
install some packages in advance
2. use the latest image
2. update azure-identity-cpp from beta to release
Signed-off-by: PowderLi <min.li@zilliz.com>
for now the assert method in segcore could accept a string information,
too many codes don't print the value they assert.
make it happy
related #28811
pr: #28812
Signed-off-by: yah01 <yah2er0ne@outlook.com>
Cherry pick from master
pr: #28802
Now segcore load system field info as well, the growing segment
assertion shall not pass with "+ 2" value
This will cause all growing segments load failure
Fix#28801
Related to #28478
See also #28524
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
before this, Milvus use container/system's memory info to get the memory
usage, which could be inaccurate.
we allocates the memory by private anon mmap,
then rss - shared would be the accurate memory usage
resolve https://github.com/milvus-io/milvus/issues/28553
pr: #28554
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
We first fill the data into a vector and then copy it into the proto,
for some types (exclude variable-length types and int8, int16),
data could be directly copied into the proto.
Sealed segment has been optimized in
https://github.com/milvus-io/milvus/pull/28106.
pr: #28323
Signed-off-by: yah01 <yah2er0ne@outlook.com>
/kind improvement
Before this, while retrieving data (query/search), we first copy the
data into a fixed vector, and then copy data from this into the proto
field.
Now we can directly copy the data into the proto field.
This optimization can't be done with int8, int16 due to the proto
doesn't provide the two types, we store them in int32s
Also, this can't be done with variable length field like string, JSON,
see https://github.com/protocolbuffers/protobuf/issues/10866. I tried
but it seems proto doesn't guarantee the memory layout as we expected,
it crashed
pr: #28106
Signed-off-by: yah01 <yah2er0ne@outlook.com>
The proto generated files is out of sync for image build env
This will cause --dirty="-dev" tag in Milvus build version
Sync changed files to avoid this case
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
