Cherry pick from master
pr: #38375
Related to #38372
This PR make drop partition only check target partition load states only
in case of concurrent releasing other partition in same collection.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
sparse vectors may have arbitrary number of non zeros and it is hard to
optimize without knowing the actual distribution of nnz. this PR adds a
metric for analyzing that.
pr: #38329
also fixed a bug of sparse when searching by pk
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
enhance :
alterindex delete properties
We have introduced a new parameter deleteKeys to the alterindex
functionality, which allows for the deletion of properties within an
index. This enhancement provides users with the flexibility to manage
index properties more effectively by removing specific keys as needed.
altercollection delete properties
We have introduced a new parameter deleteKeys to the altercollection
functionality, which allows for the deletion of properties within an
collection. This enhancement provides users with the flexibility to
manage collection properties more effectively by removing specific keys
as needed.
3.support altercollectionfield
We currently support modifying the fieldparams of a field in a
collection using altercollectionfield, which only allows changes to the
max-length attribute.
Key Points:
New Parameter - deleteKeys: This new parameter enables the deletion of
specified properties from an index. By passing a list of keys to
deleteKeys, users can remove the corresponding properties from the
index.
Mutual Exclusivity: The deleteKeys parameter cannot be used in
conjunction with the extraParams parameter. Users must choose one
parameter to pass based on their requirement. If deleteKeys is provided,
it indicates an intent to delete properties; if extraParams is provided,
it signifies the addition or update of properties.
issue: https://github.com/milvus-io/milvus/issues/37436
pr: https://github.com/milvus-io/milvus/pull/37437
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
issue: #38086
pr: #38096
cause ManualCompact api pass collection id in request, but RBAC requires
to check collection name, so grant ManualCompact api doesn't work.
This PR refine the ManualCompact api to accpet collection name in
request.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: #37115
pr: #37371#37646#37729
the old implementation update shard cache and shard client manager at
same time, which causes lots of conor case due to concurrent issue
without lock.
This PR decouple shard client manager from shard cache, so only shard
cache will be updated if delegator changes. and make sure shard client
manager will always return the right client, and create a new client if
not exist. in case of client leak, shard client manager will purge
client in async for every 10 minutes.
---------
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: congqixia <congqi.xia@zilliz.com>
issue: #37115
pr#37116 let proxy retry to get shard leader if error happens, which
cause if search/query on a unloaded collection, which will keep retrying
until ctx done.
This PR add error type check to skip retry on ErrCollectionLoaded.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: #37115
pr: #37116
casue init query node client is too heavy, so we remove
updateShardClient from leader mutex, which cause much more concurrent
cornor cases.
This PR delay query node client's init operation until `getClient` is
called, then use leader mutex to protect updating shard client progress
to avoid concurrent issues.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Cherry pick from master
pr: #36613
Previous label case broken by #36107, this PR make all inbound label
using label constants from metrics package.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #36490
pr: #36491
After the query node changes from a delegator to a worker, proxy should
skip this querynode's health check.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: #35859
pr: #36181
This PR introduce two new param: toleranceFactor and checkRequestNum,
after every checkRequestNum request has been assigned, try to compute
querynode's workload score.
if the diff is less than the toleranceFactor, replica selection policy
will fallback to round_robin, which reduce the average cost to about
500ns.
if the diff is larger than the toleranceFactor, replica selection policy
will compute querynode's score to select the target node with smallest
score in every assigment.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Cherry-pick from master
pr: #36170
Related to #35996
For `Field Partial Load` feature, Milvus shall ignore index check for
non-loaded vector field.
Also, this PR unifies the logic of index check for load collection and
load partitions tasks.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Cherry-pick from master
pr: #36107
Related to #36102
This PR use newly added `grpcSizeStatsHandler` to reduce calling
`proto.Size` since the request & response size info is recorded by grpc
framework.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
