Support get sdk type by user agent when we can't get sdk version by
connection in access log.
---------
pr: https://github.com/milvus-io/milvus/pull/30760
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
Cherry-pick from master
pr: #31363
See also #31362
This PR make datacoord garbage collection scan operation using differet
interval than other opeartion.
This interval is a newly added param item, which default value is 7*24
hours.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
fix#32059
pr: #32089
this pr fix two issues:
1. offset is not handled correctly without specify a limit
2. reduceStopForBest doesn't guarantee to return limit result even if
there are more result when there is small segment
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
issue: #29261
pr: #29998
This PR Add restful api for devops to execute rolling upgrade, including
suspend/resume balance and manual transfer segments/channels.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
/kind improvement
pr: #31271
fix: https://github.com/milvus-io/milvus/issues/31272
This pr add more metrics, which are:
Slow query count, which the duration considered as slow can be
configurable;
Number of deleted entities;
Number of entities per collection;
Number of loaded entities per collection;
Number of indexed entities;
Number of indexed entities, per collection, per index and whether it's a
vetor index;
Quota states (LongTimeTickDelay, MemoryExhuasted, DiskQuotaExhuasted)
per database;
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
issue: #31222
pr: #31256
grpcclient's `call` func return a unrecoverable error, then the caller's
retry policy also breaks due to this unrecoverable error.
This PR introduce `retry.Handle`, the new func use `func() (bool,
error)` as input parameters, which return `shouldRetry` directly, to
avoid grpcclient return a unrecoverable error
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Cherry-pick from master
pr: #31388
See also #30806
`formatKey` may cost lots of CPU on string processing under high QPS
scenario, this PR adds a formattedKeys cache preventing string operation
in each param get value.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
This PR includes the following adjustments:
1. To prevent channelCP update task backlog, only one task with the same
vchannel is retained in the updater. Additionally, the lastUpdateTime is
refreshed after the flowgraph submits the update task, rather than in
the callBack function.
2. Batch updates of multiple vchannel checkpoints are performed in the
UpdateChannelCheckpoint RPC (default batch size is 128). Additionally,
the lock for channelCPs in DataCoord meta has been switched from key
lock to global lock.
3. The concurrency of UpdateChannelCheckpoint RPCs in the datanode has
been reduced from 1000 to 10.
issue: https://github.com/milvus-io/milvus/issues/30004
pr: https://github.com/milvus-io/milvus/pull/30941
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Cherry pick from master
pr: #30757
See also #30756
This PR:
- Request disk resource when index type, version loaded with disk
- Add attribute cache for index utility
- Add `typeutil.Pair`
---------
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
flush rate control at collection level to avoid generate too much
segment.
0.1 qps by default.
issue: #29477
pr: #29567
Signed-off-by: chyezh <ye.zhen@zilliz.com>
Cherry-pick from master
pr: #30833
See also #30832
This PR removes time tick delay metrics when rootcoord GetMetrics
response does not have previously existed querynode/datanode
Also add unit tests for this case
---------
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi.Xia <congqi.xia@zilliz.com>
1. add coordinator and proxy graceful stop timeout to 5s.
3. add other work node graceful stop timeout to 900s, and we should
potentially change this to 600s when graceful stop is smooth
4. change the order of datacoord component while stop.
5. `LivenessCheck` do not perform graceful shutdown now.
issue: https://github.com/milvus-io/milvus/issues/30310
pr: #30317
also see: https://github.com/milvus-io/milvus/pull/30306
---------
Signed-off-by: chyezh <chyezh@outlook.com>
Allows proactive warming up of chunk cache. Original vector data will be
asynchronously loaded into the chunk cache during the load process. It
has the potential to significantly reduce query/search latency for a
certain duration after the load, albeit with a concurrent increase in
disk usage.
issue: https://github.com/milvus-io/milvus/issues/30181
pr: https://github.com/milvus-io/milvus/pull/30182
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
patch search cache param from index configs when index meta could not
get the search cache size key
issue: #30113
pr: #30119
Signed-off-by: xianliang <xianliang.li@zilliz.com>
