Cherry-pick from master
pr: #29315
See also #29113
- Unify partition info refresh logic
- Prevent parse partition names for each partition key search request
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
cherry-pick from master
pr: #29343
See also #29332
The segment may be released before or during the request when delegator
tries to forward delete request to yet. Currently, these two situation
returns different error code.
In this particular case, ErrSegmentNotLoaded and ErrSegmentNotFound
shall both be ignored preventing return search service unavailable by
mistake.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Cherry pick from master
pr: #29328
See also #29327
Change channel checkpoint metrics to unix seconds instead of checkpoint
timestamp lag value
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
related to #28549
pr: #28626
1. avoid duplicated sync segments under syncing states
2. add jitter to avoid sync segments at the same time
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
many growing segments may be created in a short time and there is no
restriction to the process, the CGO call will leave many threads
related: https://github.com/milvus-io/milvus/issues/29282
pr: #29306
Signed-off-by: yah01 <yah2er0ne@outlook.com>
issue: #23726
pr: #29231
This PR add control config to querycoord's background auto balance
channel operation
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: #29243
only rootCoord read the configuration item `builtinRoles`, so proxy
never know whether the role to be deleted is builtin.
Signed-off-by: PowderLi <min.li@zilliz.com>
issue: #29004
master pr: #29055
add a new parameter: `params`, which is a map[string]float64;
but now only 2 valid item: radius + range_filter;
Signed-off-by: PowderLi <min.li@zilliz.com>
See also #29156
FlushTs need to to be reset to MaxUint64 after channel checkpoint is
after this timestamp. Otherwise, the segment will be shattered and flush
queue will be filled with tasks
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #28622
pr: #29216
After we support balance segment with growing segment count #28623, if
we balance segment and channel at same time, some segments need to be
rebalanced after balance channel finish.
This PR skip balance segment when channel need be balanced.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
See also #29223
cherry pick part of master commit
pr: #29224
Make `conc.Pool` resizable by adding Resize method for it.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #23726
pr: #28469
1. enable auto balance channel between nodes in querycoord
2. make `genSegmentPlan` reuse the `AssignSegment` logic
3. make `genChannelPlan` reuse the `AssignChannel` logic
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Cherry-pick from master
pr: #29195
See also #29113
rand.Seed is deprecated and cost noticable CPU time during heavy payload
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
we found the load got stuck probably, and reviewed the logs.
the target observer seems not working, the reason is the taskDispatcher
removes the task in a goroutine, and modifies the task status after
committing the task into the goroutine pool, but this may happen after
the task removed, which leads to the task will never be removed
related #29086
pr: #29191
Signed-off-by: yah01 <yang.cen@zilliz.com>
issue: #28622
pr: #28623
query node with delegator will has more rows than other query node due
to delgator loads all growing rows.
This PR enable the balance segment which based on the num of growing
rows in leader view.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Cherry pick from master
pr: #29154
See also #29177
Add a config item for partition name as regexp feature and disable it by
default
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Cherry pick from master
pr: #29143
See also #29113
Using zap.Stringer log field will evaluate log field value only when log
level meets the configuration, which could save some CPU time in search
route
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
This pull request enhances the logging functionality in the code related
to target updating. It adds more logs about the condition satisfying
when updating the target. The logs provide additional information about
the collection ID, replica number, channel readiness, segment readiness,
and leader view readiness. These logs will help in troubleshooting and
monitoring the target updating process.
pr: #29090
Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
