issue: https://github.com/milvus-io/milvus/issues/41690
This commit implements partial search result functionality when query
nodes go down, improving system availability during node failures. The
changes include:
- Enhanced load balancing in proxy (lb_policy.go) to handle node
failures with retry support
- Added partial search result capability in querynode delegator and
distribution logic
- Implemented tests for various partial result scenarios when nodes go
down
- Added metrics to track partial search results in querynode_metrics.go
- Updated parameter configuration to support partial result required
data ratio
- Replaced old partial_search_test.go with more comprehensive
partial_result_on_node_down_test.go
- Updated proto definitions and improved retry logic
These changes improve query resilience by returning partial results to
users when some query nodes are unavailable, ensuring that queries don't
completely fail when a portion of data remains accessible.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Return `false` in the `Process()` function for `executing` or
`pipelining` state `l0Compaction`. This prevents the `l0Compaction` task
from being removed from the `CompactionInspector`'s executing queue,
thereby avoiding concurrent execution of `l0Compaction` and `Stats`.
issue: https://github.com/milvus-io/milvus/issues/42008
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Related to #40756
Large nq will naturally increase query time, which causing lots of slow
log when user NQ numbers are very large.
This PR make slow search counts span per nq (using avg val) to decide
whether one request is slow or not.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Related to #39718
The absent nullable field shall be filled at server-side in nullable
design. While the implementation here was buggy causing the feature was
not able to serve.
This PR make proxy fill the field data in correct format so that field
data with absent column(s) will be accepted.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
See #42091
In this PR,
- Added a new `-t` option to specify the build type (Debug/Release).
- Updated the Makefile to pass the build type to the 3rdparty build
script.
- Included usage instructions for the script to improve user guidance.
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
This PR fixes a minor typo in the README file of the datanode module.
Corrected "imformation" to "information".
Signed-off-by: hckex <33862757+hckex@users.noreply.github.com>
issue: #42073
Enhanced the update-milvus-api Makefile command to update milvus-proto
version across all 4 go.mod files in the project:
- Update main go.mod (root module)
- Update client/go.mod (client module)
- Update pkg/go.mod (pkg module)
- Update tests/go_client/go.mod (test client module)
Improvements:
- Add intelligent version detection (git tag vs commit ID)
- Add comprehensive error handling and validation
- Add detailed progress feedback and success reporting
- Add proper directory existence checks
- Add UPDATE_MILVUS_API.md documentation with usage examples
This ensures all modules use the same milvus-proto version, improving
project consistency and maintainability.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Optimize session management by reducing the TTL (Time To Live) value for
service registration from 30 seconds to 10 seconds. This change improves
the system's ability to detect service failures more quickly and
enhances overall cluster responsiveness.
Changes include:
- Update default session TTL from 30s to 10s in milvus.yaml
- Adjust DefaultSessionTTL constant from 30 to 10 seconds
- Update SessionTTL default value from 60 to 10 seconds
- Maintain consistent TTL values across configuration files
This optimization reduces the time required for the system to detect
when services become unavailable, leading to faster failover and
improved cluster stability during node failures or network issues.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Rename `imeta` to `importMeta` to improve readability, and enhance
import related context usage.
issue: https://github.com/milvus-io/milvus/issues/41123
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
#41846#41894
Resolve SN OOM issue during small file loading in Woodpecker;
Correct WP fence/close execution order;
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
issue: https://github.com/milvus-io/milvus/issues/41690
- Merge leader view and channel management into ChannelDistManager,
allowing a channel to have multiple delegators.
- Improve shard leader switching to ensure a single replica only has one
shard leader per channel. The shard leader handles all resource loading
and query requests.
- Refine the serviceable mechanism: after QC completes loading, sync the
query view to the delegator. The delegator then determines its
serviceable status based on the query view.
- When a delegator encounters forwarding query or deletion failures,
mark the corresponding segment as offline and transition it to an
unserviceable state.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: #41621
- Deprecate EnableWorkerSQCostMetrics parameter
- Always collect cost metrics from all search and retrieve results
- Update code with comments explaining the changes rationale
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: #41874
- Optimize balance_checker to support balancing multiple collections
simultaneously
- Add new parameters for segment and channel balancing batch sizes
- Add enableBalanceOnMultipleCollections parameter
- Update tests for balance checker
This change improves resource utilization by allowing the system to
balance multiple collections in a single trigger with configurable batch
sizes.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
