Related to #39173#39718
In storage v2, the `lack_bin_rows` cannot be used since field id is not
column group id, which will not be matched forever.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #42176
Remove the mutual exclusion constraints between channel and segment
balance tasks to allow them to run concurrently.
Changes include:
- Remove permitBalanceChannel() and permitBalanceSegment() methods from
RoundRobinBalancer
- Update ChannelLevelScoreBalancer, MultiTargetBalancer,
RowCountBasedBalancer, and ScoreBasedBalancer to remove constraint
checks
- Allow segment balance tasks to proceed even when channel balance tasks
are running
- Update test cases to reflect new behavior where balance tasks no
longer block each other
This change improves the efficiency of load balancing by removing
unnecessary coordination overhead between different types of balance
operations.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Related to #39173
This PR
- Handle storage v2 log path in local storage mode on querynode
- Ignore field info check when append index for loaded sealed segment
when using storage v2
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #41976
- make drop partition message as a broadcast message.
- add gc when drop partition message is acked.
- add a call back to handle the broadcast message when ack.
- the ack operation of broadcast message will retry until success.
Signed-off-by: chyezh <chyezh@outlook.com>
issue: #40532
- start timeticksync at rootcoord if the streaming service is not
available
- stop timeticksync if the streaming service is available
- open a read-only wal if some nodes in cluster is not upgrading to 2.6
- allow to open read-write wal after all nodes in cluster is upgrading
to 2.6
---------
Signed-off-by: chyezh <chyezh@outlook.com>
issue: #42162
- enhance: add read ahead buffer size issue #42129
- fix: rocksmq consumer's close operation may get stucked
- fix: growing segment from old arch is not flushed after upgrading
---------
Signed-off-by: chyezh <chyezh@outlook.com>
issue: #42147
- error of sync task should be returned if error is returned to avoid
checkpoint is push forward.
- fix up node id checker of UpdateChannelCheckpoint in streaming.
Signed-off-by: chyezh <chyezh@outlook.com>
This PR fixes a minor typo in a log message in the `PreExecute` method
of `internal/datanode/index/task_index.go`.
Corrected "dimesion" to "dimension".
Signed-off-by: hckex <33862757+hckex@users.noreply.github.com>
Related to #42084
Embedding node cached schema when created, causing schema mismatch after
schema change. This PR make embeddingNode use schema from metacache,
which will be updated.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: https://github.com/milvus-io/milvus/issues/41690
This commit implements partial search result functionality when query
nodes go down, improving system availability during node failures. The
changes include:
- Enhanced load balancing in proxy (lb_policy.go) to handle node
failures with retry support
- Added partial search result capability in querynode delegator and
distribution logic
- Implemented tests for various partial result scenarios when nodes go
down
- Added metrics to track partial search results in querynode_metrics.go
- Updated parameter configuration to support partial result required
data ratio
- Replaced old partial_search_test.go with more comprehensive
partial_result_on_node_down_test.go
- Updated proto definitions and improved retry logic
These changes improve query resilience by returning partial results to
users when some query nodes are unavailable, ensuring that queries don't
completely fail when a portion of data remains accessible.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Return `false` in the `Process()` function for `executing` or
`pipelining` state `l0Compaction`. This prevents the `l0Compaction` task
from being removed from the `CompactionInspector`'s executing queue,
thereby avoiding concurrent execution of `l0Compaction` and `Stats`.
issue: https://github.com/milvus-io/milvus/issues/42008
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Related to #40756
Large nq will naturally increase query time, which causing lots of slow
log when user NQ numbers are very large.
This PR make slow search counts span per nq (using avg val) to decide
whether one request is slow or not.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Related to #39718
The absent nullable field shall be filled at server-side in nullable
design. While the implementation here was buggy causing the feature was
not able to serve.
This PR make proxy fill the field data in correct format so that field
data with absent column(s) will be accepted.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
See #42091
In this PR,
- Added a new `-t` option to specify the build type (Debug/Release).
- Updated the Makefile to pass the build type to the 3rdparty build
script.
- Included usage instructions for the script to improve user guidance.
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
This PR fixes a minor typo in the README file of the datanode module.
Corrected "imformation" to "information".
Signed-off-by: hckex <33862757+hckex@users.noreply.github.com>
issue: #42073
Enhanced the update-milvus-api Makefile command to update milvus-proto
version across all 4 go.mod files in the project:
- Update main go.mod (root module)
- Update client/go.mod (client module)
- Update pkg/go.mod (pkg module)
- Update tests/go_client/go.mod (test client module)
Improvements:
- Add intelligent version detection (git tag vs commit ID)
- Add comprehensive error handling and validation
- Add detailed progress feedback and success reporting
- Add proper directory existence checks
- Add UPDATE_MILVUS_API.md documentation with usage examples
This ensures all modules use the same milvus-proto version, improving
project consistency and maintainability.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Optimize session management by reducing the TTL (Time To Live) value for
service registration from 30 seconds to 10 seconds. This change improves
the system's ability to detect service failures more quickly and
enhances overall cluster responsiveness.
Changes include:
- Update default session TTL from 30s to 10s in milvus.yaml
- Adjust DefaultSessionTTL constant from 30 to 10 seconds
- Update SessionTTL default value from 60 to 10 seconds
- Maintain consistent TTL values across configuration files
This optimization reduces the time required for the system to detect
when services become unavailable, leading to faster failover and
improved cluster stability during node failures or network issues.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
