Optimized JSON filter execution by introducing
ProcessJsonStatsChunkPos() for unified position calculation and
GetNextBatchSize() for better batch processing.
Improved JSON key generation by replacing manual path joining with
milvus::Json::pointer() and adjusted slot size calculation for JSON key
index jobs.
Updated the task slot calculation logic in calculateStatsTaskSlot() to
handle the increased resource needs of JSON key index jobs.
issue: https://github.com/milvus-io/milvus/issues/41378https://github.com/milvus-io/milvus/issues/41218
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
feat: Ignore reporting index metrics for non-existent indexes
Remove the reporting of index metrics for non-existent indexes in the
`getCollectionMetrics` function. This change improves the code by
skipping unnecessary operations and reduces log noise.
issue: https://github.com/milvus-io/milvus/issues/41280
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
## What this PR does
This PR fixes an issue where the `PreCheck` function in DataCoord logs
unnecessary warnings
when attempting to retrieve 'dim' from non-vector fields.
The change adds a check to only call `GetDimFromParams` when the field
type is a vector type.
## Related issue
Fixes#41287
---------
Signed-off-by: 박상호 <sangho@rapportlabs.kr>
Signed-off-by: Sangho Park <hoyaspark@gmail.com>
Make DataNode use compaction parameters from request instead of
configuration.
issue: https://github.com/milvus-io/milvus/issues/41123
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Merge RootCoord, DataCoord And QueryCoord into MixCoord
Make Session into one
issue : https://github.com/milvus-io/milvus/issues/37764
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
after the pr merged, we can support to insert, upsert, build index,
query, search in the added field.
can only do the above operates in added field after add field request
complete, which is a sync operate.
compact will be supported in the next pr.
#39718
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
With concurrenct L0 compaction
(https://github.com/milvus-io/milvus/pull/36816), delta logs might be
written to the same L1 segment, causing logID duplication when using the
incremental beginLogID. This PR removes the beginLogID mechanism and
instead passes a log ID range, where the number of IDs in the range
equals the number of compaction segment binlogs multiplied by an
expansion factor.
issue: https://github.com/milvus-io/milvus/issues/40207
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Related to #40866
This PR:
- update go-api/v2 and support partition id/channel/segment level manual
compaction
- refines the compaction trigger implementation
- unify the compaction signal usage
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #40292
related to #39552
- Fix incorrect delete checkpoint usage in SyncDistribution
- Change checkpoint parameter from action.GetCheckpoint() to
action.GetDeleteCP() in SyncTargetVersion call
- This resolves the issue where delete buffer data was being cleaned
prematurely due to wrong checkpoint reference
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Related to #40388
The small segments may be put into bucket twice due to value parameter
of Knapsnap.packWith
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Introduce a batch subscription mechanism in msgdispatcher: the
msgdispatcher now includes a vchannel watch task queue, where all
vchannels in the queue will subscribe to the MQ only once and pull
messages from the oldest vchannel checkpoint to the latest.
issue: https://github.com/milvus-io/milvus/issues/39862
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
issue: #38399
related PR: #39522
- Just implement exclusive broadcaster between broadcast message with
same resource key to keep same order in different wal.
- After simplify the broadcast model, original watch-based broadcast is
too complicated and redundant, remove it.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
