Related to #42629
This PR handles converting default value to int8/int18 scalar with int32
default value definition
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #42028
- limit the concurrency of zstd compression.
- zstd.go modified from
`github.com/apache/arrow/go/v17/parquet/compress/ztsd.go`
- may be related to #42129
Signed-off-by: chyezh <chyezh@outlook.com>
issue: #39333#41570
Fix delete record missing issue introduced in PR #39552 L0 refactoring:
- Use delete checkpoint as consume start position when deleteCP <
channelCP
- Add logging when delete checkpoint is used instead of seek position
- Prevent delete record loss when deleteCP is earlier than default
channelCP
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue #42492
- streamingcoord start before old rootcoord.
- streaming balancer will check the node session synchronously to avoid
redundant operation when cluster startup.
- ddl operation will check if streaming enabled, if the streaming is not
enabled, it will use msgstream.
- msgstream will initialize if streaming is not enabled, and stop when
streaming is enabled.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
Related to #39173
Like logic in #41919, storage v2 fs shall use complete paths with
bucketName prefix to be compatible with its definition. This PR fills
bucket name from config when creating reader for compaction tasks.
NOTE: the bucket name shall be read from task params config for
compaction task pooling.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Related to #39173
`nullable` flag is crucial for serde logic of v2 writer, missing this
flag causes logic bug for v2 nullalbe data.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #42098#42404
related to: ##42009 #41937
Implement new method to handle partition removal from next target
without directly modifying current target.
Changes include:
- Add RemovePartitionFromNextTarget method and deprecate RemovePartition
- Update target_observer to use new method for ReleasePartition
operations
- Add unit tests and mock methods for new functionality
This ensures that all changes to next target will propagates to
delegator's query view.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Increase insert buffer size from 16MB to 64MB, while keeping delete
buffer size at 16MB.
issue: https://github.com/milvus-io/milvus/issues/42518
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Each cell of a field should be written to its own mmap file, rather than
writing all cells of the field into a single mmap file.
related: #39173
---------
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
the manager's logging lambda should not capture the pipeline object
this creates a circular reference between the manager and the pipeline
object, making it impossible for both to be GC-ed.
issue: https://github.com/milvus-io/milvus/issues/42581
Signed-off-by: Buqian Zheng <buqianzheng@Buqians-MacBook-Air.local>
Co-authored-by: Buqian Zheng <buqianzheng@Buqians-MacBook-Air.local>
Previous PR: #42017 did not work due to following updated points by this
PR:
- Initialize the `name_map`, which not touched at all before
- Trim the thread name under 15 characters to fit syscall limit
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #42523
Resolve deadlock issue in GroupedAllocator where resource release
notifications fail to propagate across hierarchy levels, causing child
allocators to wait indefinitely.
Changes include:
- Add recursive notify() method to GroupedAllocator
- Ensure all child allocators receive resource release notifications
- Fix TOCTOU race condition in hierarchical resource management
This resolves the intermittent test timeout in TestGroupedAllocator test
that exhibited deadlock due to missed condition notifications.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Related to #42530
The cluster id is missing when drop worker drop causing redoing task on
report duplicated task error.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #42498
- fix: sealed segment cannot be flushed after upgrading
- fix: get mvcc panic when upgrading
- ignore the L0 segment when graceful stop of querynode.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
