Issue: #45129
<test>: <add new test case>
<also delete duplicate test case>
On branch feature/partial-update
Changes to be committed:
modified: milvus_client/test_milvus_client_partial_update.py
modified: milvus_client/test_milvus_client_upsert.py
---------
Signed-off-by: Eric Hou <eric.hou@zilliz.com>
Co-authored-by: Eric Hou <eric.hou@zilliz.com>
Related to #43028
Initialize the schema version field when creating a new collection
instance in QueryNode. The schema version is extracted from loadMetaInfo
and assigned to the collection, ensuring proper schema version tracking
and consistency across the distributed system.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #43897
- Alter collection/database is implemented by WAL-based DDL framework
now.
- Support AlterCollection/AlterDatabase in wal now.
- Alter operation can be synced by new CDC now.
- Refactor some UT for alter DDL.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
issue: #44172, #45210, #44851
kafka will auto reset the offset to "latest" if the offset is
Out-of-range. the recovery of milvus wal cannot read any message from
that. So once the offset is out-of-range, kafka should read from eariest
to read the latest uncleared data.
https://kafka.apache.org/documentation/#consumerconfigs_auto.offset.reset
Signed-off-by: chyezh <chyezh@outlook.com>
issue: #43897
- Load/Release collection/partition is implemented by WAL-based DDL
framework now.
- Support AlterLoadConfig/DropLoadConfig in wal now.
- Load/Release operation can be synced by new CDC now.
- Refactor some UT for load/release DDL.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
Related to #45157
Fix a bug where DataNode panics when building json stats index throws an
exception before the writer is initialized. The destructor would call
Close() on an uninitialized packed_writer_ pointer, causing a null
pointer dereference.
Changes:
- Add null check for packed_writer_ before calling Flush() and Close()
- Prevents null pointer dereference in edge cases
- Ignore close status as this is a cleanup operation
This ensures safe cleanup even when initialization fails due to
exceptions.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #43897
- Part of collection/index related DDL is implemented by WAL-based DDL
framework now.
- Support following message type in wal, CreateCollection,
DropCollection, CreatePartition, DropPartition, CreateIndex, AlterIndex,
DropIndex.
- Part of collection/index related DDL can be synced by new CDC now.
- Refactor some UT for collection/index DDL.
- Add Tombstone scheduler to manage the tombstone GC for collection or
partition meta.
- Move the vchannel allocation into streaming pchannel manager.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
issue: #45150
Removed the maximum limit constraint (value range [1, 10]) for vector
fields in a collection to support more flexible schema design.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Related to #44509
Fix a bug where QueryNodeNumEntities metrics were not updated for
collections with zero segments, causing stale metrics when all segments
are flushed or compacted.
The previous implementation used separate loops: one to update size
metrics for all collections, and another to update num entities metrics
only for collections present in the grouped segments map. Collections
with no segments were skipped in the second loop, leaving their
NumEntities metrics stale.
Changes:
- Consolidate size and num entities metric updates into single loop
- Iterate over all collections instead of grouped segments
- Get collection metadata from manager instead of segment instances
- Correctly set NumEntities to 0 for collections with no segments
- Apply the same fix to both growing and sealed segment processing
- Add nil check for collection metadata before processing
This ensures all collection metrics are updated consistently, even when
segment count drops to zero.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #45141
- make ack of broadcaster cannot canceled by rpc.
- make clone for assignment snapshot of wal balancer.
- add server id for GetReplicateCheckpoint to avoid failure.
Signed-off-by: chyezh <chyezh@outlook.com>
issue: #45088, #45086
- Message on control channel should trigger the checkpoint update.
- LastConfrimedMessageID should be recovered from the minimum of
checkpoint or the LastConfirmedMessageID of uncommitted txn.
- Add more log info for wal debugging.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
Related to #45105
This commit refactors the test MilvusClient wrapper to leverage Go's
embedding pattern and improves test organization with subtests.
**File**: `tests/go_client/base/milvus_client.go`
- **Use `typeutil.NewSet` for rate limiting**: Replace map-based
`rateLogMethods` with `typeutil.NewSet` for cleaner and more efficient
membership checking
- **Embed `*client.Client` directly**: Change `MilvusClient` structure
from wrapping the client as a field to embedding it directly
- **Remove ~380 lines of wrapper methods**: All wrapper methods
(database, collection, partition, index, read/write, RBAC, etc.) are now
unnecessary thanks to Go's embedding feature, which automatically
promotes embedded methods to the outer type
- **Simplify initialization**: Update `NewMilvusClient` and `Close` to
use embedded client directly
- **Fix typo**: Correct comment "Ike the actual method" → "Invoke the
actual method"
**File**: `tests/go_client/testcases/search_test.go`
- **Wrap assertions in subtests**: Each search expression test is now
wrapped in `t.Run()` with descriptive names
- **Dynamic subtest naming**: Format:
`expr={expression}_dynamic-{true/false}` for clear test identification
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
relate: https://github.com/milvus-io/milvus/issues/43687
Add global analyzer options, avoid having to merge some milvus params
into user's analyzer params.
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
Related to #44956
Add manifest_path field throughout the data path to support LOON Storage
V2 manifest tracking. The manifest stores metadata for segment data
files and enables the unified Storage V2 FFI interface.
Changes include:
- Add manifest_path field to SegmentInfo and SaveBinlogPathsRequest
proto messages
- Add UpdateManifest operator to datacoord meta operations
- Update metacache, sync manager, and meta writer to propagate manifest
paths
- Include manifest_path in segment load info for query coordinator
This is part of the Storage V2 FFI interface integration.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
