Zhen Ye 02e2170601
enhance: cherry pick patch of new DDL framework and CDC 2 (#45241)
issue: #43897, #44123
pr: #45224
also pick pr: #45216,#45154,#45033,#45145,#45092,#45058,#45029

enhance: Close channel replicator more gracefully (#45029)

issue: https://github.com/milvus-io/milvus/issues/44123

enhance: Show create time for import job (#45058)

issue: https://github.com/milvus-io/milvus/issues/45056

fix: wal state may be unconsistent after recovering from crash (#45092)

issue: #45088, #45086

- Message on control channel should trigger the checkpoint update.
- LastConfrimedMessageID should be recovered from the minimum of
checkpoint or the LastConfirmedMessageID of uncommitted txn.
- Add more log info for wal debugging.

fix: make ack of broadcaster cannot canceled by client (#45145)

issue: #45141

- make ack of broadcaster cannot canceled by rpc.
- make clone for assignment snapshot of wal balancer.
- add server id for GetReplicateCheckpoint to avoid failure.

enhance: support collection and index with WAL-based DDL framework
(#45033)

issue: #43897

- Part of collection/index related DDL is implemented by WAL-based DDL
framework now.
- Support following message type in wal, CreateCollection,
DropCollection, CreatePartition, DropPartition, CreateIndex, AlterIndex,
DropIndex.
- Part of collection/index related DDL can be synced by new CDC now.
- Refactor some UT for collection/index DDL.
- Add Tombstone scheduler to manage the tombstone GC for collection or
partition meta.
- Move the vchannel allocation into streaming pchannel manager.

enhance: support load/release collection/partition with WAL-based DDL
framework (#45154)

issue: #43897

- Load/Release collection/partition is implemented by WAL-based DDL
framework now.
- Support AlterLoadConfig/DropLoadConfig in wal now.
- Load/Release operation can be synced by new CDC now.
- Refactor some UT for load/release DDL.

enhance: Don't start cdc by default (#45216)

issue: https://github.com/milvus-io/milvus/issues/44123


fix: unrecoverable when replicate from old (#45224)

issue: #44962

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: chyezh <chyezh@outlook.com>
Co-authored-by: yihao.dai <yihao.dai@zilliz.com>
2025-11-04 01:35:33 +08:00

77 lines
2.1 KiB
Go

package model
import (
pb "github.com/milvus-io/milvus/pkg/v2/proto/etcdpb"
)
type Partition struct {
PartitionID int64
PartitionName string
PartitionCreatedTimestamp uint64
CollectionID int64
State pb.PartitionState
}
func (p *Partition) Available() bool {
return p.State == pb.PartitionState_PartitionCreated
}
func (p *Partition) Clone() *Partition {
return &Partition{
PartitionID: p.PartitionID,
PartitionName: p.PartitionName,
PartitionCreatedTimestamp: p.PartitionCreatedTimestamp,
CollectionID: p.CollectionID,
State: p.State,
}
}
func ClonePartitions(partitions []*Partition) []*Partition {
clone := make([]*Partition, 0, len(partitions))
for _, partition := range partitions {
clone = append(clone, partition.Clone())
}
return clone
}
func (p *Partition) Equal(other Partition) bool {
return p.PartitionName == other.PartitionName
}
func CheckPartitionsEqual(partitionsA, partitionsB []*Partition) bool {
if len(partitionsA) != len(partitionsB) {
return false
}
mapA := make(map[string]*Partition)
for _, p := range partitionsA {
mapA[p.PartitionName] = p
}
for _, p := range partitionsB {
if other, exists := mapA[p.PartitionName]; !exists || !p.Equal(*other) {
return false
}
}
return true
}
func MarshalPartitionModel(partition *Partition) *pb.PartitionInfo {
return &pb.PartitionInfo{
PartitionID: partition.PartitionID,
PartitionName: partition.PartitionName,
PartitionCreatedTimestamp: partition.PartitionCreatedTimestamp,
CollectionId: partition.CollectionID,
State: partition.State,
}
}
func UnmarshalPartitionModel(info *pb.PartitionInfo) *Partition {
return &Partition{
PartitionID: info.GetPartitionID(),
PartitionName: info.GetPartitionName(),
PartitionCreatedTimestamp: info.GetPartitionCreatedTimestamp(),
CollectionID: info.GetCollectionId(),
State: info.GetState(),
}
}