mirror of
https://gitee.com/milvus-io/milvus.git
synced 2025-12-06 09:08:43 +08:00
03f5d7c0a5
Related to #44956 **New Translator (C++)** - Added `ManifestGroupTranslator` (`internal/core/src/segcore/storagev2translator/`) - Translates manifest-based column groups to Milvus internal format - Implements `GroupCTMeta` interface for chunk-based column access - Supports both memory and mmap storage modes - Handles cache warmup policies for vector and scalar data **ChunkedSegmentSealedImpl** (`internal/core/src/segcore/ChunkedSegmentSealedImpl.cpp:333`) - Added `LoadColumnGroups(const std::string& manifest_path)`: Main entry point for manifest-based loading - Creates milvus-storage Reader from manifest file - Parallelizes column group loading using thread pool - Aggregates loading exceptions and reports errors - Added `LoadColumnGroup()`: Loads individual column group - Extracts field IDs from column group metadata - Creates ManifestGroupTranslator for each column group - Builds ProxyChunkColumn for field access - Special handling for timestamp field index construction **SegmentGrowingImpl** (`internal/core/src/segcore/SegmentGrowingImpl.cpp`) - Added similar `LoadColumnGroups()` and `LoadColumnGroup()` methods for growing segments - Maintains consistency with sealed segment loading path Storage FFI Utilities **loon_ffi/util** (`internal/core/src/storage/loon_ffi/util.cpp`) - Added `MakeInternalPropertiesFromStorageConfig()`: Converts C storage config to internal Properties - Maps all storage configuration fields (S3, GCS, Azure, local) - Handles SSL, IAM, virtual host settings - Configures connection timeouts and max connections - Added `MakeInternalLocalProperies()`: Creates local filesystem properties - Added `ToCStorageConfig()`: Converts Go StorageConfig to C representation - Added `GetColumnGroups()`: Extracts column groups from manifest file using Transaction API Protocol Buffer Changes **segcore.proto** (`pkg/proto/segcore.proto:121`) - Added `manifest_path` field to `SegmentLoadInfo` message - Enables passing manifest file path from Go layer to C++ core Go Integration **segment.go** (`internal/util/segcore/segment.go:372`) - Updated `ConvertToSegcoreSegmentLoadInfo()` to propagate `ManifestPath` field - Bridges QueryNode segment load info to Segcore format --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>