sthuang 238bd30f42
fix: [StorageV2] end to end minor issues for sync, stats, and load (#42948)
Fix issues in end-to-end tests: 
1. **Split column groups based on schema**, rather than estimating by
average chunk row size. **Ensure column group consistency within a
segment**, to avoid errors caused by loading multiple column group
chunks simultaneously.
2. **Use sorted segmentId** when generating the stats binlog path, to
ensure consistent and correct file path resolution.
3. **Determine field IDs as follows**:
For multi-column column groups, retrieve the field ID list from
metadata.
For single-column column groups, use the column group ID directly as the
field ID.

related: #39173 
fix: #42862

---------

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-06-27 14:44:42 +08:00
..

Data Node

DataNode is the component to write insert and delete messages into persistent blob storage, for example MinIO or S3.

Dependency

  • KV store: a kv store that persists messages into blob storage.
  • Message stream: receive messages and publish information
  • Root Coordinator: get the latest unique IDs.
  • Data Coordinator: get the flush information and which message stream to subscribe.