mirror of
https://gitee.com/milvus-io/milvus.git
synced 2025-12-06 17:18:35 +08:00
c102fa8b0b
The Out of Memory (OOM) error occurs because a handler retains the entire ImportRecordBatch in memory. Consequently, even when child arrays within the batch are flushed, the memory for the complete batch is not released. We temporarily fixed by deep copying record batch in #43724. The proposed fix is to split the RecordBatch into smaller sub-batches by column group. These sub-batches will be transferred via CGO, then reassembled before being written to storage using the Storage V2 API. Thus we can achieve zero-copy and only transferring references in CGO. related: #43310 Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>