yihao.dai 9fbd41a97d

fix: Adjust binlog and parquet reader buffer size for import (#43495) ...

1. Modify the binlog reader to stop reading a fixed 4096 rows and
instead use the calculated bufferSize to avoid generating small binlogs.
2. Use a fixed bufferSize (32MB) for the Parquet reader to prevent OOM.

issue: https://github.com/milvus-io/milvus/issues/43387

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

2025-07-23 21:28:54 +08:00

..

hash_test.go

fix: Fix no candidate segments error for small import (#41771)

2025-05-14 15:30:21 +08:00

hash.go

enhance: bulkinsert handles nullable/default (#42127)

2025-05-28 18:02:28 +08:00

memory_allocator_test.go

fix: Prevent import from generating small binlogs (#43132)

2025-07-07 21:32:47 +08:00

memory_allocator.go

fix: Prevent import from generating small binlogs (#43132)

2025-07-07 21:32:47 +08:00

mock_task_manager.go

fix: Prevent import from generating small binlogs (#43132)

2025-07-07 21:32:47 +08:00

mock_task.go

fix: Prevent import from generating small binlogs (#43132)

2025-07-07 21:32:47 +08:00

pool_test.go

fix: Prevent import from generating small binlogs (#43132)

2025-07-07 21:32:47 +08:00

pool.go

fix: Prevent import from generating small binlogs (#43132)

2025-07-07 21:32:47 +08:00

scheduler_test.go

fix: Prevent import from generating small binlogs (#43132)

2025-07-07 21:32:47 +08:00

scheduler.go

fix: Adjust binlog and parquet reader buffer size for import (#43495)

2025-07-23 21:28:54 +08:00

task_import.go

fix: Fix import fileStats incorrectly set to nil (#43463)

2025-07-22 12:37:01 +08:00

task_l0_import_test.go

enhance: ChunkManager is no longer created during datanode initialization (#42791)

2025-06-17 17:06:38 +08:00

task_l0_import.go

fix: Fix import fileStats incorrectly set to nil (#43463)

2025-07-22 12:37:01 +08:00

task_l0_preimport.go

fix: Fix import fileStats incorrectly set to nil (#43463)

2025-07-22 12:37:01 +08:00

task_manager_test.go

enhance: Use v2 package name for pkg module (#39990)

2025-02-22 23:15:58 +08:00

task_manager.go

enhance: Use v2 package name for pkg module (#39990)

2025-02-22 23:15:58 +08:00

task_preimport.go

fix: Fix import fileStats incorrectly set to nil (#43463)

2025-07-22 12:37:01 +08:00

task.go

fix: Prevent import from generating small binlogs (#43132)

2025-07-07 21:32:47 +08:00

util_test.go

enhance: bulkinsert handle nullable/defaultValue/functionOutput fields (#42956)

2025-07-04 14:20:44 +08:00

util.go

fix: Fix remaining issues with Datanode pooling and StorageV2 (#43147)

2025-07-10 14:26:48 +08:00