Chun Han 3739446a33

enhance: refine array view to optimize memory usage(#38736) (#38808) ...

related: #38736

700m data, array_length=10
non-mmap_offsets_uint64: 2.0G
mmap_offsets_uint64: 1.1G
mmap_offsets_uint32: 880MB

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>

2025-01-07 13:26:55 +08:00

..

azure

enhance: optimize milvus core building (#35610)

2024-08-23 12:35:02 +08:00

azure-blob-storage

enhance: optimize milvus core building (#35610)

2024-08-23 12:35:02 +08:00

gcp-native-storage

enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214)

2024-09-30 13:23:32 +08:00

opendal

enhance: optimize milvus core building (#35610)

2024-08-23 12:35:02 +08:00

AliyunCredentialsProvider.cpp

Migrate the ability to upload and download binlog to cpp (#22984)

2023-06-25 14:38:44 +08:00

AliyunCredentialsProvider.h

Migrate the ability to upload and download binlog to cpp (#22984)

2023-06-25 14:38:44 +08:00

AliyunSTSClient.cpp

Migrate the ability to upload and download binlog to cpp (#22984)

2023-06-25 14:38:44 +08:00

AliyunSTSClient.h

Migrate the ability to upload and download binlog to cpp (#22984)

2023-06-25 14:38:44 +08:00

BinlogReader.cpp

Refactor duplicate error class into one place (#26985)

2023-09-11 20:43:17 +08:00

BinlogReader.h

Refactor duplicate error class into one place (#26985)

2023-09-11 20:43:17 +08:00

ChunkCache.cpp

fix: Fix chunked segment can not warmup using mmap (#38492)

2024-12-17 13:42:45 +08:00

ChunkCache.h

fix: Fix chunked segment can not warmup using mmap (#38492)

2024-12-17 13:42:45 +08:00

ChunkManager.cpp

fix: minio ssl compatible issue (#31607)

2024-03-27 14:41:20 +08:00

ChunkManager.h

fix: wrong path spelling when use rootpath in segcore (#37453)

2024-11-07 11:22:25 +08:00

CMakeLists.txt

enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214)

2024-09-30 13:23:32 +08:00

DataCodec.cpp

enhance: eliminate compile warnings (part2) (#38535)

2024-12-25 15:30:50 +08:00

DataCodec.h

enhance: eliminate compile warnings (part2) (#38535)

2024-12-25 15:30:50 +08:00

DiskFileManagerImpl.cpp

enhance: catch std::stoi exception and improve error msg (#36267)

2024-09-14 16:17:08 +08:00

DiskFileManagerImpl.h

feat: support keyword text match (#35923)

2024-09-10 15:11:08 +08:00

Event.cpp

fix: Fix conan gdal building problem (#37338)

2024-10-31 21:04:16 +08:00

Event.h

feat: support chunked column for sealed segment (#35764)

2024-10-12 15:04:52 +08:00

FileManager.h

fix:fix incorrect dir operations when create or load inverted index (#38359)

2024-12-17 20:06:45 +08:00

IndexData.cpp

Add code for PanicInfo (#27364)

2023-09-27 12:01:28 +08:00

IndexData.h

[Cherry-Pick] Remove arrow uasge in FieldData (#22726)

2023-03-20 10:41:56 +08:00

InsertData.cpp

enhance: support null in c data_datacodec and load null value (#32183)

2024-07-23 16:07:51 +08:00

InsertData.h

feat: support chunked column for sealed segment (#35764)

2024-10-12 15:04:52 +08:00

LocalChunkManager.cpp

fix: Fix panic caused by removing directory (#38622)

2025-01-06 10:54:54 +08:00

LocalChunkManager.h

Migrate the ability to upload and download binlog to cpp (#22984)

2023-06-25 14:38:44 +08:00

LocalChunkManagerSingleton.h

Add ChunkCache: support get vector from storage (#26142)

2023-09-15 10:21:20 +08:00

MemFileManagerImpl.cpp

enhance: remove unused code for StorageV2 (#35132)

2024-08-01 12:08:13 +08:00

MemFileManagerImpl.h

enhance: remove unused code for StorageV2 (#35132)

2024-08-01 12:08:13 +08:00

MinioChunkManager.cpp

enhance: add scalar filtering and vector search latency metrics (#34785)

2024-07-19 14:01:39 +08:00

MinioChunkManager.h

enhance: Enhance and correct exception module (#33705)

2024-06-23 21:22:01 +08:00

MmapChunkManager.cpp

enhance: refine array view to optimize memory usage(#38736) (#38808)

2025-01-07 13:26:55 +08:00

MmapChunkManager.h

enhance: use segment id and type to register in MmapChunkManager and opt malloc in variableChunk (#33993)

2024-06-20 17:42:02 +08:00

MmapManager.h

fix: Fix chunked segment can not warmup using mmap (#38492)

2024-12-17 13:42:45 +08:00

parquet_c.h

enhance: delete some no lint code (#32182)

2024-04-26 14:15:26 +08:00

PayloadReader.cpp

feat: support chunked column for sealed segment (#35764)

2024-10-12 15:04:52 +08:00

PayloadReader.h

feat: support chunked column for sealed segment (#35764)

2024-10-12 15:04:52 +08:00

PayloadStream.cpp

Refactor duplicate error class into one place (#26985)

2023-09-11 20:43:17 +08:00

PayloadStream.h

enhance: support null in c data_datacodec and load null value (#32183)

2024-07-23 16:07:51 +08:00

PayloadWriter.cpp

enhance: support null in c data_datacodec and load null value (#32183)

2024-07-23 16:07:51 +08:00

PayloadWriter.h

enhance: support null in c data_datacodec and load null value (#32183)

2024-07-23 16:07:51 +08:00

RemoteChunkManagerSingleton.h

enhance: change opendal as compile configurable (#30384)

2024-02-20 19:16:52 +08:00

SafeQueue.h

enhance: eliminate compile warnings (part2) (#38535)

2024-12-25 15:30:50 +08:00

storage_c.cpp

enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214)

2024-09-30 13:23:32 +08:00

storage_c.h

enhance: add scalar filtering and vector search latency metrics (#34785)

2024-07-19 14:01:39 +08:00

TencentCloudCredentialsProvider.cpp

feat: Support tencent cloud object storage for milvus (#30163)

2024-01-23 11:28:56 +08:00

TencentCloudCredentialsProvider.h

feat: Support tencent cloud object storage for milvus (#30163)

2024-01-23 11:28:56 +08:00

TencentCloudSTSClient.cpp

feat: Support tencent cloud object storage for milvus (#30163)

2024-01-23 11:28:56 +08:00

TencentCloudSTSClient.h

feat: Support tencent cloud object storage for milvus (#30163)

2024-01-23 11:28:56 +08:00

ThreadPool.cpp

enhance: improve the segcore logs (#29372)

2023-12-23 21:52:43 +08:00

ThreadPool.h

enhance: limit the max pool size to 16 (#30371)

2024-01-31 14:13:06 +08:00

ThreadPools.cpp

enhance: improve the segcore logs (#29372)

2023-12-23 21:52:43 +08:00

ThreadPools.h

enhance: improve the segcore logs (#29372)

2023-12-23 21:52:43 +08:00

Types.h

enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214)

2024-09-30 13:23:32 +08:00

Util.cpp

fix: wrong path spelling when use rootpath in segcore (#37453)

2024-11-07 11:22:25 +08:00

Util.h

feat: support chunked column for sealed segment (#35764)

2024-10-12 15:04:52 +08:00