congqixia a2a51c489e

fix: [2.4] Check resource when loading deltalogs (#37195) (#37263) ...

Cherry pick from master
pr: #37195
Related to #36887

`LoadDeltaLogs` API did not check memory usage. When system is under
high delete load pressure, this could result into OOM quit.

This PR add resource check for `LoadDeltaLogs` actions and separate
internal deltalog loading function with public one.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

2024-10-30 11:54:41 +08:00

..

metricsutil

enhance: add more metrics for cache and search (#31777)

2024-04-10 10:55:17 +08:00

state

fix: async warmup will be blocked by state lock (#33687)

2024-06-07 14:23:54 +08:00

cgo_util.go

enhance: optimize milvus core building (#35660)

2024-08-27 18:55:00 +08:00

collection.go

enhance: [Cherry-pick]Add collection name label for some metric (#36951) (#37159)

2024-10-29 17:38:22 +08:00

count_reducer_test.go

enhance: retrieve output fields after local reduce (#32346)

2024-04-25 09:49:26 +08:00

count_reducer.go

enhance: retrieve output fields after local reduce (#32346)

2024-04-25 09:49:26 +08:00

default_limit_reducer.go

fix: two-phase retrieval on lru-segment (#32945) (#33313)

2024-05-23 16:25:40 +08:00

index_attr_cache_test.go

fix: predict inverted index resource usage more reasonably (#31615)

2024-03-27 19:33:09 +08:00

index_attr_cache.go

enhance: optimize milvus core building (#35660)

2024-08-27 18:55:00 +08:00

load_field_data_info.go

enhance: optimize milvus core building (#35660)

2024-08-27 18:55:00 +08:00

load_index_info.go

enhance: [2.4] the estimate method when loading the collection (#36728)

2024-10-11 10:20:45 +08:00

manager_test.go

enhance: [cherry-pick]growing segment support mmap (#34110)

2024-06-25 14:52:07 +08:00

manager.go

enhance: [2.4] Batch forward delete when using DirectForward (#37076) (#37107)

2024-10-25 11:53:29 +08:00

mock_collection_manager.go

enhance: [Cherry-pick]Add collection name label for some metric (#36951) (#37159)

2024-10-29 17:38:22 +08:00

mock_data.go

enhance: Update protobuf-go to protobuf-go v2 (#34394) (#35555)

2024-08-21 18:50:58 +08:00

mock_loader.go

feat: LRU cache implementation (#32567)

2024-05-06 20:29:30 +08:00

mock_segment_manager.go

fix: load operation when segment is on releasing (#33699)

2024-06-14 11:47:57 +08:00

mock_segment.go

enhance: Use BatchPkExist to reduce bloom filter func call cost (#33752)

2024-06-12 17:45:58 +08:00

OWNERS

Fix offline set always nil (#23340)

2023-04-11 14:42:30 +08:00

plan_test.go

enhance: Update protobuf-go to protobuf-go v2 (#34394) (#35555)

2024-08-21 18:50:58 +08:00

plan.go

enhance: optimize milvus core building (#35660)

2024-08-27 18:55:00 +08:00

pool_test.go

enhance: Execute bloom filter apply in parallel to speed up segment predict (#33793)

2024-06-13 14:14:04 +08:00

pool.go

fix: [2.4] Use singleton delete pool and avoid goroutine leakage (#37225)

2024-10-29 14:44:23 +08:00

reduce_test.go

feat: [2.4] provide more general configuration to control mmap behavior (#35609)

2024-08-23 12:35:02 +08:00

reduce.go

enhance: optimize milvus core building (#35660)

2024-08-27 18:55:00 +08:00

reducer_test.go

fix: two-phase retrieval on lru-segment (#32945) (#33313)

2024-05-23 16:25:40 +08:00

reducer.go

fix: [2.4] Return record with largest timestamp for entires with same PK (#33936) (#34024)

2024-06-21 14:14:01 +08:00

result_sorter.go

Fix bug of missing JSON type when sorting retrieve results (#25412) (#25455)

2023-07-10 20:02:28 +08:00

result_test.go

enhance: [skip e2e][2.4] Add unittest for reducing duplicated pk from multi segments (#36433) (#36460)

2024-09-24 18:09:14 +08:00

result.go

fix: [2.4] overwrite correct selection when pk duplicated (#35826) (#36274)

2024-09-14 14:27:07 +08:00

retrieve_test.go

feat: [2.4] provide more general configuration to control mmap behavior (#35609)

2024-08-23 12:35:02 +08:00

retrieve.go

fix: two-phase retrieval on lru-segment (#32945) (#33313)

2024-05-23 16:25:40 +08:00

search_test.go

feat: [2.4] provide more general configuration to control mmap behavior (#35609)

2024-08-23 12:35:02 +08:00

search.go

fix: add request resource timeout for lazy load, refactor context usage in cache (#32709)

2024-05-07 16:33:30 +08:00

segment_do.go

fix: two-phase retrieval on lru-segment (#32945) (#33313)

2024-05-23 16:25:40 +08:00

segment_interface.go

enhance: Use BatchPkExist to reduce bloom filter func call cost (#33752)

2024-06-12 17:45:58 +08:00

segment_l0.go

enhance: Enhance segment log (#36848) (#36849)

2024-10-15 20:43:30 +08:00

segment_loader_test.go

fix: load operation when segment is on releasing (#33699)

2024-06-14 11:47:57 +08:00

segment_loader.go

fix: [2.4] Check resource when loading deltalogs (#37195) (#37263)

2024-10-30 11:54:41 +08:00

segment_test.go

feat: [2.4] provide more general configuration to control mmap behavior (#35609)

2024-08-23 12:35:02 +08:00

segment.go

enhance: Support memory mode chunk cache (#35347) (#35836)

2024-10-18 17:03:25 +08:00

statistics.go

Protect segment from being released while query/search (#26322)

2023-08-16 18:38:17 +08:00

trace.go

enhance: optimize milvus core building (#35660)

2024-08-27 18:55:00 +08:00

utils_test.go

feat: [2.4] provide more general configuration to control mmap behavior (#35609)

2024-08-23 12:35:02 +08:00

utils.go

enhance: [2.4]use growingMmapEnabled to control the behavior of interim index, not vectorField (#36391)

2024-10-17 20:23:25 +08:00

validate.go

fix: [2.4] Add defer Unpin when error happens (#36620) (#36665)

2024-10-08 18:29:20 +08:00