wei liu 3fbee154f6

enhance: Remove large segment ID arrays from QueryNode logs (#45719) ...

issue: #45718

Logging complete segment ID arrays caused excessive log volume (3-6 TB
for 200k segments). Remove arrays from logger fields and keep only
segment counts for observability.

Changes:
- Remove requestSegments/preparedSegments arrays from Load logger
- Remove segmentIDs from BM25 stats logs
- Remove entries structure from sync distribution log

This reduces log volume by 99.99% for large-scale operations.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>

2025-11-20 17:18:14 +08:00

..

metricsutil

enhance: Use v2 package name for pkg module (#39990)

2025-02-22 23:15:58 +08:00

state

enhance: Use v2 package name for pkg module (#39990)

2025-02-22 23:15:58 +08:00

cgo_util.go

enhance: Use v2 package name for pkg module (#39990)

2025-02-22 23:15:58 +08:00

collection_test.go

enhance: [AddField] Reopen loaded segments after AddField (#41529)

2025-04-26 08:48:39 +08:00

collection.go

enhance: set schema version when creating new collection (#45263)

2025-11-04 10:15:32 +08:00

count_reducer_test.go

enhance: Use v2 package name for pkg module (#39990)

2025-02-22 23:15:58 +08:00

count_reducer.go

enhance: add storage resource usage for vector search (#44308)

2025-09-19 20:20:02 +08:00

default_limit_reducer.go

enhance: Use v2 package name for pkg module (#39990)

2025-02-22 23:15:58 +08:00

disk_usage_fetcher.go

enhance: Implement dynamic interval updates for ticker components (#43865)

2025-08-21 10:07:47 +08:00

index_attr_cache_test.go

fix: Add AiSAQ index type RAM estimation implementation on the query node. (#45246)

2025-11-06 08:53:34 +08:00

index_attr_cache.go

fix: Add AiSAQ index type RAM estimation implementation on the query node. (#45246)

2025-11-06 08:53:34 +08:00

load_index_info.go

enhance: support json stats with shredding design (#42534)

2025-09-01 10:49:52 +08:00

manager_test.go

enhance: add a disk quota for the loaded binlog size to prevent load failure of querynode (#44893)

2025-10-19 19:44:01 +08:00

manager.go

enhance: add a disk quota for the loaded binlog size to prevent load failure of querynode (#44893)

2025-10-19 19:44:01 +08:00

mock_collection_manager.go

enhance: [AddField] Reopen loaded segments after AddField (#41529)

2025-04-26 08:48:39 +08:00

mock_loader.go

enhance: Introduce sparse filter in query (#44347)

2025-09-23 09:58:09 +08:00

mock_segment_manager.go

enhance: add a disk quota for the loaded binlog size to prevent load failure of querynode (#44893)

2025-10-19 19:44:01 +08:00

mock_segment.go

enhance: Move segment loading logic from Go layer to segcore for self-managed loading (#45488)

2025-11-14 11:21:37 +08:00

OWNERS

Fix offline set always nil (#23340)

2023-04-11 14:42:30 +08:00

pool_test.go

enhance: Use v2 package name for pkg module (#39990)

2025-02-22 23:15:58 +08:00

pool.go

enhance: Use v2 package name for pkg module (#39990)

2025-02-22 23:15:58 +08:00

reducer_test.go

enhance: Use v2 package name for pkg module (#39990)

2025-02-22 23:15:58 +08:00

reducer.go

enhance: Use v2 package name for pkg module (#39990)

2025-02-22 23:15:58 +08:00

result_sorter.go

feat: add timesatmptz data type (#44005)

2025-08-26 15:59:53 +08:00

result_test.go

enhance: add storage resource usage for vector search (#44308)

2025-09-19 20:20:02 +08:00

result.go

enhance: add storage resource usage for vector search (#44308)

2025-09-19 20:20:02 +08:00

retrieve_test.go

enhance: Introduce sparse filter in query (#44347)

2025-09-23 09:58:09 +08:00

retrieve.go

enhance: Introduce sparse filter in query (#44347)

2025-09-23 09:58:09 +08:00

search_reduce_test.go

enhance: Use v2 package name for pkg module (#39990)

2025-02-22 23:15:58 +08:00

search_reduce.go

fix: empty search groupby result causing crash(#43137) (#43214)

2025-07-10 12:04:48 +08:00

search_test.go

enhance: cachinglayer: resource management for segment loading (#43846)

2025-08-29 11:37:50 +08:00

search.go

enhance: Use v2 package name for pkg module (#39990)

2025-02-22 23:15:58 +08:00

segcore.go

enhance: move segcore codes of segment into one package (#37722)

2024-11-29 10:22:36 +08:00

segment_do.go

enhance: Use v2 package name for pkg module (#39990)

2025-02-22 23:15:58 +08:00

segment_filter.go

enhance: Introduce sparse filter in query (#44347)

2025-09-23 09:58:09 +08:00

segment_interface.go

enhance: Move segment loading logic from Go layer to segcore for self-managed loading (#45488)

2025-11-14 11:21:37 +08:00

segment_l0.go

enhance: Move segment loading logic from Go layer to segcore for self-managed loading (#45488)

2025-11-14 11:21:37 +08:00

segment_loader_test.go

enhance: remove logical usage checks during segment loading (#44743)

2025-10-13 14:21:58 +08:00

segment_loader.go

enhance: Remove large segment ID arrays from QueryNode logs (#45719)

2025-11-20 17:18:14 +08:00

segment_test.go

enhance: cachinglayer: resource management for segment loading (#43846)

2025-08-29 11:37:50 +08:00

segment.go

enhance: Move segment loading logic from Go layer to segcore for self-managed loading (#45488)

2025-11-14 11:21:37 +08:00

statistics.go

Protect segment from being released while query/search (#26322)

2023-08-16 18:38:17 +08:00

trace.go

enhance: optimize milvus core building (#35610)

2024-08-23 12:35:02 +08:00

utils_test.go

fix: Check whether index type support mmap even param enabled (#40614)

2025-03-20 15:52:13 +08:00

utils.go

enhance: add a disk quota for the loaded binlog size to prevent load failure of querynode (#44893)

2025-10-19 19:44:01 +08:00

validate.go

enhance: Introduce sparse filter in query (#44347)

2025-09-23 09:58:09 +08:00