19 Commits

Author SHA1 Message Date
congqixia
6452d146af
enhance: move jemalloc_stats from pkg to internal/util/segcore (#46560)
Related to #46133

Move jemalloc_stats.go and its test file from pkg/util/hardware to
internal/util/segcore. This is a more appropriate location because:
- jemalloc_stats depends on milvus_core C++ library via cgo
- The pkg directory should remain independent of internal C++
dependencies
- segcore is the natural home for core memory allocator utilities

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Improved internal code organization by reorganizing memory statistics
collection infrastructure for better maintainability and modularity. No
impact on end-user functionality or behavior.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-12-24 19:03:18 +08:00
zhagnlu
8f0b7983ec
enhance: add jemalloc cached monitor (#46041)
#46133

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-12-09 19:53:13 +08:00
Zhen Ye
e97e44d56e
enhance: limit the gc concurrency when cpu is high (#43059)
issue: #42833

Signed-off-by: chyezh <chyezh@outlook.com>
2025-07-04 09:22:43 +08:00
Zhen Ye
f62a2e281e
enhance: add MustAsMutable... function, cooldown notifier, hardware listener (#41637)
issue: #41544

Signed-off-by: chyezh <chyezh@outlook.com>
2025-05-06 01:16:52 +08:00
Julien Salleyron
9de0c84576
fix: Allow to compile on windows (#41448)
This PR fixes #41384 .

When using milvus client and compile on windows, the compilation failed
with the undefined RSS error.

On windows, the way to get memory used is the same as on darwin.

Signed-off-by: Julien Salleyron <julien.salleyron@gmail.com>
2025-04-24 20:34:38 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990)
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
congqixia
57e5652f1a
enhance: Log error instead of panicking if load lock wait timeout (#39308)
Related to #39205
Previous PR #39206

This PR change wait timeout behavior to log error and return to avoid
making other collection read failure in only some collections have
deadlock

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-16 02:31:02 +08:00
presburger
38881bf591
enhance: prevent multiple query nodes from causing excessive occupancy of a single node, leading to GPU memory overflow (#39276) (#38617)
issue: #39276

Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
2025-01-15 20:15:01 +08:00
jaime
22b8f6a58e
enhance: excessive disk usage logs generated if the path does not exist (#38821)
issue: #38820

Signed-off-by: jaime <yun.zhang@zilliz.com>
2025-01-09 19:43:04 +08:00
jaime
8ed019735c
enhance: add disk stats within system metrics (#38033)
issue: ##36621

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-06 16:32:41 +08:00
jaime
84566dcb11
enhance: [skip e2e] set logrus log level to reduce output error logs (#30479)
issue: https://github.com/milvus-io/milvus/issues/30295

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-02-12 08:32:51 +08:00
jaime
ee26561b5d
fix: remove checking if running inside container (#29940)
issue: https://github.com/milvus-io/milvus/issues/29846

Since checking if Milvus running inside the container is difficult,
which is decided by the system version and cgroup version, we try to get
the memory limit of the container each time instead of checking whether
inside the container first.

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-01-14 10:24:52 +08:00
jaime
90984a2cab
enhance: Support read hardware metrics for cgroupv2 (#29850)
issue: https://github.com/milvus-io/milvus/issues/29846

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-01-12 10:30:51 +08:00
yah01
c96d07682e
enhance: enhance the accuracy of memory usage (#28554)
before this, Milvus use container/system's memory info to get the memory
usage, which could be inaccurate.

we allocates the memory by private anon mmap,
then `rss - shared` would be the accurate memory usage

resolve #28553

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-23 15:12:23 +08:00
yah01
90e2c63d9e
Fix getting incorrect CPU num (#28146)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-11-06 06:02:16 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
congqixia
e8f1b1736e
Remove log.Error(err.error())-style log (#26783)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-01 13:09:01 +08:00
yah01
9723787141
Calculate memory usage without page cache (#26389)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-16 17:04:17 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00