Zhen Ye
082ca62ec1
enhance: support balancer interface for streaming client to fetch streaming node information ( #43969 )
...
issue: #43968
- Add ListStreamingNode/GetWALDistribution to fetch streaming node info
- Add SuspendRebalance/ResumeRebalance to enable or stop balance
- Add FreezeNodeIDs/DefreezeNodeIDs to freeze target node
Signed-off-by: chyezh <chyezh@outlook.com>
2025-08-21 15:55:47 +08:00
zhagnlu
c04d678ad4
enhance: make segcore params effective without restarting milvus ( #43231 )
...
#43230
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-08-08 10:33:48 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module ( #39990 )
...
Related to #39095
https://go.dev/doc/modules/version-numbers
Update pkg version according to golang dep version convention
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
jaime
8a4ac8cccd
enhance: expose more metrics data ( #39456 )
...
issue: #36621 #39417
1. Adjust the server-side cache size.
2. Add source information for configurations.
3. Add node ID for compaction and indexing tasks.
4. Resolve localhost access issues to fix health check failures for
etcd.
Signed-off-by: jaime <yun.zhang@zilliz.com>
2025-02-07 11:50:50 +08:00
Zhen Ye
285289d5b0
enhance: implement kafka for wal ( #38598 )
...
issue: #38399
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-25 10:22:50 +08:00
Zhen Ye
118678bd91
enhance: make runtime config into a global environment table ( #38671 )
...
issue: #38399
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-24 21:00:49 +08:00
Ted Xu
a9538f6e96
fix: config file validator missed the file tail validation ( #37608 )
...
See: #32168
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-11-13 15:14:30 +08:00
Ted Xu
d9a40784a2
fix: fallback params may be overridden ( #35972 )
...
See #35756
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-09-05 16:19:04 +08:00
wei liu
853d2f33e0
enhance: Trim space during parsing string list from string ( #35546 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-19 20:48:55 +08:00
wei liu
344dc6a9f8
enhance: enable to set load config in cluster level ( #35169 )
...
issue: #35170
This PR enable to set load configs in cluster level, such as replicas
and resource groups. then when load collections will use the load
config.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-07 12:38:21 +08:00
congqixia
8b5754f7fe
enhance: Add segment seal proportion jitter ( #34636 )
...
See also #34574
Add jitter for segment seal proportion to avoid seal operation burst in
short period of time.
This PR also fix license header in paramtable pkg.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-15 14:47:39 +08:00
chyezh
1bc3c0b925
enhance: implement balancer at streaming coord ( #34435 )
...
issue: #33285
- add balancer implementation
- add channel count fair balance policy
- add channel assignment discover grpc service
Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-11 09:58:48 +08:00
congqixia
94fb5805d5
fix: Use raw parameter value to perform CAS ( #34343 )
...
See also #34342
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-03 16:00:08 +08:00
congqixia
b90999b741
fix: Compare config value then swap when caching param value ( #33785 )
...
See also #33784
This PR change the behavior of `SetCacheValue` of config manager:
- Use mutex and map instead of concurrent map for `configCache`
- Compare config raw value before set cache value
With this implementation, concurrent caching & eviction shall always
have current output:
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 |get | |old value|null |
|t1 |CAS OK | |old value|old value|
|t2 | |update |new value|old value|
|t3 | |eviction|new value|null |
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 |get | |old value|null |
|t1 | |update |new value|null |
|t2 |CAS fail| |old value|null |
|t3 | |eviction|new value|null |
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 | |update |new value|null |
|t1 |get | |new value|null |
|t2 |CAS OK | |new value|new value|
|t3 | |eviction|new value|null |
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 | |update |new value|null |
|t1 |get | |new value|null |
|t2 | |eviction|new value|null |
|t3 |CAS OK | |new value|new value|
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 | |update |new value|null |
|t1 | |eviction|new value|null |
|t2 |get | |new value|null |
|t3 |CAS OK | |new value|new value|
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-12 18:51:56 +08:00
chyezh
641f702f64
fix: add request resource timeout for lazy load, refactor context usage in cache ( #32709 )
...
issue: #32663
- Use new param to control request resource timeout for lazy load.
- Remove the timeout parameter of `Do`, remove `DoWait`. use `context`
to control the timeout.
- Use `VersionedNotifier` to avoid notify event lost and broadcast,
remove the redundant goroutine in cache.
related dev pr: #32684
Signed-off-by: chyezh <chyezh@outlook.com>
2024-05-07 16:33:30 +08:00
aoiasd
b724753137
enhance: Add runtime config to paramtable ( #31006 )
...
relate: https://github.com/milvus-io/milvus/issues/30806
Avoid use string convert or format function when get some runtime
parameter
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-15 11:07:06 +08:00
Chun Han
3298e64bd3
enhance: cache config values for saving cpu cycles to parse config item ( #30947 )
...
related: #30958
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-03-12 11:09:04 +08:00
PowderLi
bcd6865b29
enhance: add 3 builtin roles ( #28961 )
...
issue: #28960 [milvus-proto
#212 ](https://github.com/milvus-io/milvus-proto/issues/212 )
add new configuration: builtinRoles
user can define roles in config file: `milvus.yaml`
there is an example:
1. db_ro, only have read privileges, include load
2. db_rw, read and write privileges, include create/drop/rename
collection
3. db_admin, not only read and write privileges, but also user
administration
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-18 14:28:41 +08:00
aoiasd
89d8ce2f73
enhance: refine access log to support format access log by yaml and print name info. ( #28319 )
...
relate: https://github.com/milvus-io/milvus/issues/28086
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-28 15:32:31 +08:00
yah01
9c55a7f422
Add worker num as one of load resource ( #26045 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-01 21:47:06 +08:00
MrPresent-Han
f4e72cb170
remove sync segmentLastExpire every time when assigning( #25271 ) ( #25316 ) ( #25557 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-07-24 14:11:07 +08:00
MrPresent-Han
ea3817fbbc
refinement towards sync_cp_lag_too_behind_policy to avoid submit syncTasks too frequently( #25441 ) ( #25442 ) ( #25500 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-07-12 20:22:29 +08:00
chyezh
d7d61f529c
[Feature|Pick] enable scheduler policy and add user-task-polling policy ( #24839 )
...
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-07-03 18:24:25 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
