SimFG
b58a5617ef
enhance: add the seal segment when dispatch delete msgs ( #34565 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-10 19:14:51 +08:00
congqixia
80b620ebcf
fix: Check err is ErrKeyNotFound when CASCachedValue ( #34488 )
...
See also #33785
When config item is not present in paramtable, CAS fails due to
GetConfig returns error.
This PR make this returned err instance of ErrKeyNotFound and check
error type in \`CASCachedValue\` methods.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-08 22:00:16 +08:00
congqixia
c689ef4822
fix: Remove eviction operations out of lock ( #33834 )
...
See also #33823
`EvictCacheValueByFormat` may be block by on going `CASCacheValue` and
cause possible deadlock
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-13 21:15:55 +08:00
congqixia
b90999b741
fix: Compare config value then swap when caching param value ( #33785 )
...
See also #33784
This PR change the behavior of `SetCacheValue` of config manager:
- Use mutex and map instead of concurrent map for `configCache`
- Compare config raw value before set cache value
With this implementation, concurrent caching & eviction shall always
have current output:
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 |get | |old value|null |
|t1 |CAS OK | |old value|old value|
|t2 | |update |new value|old value|
|t3 | |eviction|new value|null |
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 |get | |old value|null |
|t1 | |update |new value|null |
|t2 |CAS fail| |old value|null |
|t3 | |eviction|new value|null |
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 | |update |new value|null |
|t1 |get | |new value|null |
|t2 |CAS OK | |new value|new value|
|t3 | |eviction|new value|null |
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 | |update |new value|null |
|t1 |get | |new value|null |
|t2 | |eviction|new value|null |
|t3 |CAS OK | |new value|new value|
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 | |update |new value|null |
|t1 | |eviction|new value|null |
|t2 |get | |new value|null |
|t3 |CAS OK | |new value|new value|
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-12 18:51:56 +08:00
aoiasd
2422084a29
fix: paramtable cache cause dynamic config non-dynamic ( #33473 )
...
relate: https://github.com/milvus-io/milvus/issues/33461
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-06-04 11:39:46 +08:00
Xiaofan
b6fefee0cf
fix: etcd not connectable when auth enabled ( #31633 )
...
Fix etcd config source didn't respect auth enabled
Also removed pulsar recoverable error when pulsar return ConsumerBusy.
It could happen that pulsar didn't find the original consumer is dead
and recover takes some time.
fix #31631
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-04-01 15:23:19 +08:00
congqixia
74b7de3814
enhance: Cache formatted key for param item ( #31388 )
...
See also #30806
`formatKey` may cost lots of CPU on string processing under high QPS
scenario, this PR adds a formattedKeys cache preventing string operation
in each param get value.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-19 14:05:05 +08:00
aoiasd
b724753137
enhance: Add runtime config to paramtable ( #31006 )
...
relate: https://github.com/milvus-io/milvus/issues/30806
Avoid use string convert or format function when get some runtime
parameter
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-15 11:07:06 +08:00
Chun Han
3298e64bd3
enhance: cache config values for saving cpu cycles to parse config item ( #30947 )
...
related: #30958
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-03-12 11:09:04 +08:00
aoiasd
bbff9193d9
enhance: support clean paramtable config event in test ( #30534 )
...
relate: https://github.com/milvus-io/milvus/issues/30441
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-02-20 14:16:51 +08:00
aoiasd
8385157717
enhance: adjust config source for support config event use paramtable ( #29995 )
...
Adjust config source for support config event which for dynamic config
could use paramtable and not deadlock.
relate: https://github.com/milvus-io/milvus/issues/29807
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-01-26 19:07:00 +08:00
congqixia
d73b534f1e
fix: use atomic.Pointer to store EventHandler in case of data race ( #30205 )
...
Resolves #30204
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-23 19:09:04 +08:00
wei liu
797847904c
enhance: Change some frequency log to rated level ( #29720 )
...
This PR change some frequency log to rated level
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-01-11 16:30:50 +08:00
wei liu
1f759837c4
fix: remove Unnecessary lock in config manager ( #29836 )
...
issue: #29709 #291712
to avoid concurrent recursive RLock and Lock cause deadlock, This PR
remove the unnecessary lock in config manager
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-01-11 13:48:49 +08:00
MrPresent-Han
ed644983e2
enhance: add param for bloomfilter( #29388 ) ( #29490 )
...
related: #29388
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-12-28 18:10:46 +08:00
aoiasd
89d8ce2f73
enhance: refine access log to support format access log by yaml and print name info. ( #28319 )
...
relate: https://github.com/milvus-io/milvus/issues/28086
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-28 15:32:31 +08:00
congqixia
f1fc19e8a9
enhance: Add unittest for config.EventDispatcher ( #28552 )
...
These is no enough unittest case for EventDispatcher, see also #28540 .
This PR add unit test case for all methods for EventDispatcher
Related to #28538
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-18 19:12:20 +08:00
congqixia
c948a437a9
fix: protect EventDispatcher map with mutex ( #28540 )
...
Add mutex protection for `EventDispatcher.registry` map
Fix #28538
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-17 20:32:20 +08:00
SimFG
7dda2e8814
Change some log level in the pkg package ( #28181 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-08 23:34:22 +08:00
SimFG
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam ( #26725 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-05 10:31:48 +08:00
yah01
c3f5856fbc
Fix data race for config with FileSource ( #26518 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-24 09:18:24 +08:00
congqixia
0bc03ede0d
Add eventlog pkg and support grpc streaming event observation ( #25812 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-25 17:23:01 +08:00
Gao
de6e4817a2
Support dynamic tuning config ( #25152 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-07-03 15:18:24 +08:00
yah01
ebd0279d3f
Check error by Error() and NoError() for better report message ( #24736 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
wei liu
8e3ba74648
fix qc service unstable ut ( #24340 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-24 18:49:25 +08:00
congqixia
7e9ef36de4
Fix TestConfigFromRemote/close_manager ut is not stable ( #24279 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-22 09:59:25 +08:00
congqixia
3a66e1de65
Use suite for integration test ( #24253 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-19 15:57:24 +08:00
congqixia
de4b4dafef
Refresh etcd source config with Searializable option ( #23954 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-09 11:52:40 +08:00
Enwei Jiao
086f3bd748
Add it for refresh config ( #23773 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-05-06 17:34:39 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
