XuanYang-cn
20534a3f7b
fix: [cp24]Saperate L0 and Mix trigger interval ( #37319 )
...
See also: #37108
pr: #37190
- Add MixCompactionTriggerInterval, default 60s
- Add L0CompactionTriggerInterval, default 10s
- Export Single related compaction configs
- Raise SingleCompactionDeltaLogMaxSize from 2MB to 16MB
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-11-06 11:10:26 +08:00
jaime
da2d3fb430
enhance: enable manual compaction for collections without indexes ( #36581 )
...
issue: #36576
pr: #36577
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-08 14:51:20 +08:00
wei liu
e2542a1bf5
enhance: Update protobuf-go to protobuf-go v2 ( #34394 ) ( #35555 )
...
issue: #34252
pr: #34394 #35072 #35084
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-21 18:50:58 +08:00
cai.zhang
c340f387cf
enhance: [cherry-pick] Change the fixed value to a ratio for clustering segment size ( #35075 )
...
issue: #34495
master pr: #35076
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-31 10:32:00 +08:00
wayblink
1e5c71d550
fix: [cherry-pick] fix dropped segment still visible after dropped by L2 single compaction ( #35006 )
...
bug: #35003
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-26 13:47:48 +08:00
cai.zhang
74adedf750
enhance: Optimized the GC logic to ensure that memory is released in time ( #34950 )
...
issue: #34703
master pr: #34949
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-24 14:07:43 +08:00
Chun Han
ae1636c2be
fix: refine handling type for segment pruner( #34923 ) ( #34926 )
...
related: #34923
pr: https://github.com/milvus-io/milvus/pull/34925
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-07-24 12:05:44 +08:00
wayblink
8f3c126129
enhance: [cherry-pick] support l2 single compaction ( #34929 )
...
#34928
pr: #34935
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-24 11:47:50 +08:00
cai.zhang
4ed62e9dbb
enhance: [cherry-pick] Add integration test for clustering compaction ( #34860 )
...
issue: #34792
master pr: #34881
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2024-07-22 17:49:42 +08:00
wayblink
21973a600d
enhance: [cherry-pick] refine clustering compaction basic it ( #34794 )
...
issue: #34792
pr: #34793
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-21 20:05:41 +08:00
XuanYang-cn
edefc3cbb5
enhance: [skip e2e]Enable compaction it test ( #34526 ) ( #34720 )
...
pr: #34526
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-07-16 18:19:38 +08:00
wayblink
99586066f5
feat: [cherry-pick] Major compaction ( #34326 )
...
This PR cherry-picks the following commits:
fix: speed up segment lookup via channel name in datacoord (#33530 )
needed by the next commit
feat: Major compaction (#33620 )
issue: #30633
pr: #33620
---------
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: yiwangdr <80064917+yiwangdr@users.noreply.github.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
2024-07-02 18:29:01 +08:00
zhenshan.cao
14a11e379c
enhance: Refactor Compaction to enable persistence( #33265 ) ( #34268 )
...
pr : #33265
issue #33586
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-07-01 19:32:07 +08:00
yihao.dai
b1e74dc7cb
enhance: [cherry-pick] Decouple compaction from shard ( #34157 )
...
This PR cherry-picks the following commits:
- Implement task limit control logic in datanode.
https://github.com/milvus-io/milvus/pull/32881
- Load bf from storage instead of memory during L0 compaction.
https://github.com/milvus-io/milvus/pull/32913
- Remove dependencies on shards (e.g. SyncSegments, injection).
https://github.com/milvus-io/milvus/pull/33138
- Rename Compaction interface to CompactionV2.
https://github.com/milvus-io/milvus/pull/33858
- Remove the unused residual compaction logic.
https://github.com/milvus-io/milvus/pull/33932
issue: https://github.com/milvus-io/milvus/issues/32809
pr: https://github.com/milvus-io/milvus/pull/32881 ,
https://github.com/milvus-io/milvus/pull/32913 ,
https://github.com/milvus-io/milvus/pull/33138 ,
https://github.com/milvus-io/milvus/pull/33858 ,
https://github.com/milvus-io/milvus/pull/33932
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-06-25 20:22:03 +08:00
