Xianhui Lin
a72492169f
feat: add NotifyDropPartition in mixcoord for droppartition in dc ( #42029 )
...
add NotifyDropPartition in mixcoord for droppartition in dc
issue:https://github.com/milvus-io/milvus/issues/41976
https://github.com/milvus-io/milvus/issues/41542
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-05-23 18:32:26 +08:00
Xianhui Lin
f9febe3bae
enhance: Merge RootCoord, DataCoord And QueryCoord into MixCoord ( #41006 )
...
Merge RootCoord, DataCoord And QueryCoord into MixCoord
Make Session into one
issue : https://github.com/milvus-io/milvus/issues/37764
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-04-11 16:36:30 +08:00
XuanYang-cn
4bebca6416
enhance: Replace currRows with NumOfRows ( #40074 )
...
See also: #40068
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-03-10 12:16:03 +08:00
sthuang
63a7c4570e
feat: storage v2 sync ( #39663 )
...
related: #39173
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-03-05 11:22:15 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module ( #39990 )
...
Related to #39095
https://go.dev/doc/modules/version-numbers
Update pkg version according to golang dep version convention
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
congqixia
5d83deb3f8
fix: Use start pos ts instead for sealSegmentByLifetime policy ( #39982 )
...
Related to #39981
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-19 15:41:06 +08:00
yihao.dai
38f813bed3
enhance: Read metadata concurrently to accelerate recovery ( #38403 )
...
Read metadata such as segments, binlogs, and partitions concurrently at
the collection level.
issue: https://github.com/milvus-io/milvus/issues/37630
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-23 14:27:27 +08:00
zhenshan.cao
38b96090a8
fix: deleted the sealed segment data accidentally ( #39421 )
...
issue: https://github.com/milvus-io/milvus/issues/39333
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2025-01-20 15:29:02 +08:00
Zhen Ye
bb8d1ab3bf
enhance: make new go package to manage proto ( #39114 )
...
issue: #39095
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-10 10:49:01 +08:00
yihao.dai
d4dab3c62f
enhance: Reduce segmentManager lock granularity ( #37836 )
...
Use a channel level key lock for segments in segmentManager.
issue: https://github.com/milvus-io/milvus/issues/37633 ,
https://github.com/milvus-io/milvus/issues/37630
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-17 14:12:52 +08:00
tinswzy
7944538ade
enhance: Add ctx param to KV operation interfaces ( #38154 )
...
issue: #35917
Refine KV operation interfaces by adding a ctx param
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-05 15:16:41 +08:00
tinswzy
1dbb6cd7cb
enhance: refine the datacoord meta related interfaces ( #37957 )
...
issue: #35917
This PR refines the meta-related APIs in datacoord to allow the ctx to
be passed down to the catalog operation interfaces
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-11-26 19:46:34 +08:00
Zhen Ye
f3d9d05a28
fix: use binlog counter to trigger flush but not stats log ( #37037 )
...
issue: #36804
Signed-off-by: chyezh <chyezh@outlook.com>
2024-10-23 15:07:29 +08:00
yihao.dai
1f47d5510b
fix: Fix import segments leak in segment manager ( #36602 )
...
Directly add import segments from the meta, eliminating the dependency
on the segment manager.
issue: https://github.com/milvus-io/milvus/issues/34648
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-08 10:11:22 +08:00
congqixia
c992a61a23
enhance: Separate allocator pkg in datacoord ( #35622 )
...
Related to #28861
Move allocator interface and implementation into separate package. Also
update some unittest logic.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 10:06:56 +08:00
cai.zhang
93a500676b
fix: Remove flushed segment in segment manager generated through import ( #34649 )
...
issue: #34648
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-12 23:15:22 +08:00
congqixia
1a248f2668
enhance: Add param item for segmentFlushInterval ( #34629 )
...
See also #28817
Add paramitem for segment flush interval
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-12 18:59:35 +08:00
congqixia
be92147723
enhance: Print reason when seal segments ( #34575 )
...
See also #34574
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-11 11:40:51 +08:00
yihao.dai
3540eee977
enhance: Support L0 import ( #33514 )
...
issue: https://github.com/milvus-io/milvus/issues/33157
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-06-07 14:17:20 +08:00
smellthemoon
c61fb1eff5
enhance: do check when add not empty logpath ( #33640 )
...
meta only store logid
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-06-07 10:19:51 +08:00
SimFG
b1a1cca10b
feat: add more operation detail info for better allocation ( #30438 )
...
issue: #30436
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-03-28 06:33:11 +08:00
yihao.dai
0fe5e90e8b
enhance: Remove import v1 ( #31403 )
...
Remove all code and logic related to import v1.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-22 15:29:09 +08:00
yihao.dai
a434d33e75
feat: Add import scheduler and manager ( #29367 )
...
This PR introduces novel managerial roles for importv2:
1. ImportMeta: To manage all the import tasks;
2. ImportScheduler: To process tasks and modify their states;
3. ImportChecker: To ascertain the completion of all tasks and instigate
relevant operations.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-01 18:31:02 +08:00
yihao.dai
7ce876a072
fix: Decoupling importing segment from flush process ( #30402 )
...
This pr decoups importing segment from flush process by:
1. Exclude the importing segment from the flush policy, this approch
avoids notifying the datanode to flush the importing segment, which may
not exist.
2. When RootCoord call Flush, DataCoord directly set the importing
segment state to `Flushed`.
issue: https://github.com/milvus-io/milvus/issues/30359
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-02-03 13:01:12 +08:00
aoiasd
1d4be0d257
Adjust datacoord for L0 Delta ( #28021 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-06 15:26:16 +08:00
jaime
7f7c71ea7d
Decoupling client and server API in types interface ( #27186 )
...
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
SimFG
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
MrPresent-Han
4f6122dd4a
enable ctx traceId for assignsegment on dc( #26972 ) ( #27108 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-09-18 09:53:22 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam ( #26725 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-05 10:31:48 +08:00
MrPresent-Han
f4e72cb170
remove sync segmentLastExpire every time when assigning( #25271 ) ( #25316 ) ( #25557 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-07-24 14:11:07 +08:00
Bingyi Sun
268368031c
Remove redundant kv mocks ( #25420 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-07-07 19:38:26 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
yah01
ebd0279d3f
Check error by Error() and NoError() for better report message ( #24736 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
yah01
319f1773af
Fix segment not found if forward delete to empty segment ( #22528 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-03 14:13:49 +08:00
jaime
748a935a11
Modify segment seal policy with number of statslog files ( #21957 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-02-03 17:07:52 +08:00
Xiaofan
f1daef22a1
Fix failed bulkload segment marked as sealed ( #21799 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-31 12:41:53 +08:00
jaime
73d8509600
Improve ListSegments performance for datacoord catalog ( #21477 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-06 14:33:36 +08:00
jaime
d401608899
Add a segment seal policy by number of binlog files ( #21263 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-12-20 14:09:25 +08:00
Bingyi Sun
2390095232
Fix load uses compacted segments' binlogs ( #20655 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-11-17 20:37:10 +08:00
XuanYang-cn
22fb1d3b93
Remove dup mock_kv.go and mock_txn_kv.go ( #20267 )
...
Replace them with mockery generated mocks in kv/mocks
See also: #19289
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-11-03 14:41:35 +08:00
SimFG
a55f739608
Separate public proto files ( #19782 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
jaime
31db01b6ae
Support TTL for collection level ( #19383 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-10 20:31:22 +08:00
jaime
91df8f2d6a
Use log id instead of log path of the binlog in metastore ( #19123 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-09-25 15:56:51 +08:00
SimFG
d7f38a803d
Separate some proto files ( #19218 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
codeman
70254c3ec5
Unified catalog interface for segment ( #18289 ) ( #18290 )
...
Signed-off-by: kejiang <ke.jiang@zilliz.com>
Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-08-20 10:24:51 +08:00
Ten Thousand Leaves
f0b036a35a
Move bulk load segment lock happen early ( #17612 )
...
issue: #17600
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-27 13:56:17 +08:00
Bingyi Sun
05273309e8
Drop empty sealed segment ( #17708 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-06-24 15:16:14 +08:00
groot
f31be941bb
Fix bulkload row count bug for multi datanodes ( #17384 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-06-07 16:56:07 +08:00
Ten Thousand Leaves
e66ac6a77c
Resolve an issue with bulk load where segments in flow graph replica are not updated ( #16609 )
...
issue: #15604
/kind enhancement
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-25 11:07:47 +08:00
