241 Commits

Author SHA1 Message Date
smellthemoon
73f2bab454
enhance:add some log when create client and get component states (#28160)
/kind improvement

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-22 09:12:22 +08:00
yah01
cc952e0486
enhance: optimize forwarding level0 deletions by respecting partition (#28456)
- Cache the level 0 deletions after loading level0 segments
- Divide the level 0 deletions by partition
related: #27349

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-21 18:24:22 +08:00
aoiasd
b1fbc1f4e7
fix: datacoord l0 segment bugs (#28490)
relate: https://github.com/milvus-io/milvus/issues/27675

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-21 16:48:22 +08:00
aoiasd
13a5b9f64a
fix: query l0 segment bugs (#28558)
relate: https://github.com/milvus-io/milvus/issues/27675

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-20 17:26:23 +08:00
XuanYang-cn
40d5c902b6
Enable getting multiple segments in plan result (#28350)
Compaction plan result contained one segment for one plan. For l0
compaction would write to multiple segments, this PR expand the segments
number in plan results and refactor some names for readibility.

- Name refactory: - CompactionStateResult -> CompactionPlanResult -
CompactionResult -> CompactionSegment

See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-14 15:56:19 +08:00
XuanYang-cn
a153950b10
Change channel to Interface (#27839)
This PR changes `*channel` into RWChannel interface

See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-13 11:16:18 +08:00
wei liu
5b45a138b1
disable auto balance when old node exists (#28191)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-07 14:02:20 +08:00
aoiasd
1d4be0d257
Adjust datacoord for L0 Delta (#28021)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-06 15:26:16 +08:00
wei liu
7e0bda8b4c
remove binlog/delta log from getRecoveryInfoV2 (#27895)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-30 14:28:15 +08:00
Xiaofan
2ea7579dbb
Reduce rpc size for GetRecoveryInfoV2 (#27483)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-10-23 21:44:09 +08:00
congqixia
49516d44b4
Add ctx parameter and log tracer for watch and selectNodes (#27809)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-20 04:22:11 +08:00
yihao.dai
49b3a12804
Return newly defined merr instead of grpc unimplemented err (#27751)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-18 15:32:11 +08:00
congqixia
efe8984eca
Fix Flush error message typo (#27729)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-17 17:18:09 +08:00
wayblink
e3f2122618
Expose metrics of stanby coordinators (#27698)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-10-16 15:04:09 +08:00
yihao.dai
c882e9c87b
Fix compatibility of Flush (#27543)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-13 18:01:39 +08:00
yah01
be980fbc38
Refine state check (#27541)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
MrPresent-Han
cb71a3e235
rm dependency to rc when getting recovery info(#25363) (#27405)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-10-09 18:51:32 +08:00
yah01
6539a5ae2c
Refine DataCoord status (#27262)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-26 17:15:27 +08:00
jaime
7f7c71ea7d
Decoupling client and server API in types interface (#27186)
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
yah01
773080df24
Fix miss to handle the error of allocating ts (#27264)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-20 18:43:23 +08:00
yihao.dai
fd73213539
Increase FlushChannel timeout to 3min (#27180)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-18 19:49:22 +08:00
yah01
a6b98740b7
Replace all status with only error string (#27125)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-18 14:05:28 +08:00
yihao.dai
c162c6a4c8
Increase FlushChannel retry times (#27140)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-16 00:41:20 +08:00
yihao.dai
fe5781e2b8
Maintain compatibility of GetFlushState with old client (#27077)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-14 19:57:20 +08:00
yihao.dai
dd2cb1d44a
Flush by flush channels (#26859)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-12 21:07:19 +08:00
SimFG
0901b76732
Avoid the panic when the status of rpc response is nil (#26910)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-07 19:23:15 +08:00
wayblink
d7b6f3a9e8
Enhance FlushAll/GetFlushAllState API (#26802)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-09-06 10:35:48 +08:00
yah01
3349db4aa7
Refine errors to remove changes breaking design (#26521)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00
yah01
bed034a44b
Protect DataCoord from calculating segment lines by stale log entries num (#26523)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-08-22 14:16:22 +08:00
yihao.dai
3c62acde05
Add coll and segState lebels to DataCoordNumStoredRows monitoring (#26299)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-17 15:52:17 +08:00
congqixia
8d13717cac
Fill Collection start position timestamp in WatchInfo (#26370)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-16 09:05:32 +08:00
XuanYang-cn
830f06783a
Reduce log volumes of DC&DN (#26060)
See also: #26057

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-08-02 17:13:11 +08:00
Bingyi Sun
7613c7842c
Fix datacoord crash (#25631)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-07-17 16:54:34 +08:00
Enwei Jiao
66fdc71479
Refactor logs in DataCoord & DataNode (#25574)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-14 15:56:31 +08:00
wei liu
012fd1152a
fix datacoord consume datanode tt metrics (#25524)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-13 14:26:29 +08:00
wei liu
cba0feb119
add coordinator broker, to unify rootcoord api access (#25187)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-03 18:28:25 +08:00
jaime
18df2ba6fd
[Cherry-Pick] Support Database (#24769)
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
Xiaofan
72c5e2a41a
Fix channel reassigned to other datanodes (#25015)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-06-21 21:26:42 +08:00
wayblink
bfae6b49af
Remove datanode timetick mq, use rpc to report instead (#23156)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-06-14 14:16:38 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
congqixia
f6dec281c4
Add Statistic entity id log for datacoord (#24073)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-12 18:11:21 +08:00
Enwei Jiao
7e645dabf1
Add trace log for datacoord (#24046)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-05-12 09:53:21 +08:00
wei liu
cbfe7a45ef
fix pull target (#23491)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-18 18:30:32 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
congqixia
127867b873
Add ratedgroup for some info/warning log (#23095)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-31 15:22:23 +08:00
yihao.dai
a182594bf5
Add FlushAll support (#22942)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-03-30 11:28:25 +08:00
SimFG
b57e476089
Fix the nil point about the session (#22748)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-03-14 20:07:54 +08:00
wayblink
aa6212d5b0
[Cherry-pick] Fix channel checkpoint issues and fix GC (#22559)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: Ten Thousand Leaves <69466447+soothing-rain@users.noreply.github.com>
2023-03-09 14:13:52 +08:00
jaime
d126f06946
Decouple mq module from internal proto definition (#22536)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-03-04 23:21:50 +08:00
yah01
319f1773af
Fix segment not found if forward delete to empty segment (#22528)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-03 14:13:49 +08:00