649 Commits

Author SHA1 Message Date
jaime
7d3c0d748c
fix: the incorrect number of partitions in rootcoord meta (#35600)
issue: #35698

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-26 15:20:58 +08:00
smellthemoon
098c10922b
enhance: add some check when create collection (#35629)
not support nullable==true in vector type. check it when create
collection.

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-25 16:23:05 +08:00
wei liu
5c245d51c4
enhance: Refresh proxy cache after restore rbac meta (#35635)
issue: #35443

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-22 19:09:01 +08:00
zhagnlu
3107701fe8
enhance: optimize retrieve on dynamic field (#35580)
#35514

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-08-22 14:24:56 +08:00
Zhen Ye
4d69898cb2
enhance: support single pchannel level transaction (#35289)
issue: #33285

- support transaction on single wal.
- last confirmed message id can still be used when enable transaction.
- add fence operation for segment allocation interceptor.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-19 21:22:56 +08:00
wei liu
1d49358f82
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup (#35444)
issue: #35443

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-16 10:10:53 +08:00
yihao.dai
e58d656e65
fix: Fix compile error (#35442)
issue: https://github.com/milvus-io/milvus/issues/33285

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-13 15:46:19 +08:00
yihao.dai
efadf22802
enhance: Append create partition msg to wal (#35398)
issue: https://github.com/milvus-io/milvus/issues/33285

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-13 14:28:20 +08:00
chyezh
16b0aee97f
enhance: timetick interceptor optimization (#35287)
issue: #33285

- remove redundant goroutine by using insepctor.
- remove the coutinous non-message timetick persistence
- periodically push the time tick forward without persistent timetick
message.
- add 'message type filter' deliver filter.

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-12 18:58:25 +08:00
yihao.dai
72a175478f
enhance: Append drop partition msg to wal (#35326)
issue: https://github.com/milvus-io/milvus/issues/33285

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-07 17:28:16 +08:00
Jiquan Long
b5022b0647
enhance: add metric to record disk quota (#35305)
https://github.com/milvus-io/milvus/issues/35177

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-08-07 10:26:16 +08:00
Jiquan Long
4dbd023402
enhance: add metric to record maxInsertRate and queryNodeMemoryHighWaterLevel (#35185)
fix: https://github.com/milvus-io/milvus/issues/35177

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-08-02 10:14:13 +08:00
congqixia
de8a266d8a
enhance: Enable linux code checker (#35084)
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-30 15:53:51 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394)
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-29 11:31:51 +08:00
chyezh
d93c51d5f6
enhance: change GetVChannels rpc into GetPChannelInfo (#34940)
issue: #33285

Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-25 13:29:45 +08:00
Jiquan Long
b843c91bad
feat: record the number of times milvus turns into force-deny-writing states (#34960)
/kind improvement
fix: #34961

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-07-25 11:39:46 +08:00
chyezh
39c7e06bc5
enhance: add message and msgstream msgpack adaptor (#34874)
issue: #33285

- make message builder and message conversion type safe
- add adaptor type and function to adapt old msgstream msgpack and
message interface

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-22 20:59:42 +08:00
yihao.dai
8aab6cbfac
enhance: Organize the common modules of streamingNode and dataNode (#34773)
1. Move the common modules of streamingNode and dataNode to flushcommon
2. Add new GetVChannels interface for rootcoord

issue: https://github.com/milvus-io/milvus/issues/33285

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-22 11:33:51 +08:00
SimFG
16c002b51b
enhance: unified channel name generation and conversion (#34805)
/kind improvement
- issue: #34804

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-20 23:11:39 +08:00
SimFG
203fb554a4
enhance: support to config root user's password (#34752)
- issue: #33058

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-17 20:19:42 +08:00
congqixia
67324eb809
enhance: Add l0 segment entry num quota (#34733)
See also #34670

This PR add quota configuration for l0 segment entry number per
collection. If l0 compaction cannot keep up the insertion/upsertion
rate, this feature could back press the related rate.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-17 17:35:41 +08:00
Patrick Weizhi Xu
104d0966b7
feat: support partition key isolation (#34336)
issue: #34332

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-07-11 19:01:35 +08:00
wei liu
3c7046353a
fix: RootCoord panic during check health (#34541)
issue: #34540

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-10 09:51:44 +08:00
jaime
4365308241
enhance: support setting properties in create database request (#34510)
issue: #34493

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-09 18:16:18 +08:00
jaime
60be454db0
enhance: add disk quota and max collections into db properties (#34368)
issue: #34385

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-05 18:22:17 +08:00
jaime
0426390f06
enhance: improve check health (#33800)
issue: #34264

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-01 10:16:06 +08:00
xige-16
f8bf6909c6
fix: Fix datacoord's properties overwriten (#34146)
issue: #34144 
/kind bug

Signed-off-by: xige-16 <xige2016@gmail.com>
2024-06-26 11:56:04 +08:00
jaime
9630974fbb
enhance: move rocksmq from internal to pkg module (#33881)
issue: #33956

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-06-25 21:18:15 +08:00
jaime
e653ad27e2
fix: metrics database_num is 0 after restarting rootcoord (#34011)
issue: #34041

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-06-21 11:36:00 +08:00
wayblink
5fac2fa1d2
fix: Panic if ProcessActiveStandBy returns error (#33369)
#33368

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-06-19 11:16:00 +08:00
ArenaSu
db59249776
doc: [skip-e2e] add comments for meta table (#33403)
Add commnets for meta table in internal/rootcoord/meta_table.go.

Signed-off-by: ArenaSu <704427617@qq.com>
2024-06-13 17:54:09 +08:00
SimFG
44d7e03e56
fix: reset the RootCoordQuotaStates metric before recording this metric (#33553)
- issue: #33539

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-06-03 21:09:47 +08:00
SimFG
8f46a20957
fix: show empty collection when has granted the all privilege (#33445)
- issue: #33382

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-29 20:51:44 +08:00
SimFG
1973ebbc42
fix: quotaCenter collect metrics failed because the wrong param (#33398)
- issue: #33397

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-27 18:17:42 +08:00
SimFG
cb99e3db34
enhance: add the includeCurrentMsg param for the Seek method (#33326)
/kind improvement
- issue: #33325

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-27 10:31:41 +08:00
SimFG
dd0c6d6980
fix: the panic when db isn't existed in the rate limit interceptor (#33244)
issue: #33243

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-22 15:57:39 +08:00
SimFG
e18d5aceb6
enhance: add config to control whether to init public role permissions (#33165)
issue: #33164

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-21 22:39:46 +08:00
Xiaofan
f681c4b034
enhance: remove describe index in rootcoord broker (#33206)
fix #33205
remove the dependency between datacoord and rootcoord

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-21 14:11:39 +08:00
smellthemoon
89a7c34c7a
fix: exceed etcd limit (#33041)
#32974

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-05-14 18:45:32 +08:00
wei liu
c35797c399
enhance: expose DescribeDatabase api in proxy (#32732)
issue: #32707

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-09 22:51:30 +08:00
wayblink
42d0412e93
enhance: Add channelCPs in FlushResponse (#32044)
#32609

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-04-30 09:45:27 +08:00
SimFG
0359139299
enhance: return the auth collection list when using the show collection api (#32576)
issue: #32550
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-29 19:09:34 +08:00
wei liu
04f355a802
enhance: Enable alter database props in rootcoord (#32458)
issue: #30040

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-25 10:53:25 +08:00
SimFG
31a29a2451
enhance: avoid the repeated metric info in the proxy (#32380)
issue: #30577

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-19 10:21:20 +08:00
Jiquan Long
7add7248e2
fix: remove useless log of quota center (#32307)
/kind improvement
issue: #31272

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-04-17 16:17:19 +08:00
wei liu
0d849a6c0a
fix: fix collectionInfo leak in datacoord (#32175)
issue: #32029

lack of logic to clean collection info in datacoord's meta, This PR
clean collection info after drop channel, to avoid collection info leak
in datacoord

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-15 16:33:19 +08:00
SimFG
c012e6786f
feat: support rate limiter based on db and partition levels (#31070)
issue: https://github.com/milvus-io/milvus/issues/30577
co-author: @jaime0815

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-04-12 16:01:19 +08:00
SimFG
789e014c74
enhance: add the db id for the describe collection response (#32114)
/kind improvement
issue: #32110

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-10 21:23:20 +08:00
congqixia
25a1c9ecf0
fix: Make coordinator Register not blocked on ProcessActiveStandby (#32069)
See also #32066

This PR make coordinator register successful and let
`ProcessActiveStandBy` run async. And roles may receive stop signal and
notify servers.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-10 18:49:18 +08:00
SimFG
a63cf8625c
enhance: support to invalid the database meta cache (#32078)
issue: #32077
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-10 06:45:18 +08:00