admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00
Code Issues Packages Projects Releases Wiki Activity
20,286 Commits 146 Branches 196 Tags
Commit Graph

1495 Commits

Author SHA1 Message Date
congqixia
9c67965d6c
fix: [2.4] Re-read value after once initialization (#35643) 
Cherry-pick from master
pr: #35642
Related to #35641
See also #35271

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-22 16:30:57 +08:00
zhagnlu
946881e123
enhance: optimize retrieve on dynamic field (#35602) 
pr: #35580

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-08-22 14:18:57 +08:00
wei liu
e2542a1bf5
enhance: Update protobuf-go to protobuf-go v2 (#34394) (#35555) 
issue: #34252
pr: #34394 #35072 #35084

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-21 18:50:58 +08:00
wei liu
4bf4cbad85
enhance: Mark query node as read only after suspend (#35492) (#35586) 
issue: #34985 #35493
pr: #35492
after querynode has been suspended, it's not allow to load
segment/channel on it, which means the node is read only. to be
compatible with resource group design, after query node has been
suspend, we remove it from it's original resource group, make it a read
only query node in replica. then two things will happens:
1. it's original resource group will be lacking of query nodes, query
coord will assign new node to it.
2. querycoord will try to move out all segments/channels after querynode
has been suspended

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-20 19:00:56 +08:00
Chun Han
cf8494ef45
enhance: support httpv1/v2 throttle and add it for httpV2(#35350) (#35504) 
related: #35350
pr: https://github.com/milvus-io/milvus/pull/35470

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-20 16:32:56 +08:00
wei liu
fc344d1eae
fix: Fix proxy panic after Import related api failed (#35540) (#35559) 
issue: #35539
pr: #35540

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-19 19:42:55 +08:00
wei liu
14ec3dc357
enhance: Enable ReadOnly/ReadWrite/Admin Privilege Group to simplify RBAC grant progress (#35472) (#35543) 
issue: #35471
pr: #35472 #35515

---------

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-19 16:24:54 +08:00
wei liu
248a6ea401
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup (#35444) (#35513) 
issue: #35443
pr: #35444

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-18 13:42:55 +08:00
congqixia
555b7a6aa7
enhance: [2.4] Remove duplicated schema helper creation in proxy (#35489) (#35502) 
Cherry-pick from master
pr: #35489
Related to PRs of #35415

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-16 14:32:54 +08:00
SimFG
4fae5407dd
enhance: [2.4] remove the invalid log (#35473) 
/kind improvement
pr: #35465

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-15 14:14:53 +08:00
aoiasd
a20cb727eb
enhance:[Cherry-pick] Check by proxy rate limiter when delete get data by query. (#30891) (#35262) 
relate: https://github.com/milvus-io/milvus/issues/30927
pr: https://github.com/milvus-io/milvus/pull/30891

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-08-13 14:32:21 +08:00
smellthemoon
23052f6ac2
fix: upsert use the previous pk in insert when autoid(#34672) (#35130) 
issue: #34668 
pr: #34672

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-08-12 18:58:25 +08:00
wei liu
a4fa9ced16
fix: delete by expr failed at retry progress (#35241) (#35268) 
issue: #35240
pr: #35241
delete by expr shard the same err object between channels, so if one
channel's query failed, it will fail all channel, which will break
channel level retry policy, and make delete operation failed.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-09 16:12:18 +08:00
zhagnlu
db9e4b898a
enhance: support bitmap index (#35336) 
pr: #32902
cherry-pick bitmap index from master

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-07 20:12:18 +08:00
Jiquan Long
b11db52160
fix: error message when input is not aligned (#35322) (#35323) 
fix: https://github.com/milvus-io/milvus/issues/35321
pr: https://github.com/milvus-io/milvus/pull/35322

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-08-07 09:56:28 +08:00
congqixia
2a1fc700a2
enhance: [2.4] Add in-memory cache for casbin enforcer result (#35272) 
Cherry-pick from master
pr: #35271
See also #35270

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-06 12:14:17 +08:00
congqixia
f8444b900f
enhance: [2.4] Support proxy/delegator qn client pooling (#35195) 
Cherry pick from master
pr: #35194
See also #35196
Add param item for proxy/delegator query node client pooling and
implement pooling logic

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-02 11:24:19 +08:00
Jiquan Long
86edca8c1b
fix: support auto index for array (#35095) 
/kind branch-feature
pr: #34450

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: Zhagnlu <lu.zhang@zilliz.com>
 2024-07-30 17:57:50 +08:00
Patrick Weizhi Xu
e2fc8853b6
fix: [2.4] disallow expr when partition key isolation is enabled (#35025) 
issue: #34336 
pr: #35031

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-07-29 14:23:49 +08:00
congqixia
d16320705e
enhance: [2.4] Add Segment Level in milvus segment info APIs (#34763) (#35023) 
Cherry-pick from master
pr: #34763
See also #34746

This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-29 10:11:52 +08:00
congqixia
079276c6ff
fix: [2.4] Unify hook singleton implementation in proxy (#34888) 
Cherry-pick from master
pr: #34887
Related to #34885

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-26 18:07:53 +08:00
congqixia
0dd9af8f13
fix: [2.4] Check legacy guarantee ts when skipping alloc ts (#34982) 
Cherry-pick from master
pr: #34981
See also #34980

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-25 10:27:47 +08:00
SimFG
4e95f3835a
fix: [2.4] execute method should return an error when result is a failure (#34872) 
- issue: #34812
- pr: #34813

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-22 12:13:42 +08:00
yihao.dai
ffee6b9c27
enhance: Optimize ConsistencyLevel and GuaranteeTimestamp log (#34738) (#34824) 
/kind improvement

pr: https://github.com/milvus-io/milvus/pull/34738

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-22 11:32:04 +08:00
congqixia
d1e0d4ea4a
enhance: [2.4] Add dynamic cgo pool for proxy CGO call (#34768) (#34842) 
Cherry-pick from master
pr: #34768
Related to #34705

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-21 18:05:41 +08:00
Jiquan Long
34873a7f76
feat: record the duration waiting in the proxy queue (#34744) (#34788) 
fix: https://github.com/milvus-io/milvus/issues/34743
pr: #34744

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-07-19 15:37:44 +08:00
Patrick Weizhi Xu
3cf2e1831a
feat: [cherry-pick][2.4] support partition key isolation (#34336) (#34613) 
issue: #34332
pr: #34336

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 104d0966b7df80a251437bed74f28fa47e138397)
 2024-07-17 14:03:43 +08:00
SimFG
9cf1f03e89
fix: [2.4] the lint-fix error (#34717) 
- issue: #31293
- pr: #34033
- related pr: #34699

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-16 17:37:39 +08:00
SimFG
d607a43ea6
fix: [2.4] metaCache cleanup issue when listPolicy failed (#34697) 
- issue:#34667
- pr: #34449

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-16 14:13:45 +08:00
smellthemoon
0fdb288de7
enhance: upsert support autoid(#30342) (#34633) 
pr: #30342 
issue: #29258

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
 2024-07-15 20:53:39 +08:00
SimFG
9bcab2ebee
enhance: [2.4] use the insert rate limit for the upsert request (#34616) 
- issue: #34614
- pr: #34615

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-12 16:19:36 +08:00
aoiasd
d6fc6a9f41
fix: [Cherry-pick] Restful API use deprecate error code cause access log panic. (#34579) 
Avoid panic when use deprecate error code and use merr code replace
deprecate common code.
relate: https://github.com/milvus-io/milvus/issues/34578
pr： https://github.com/milvus-io/milvus/pull/34576

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-07-11 11:58:51 +08:00
Gao
f6cd84161c
enhance: ensure autoindex default metric type compatibility (#34479) 
issue: #34304 
pr: #34261

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-07-08 19:48:14 +08:00
jaime
326370c1be
enhance: add disk quota and max collections into db properties (#34386) 
issue: https://github.com/milvus-io/milvus/issues/34385
pr: #34368

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-05 18:22:17 +08:00
PowderLi
ba2c232331
fix: [cherry-pick] [restful v2] count(*) & hook (#34433) 
issue: #31224 #34374
pr: #34369

for query api:

1. param filter is not requried
2. param limit is useless while outputFields = [count(*)]

add hook about grpc call

---------

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-07-05 09:52:10 +08:00
aoiasd
9087b6f42e
enhance: [Cherry-Pick] support mark error as user error (#33498) (#34396) 
relate: https://github.com/milvus-io/milvus/issues/33492
pr: https://github.com/milvus-io/milvus/pull/33498
---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-07-04 10:08:10 +08:00
Chun Han
014cb7b071
enhance: use configed max topk for iterator when input topk exceeds(#34292) (#34293) 
related: #34292
pr: https://github.com/milvus-io/milvus/pull/34290

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-07-04 09:58:09 +08:00
SimFG
b3c5eb29ed
enhance: [2.4] the proxy metric in the query request (#34356) 
/kind improvement
- issue: #33306
- pr: #33307

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-03 10:00:09 +08:00
wayblink
c62bf8a0b0
fix: [Cherry-pick]Pick major compaction fixs and optimizations (#34360) 
This PR cherry-picks the following commits:

- fix: sync partitiion stats blocking balance task #33742
- fix: Fix meta prefix overlap bug #33830
- fix: Small fixs of major compaction #33929 
- fix: Fix memory buffer error & some renaming #33850
- fix: sync part stats task cannot be finished #34027 
- Add an option to enable/disable vector field clustering key #34097
- fix: fix error ignore in compactor #34169
- fix:load major compaction partial result #34052
- Use new stream segment reader in clustering compaction #34232

issue: #30633
pr: #33742 #33830 #33929 #33850 #34027 #34097 #34169 #34052 #34232

---------

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: Chun Han <116052805+MrPresent-Han@users.noreply.github.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-07-03 09:53:37 +08:00
wayblink
99586066f5
feat: [cherry-pick] Major compaction (#34326) 
This PR cherry-picks the following commits:
fix: speed up segment lookup via channel name in datacoord (#33530)
needed by the next commit
  feat: Major compaction (#33620)

issue: #30633
pr: #33620

---------

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: yiwangdr <80064917+yiwangdr@users.noreply.github.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
 2024-07-02 18:29:01 +08:00
congqixia
975f3bbeab
enhance: [2.4] Refine max length exceeded error message (#34300) (#34323) 
Cherry-pick from master
pr: #34300
This PR make varchar & string array field max length exceeded error
message clearer. Also fixed a minor issue that error string format and
argument number not match.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-02 15:52:09 +08:00
aoiasd
bc8ca593bd
enhance: [Cherry-Pick] Update access log (#34295) 
Support restful api,print with consistency level and add new write
cache.
relate: https://github.com/milvus-io/milvus/issues/31823
pr: https://github.com/milvus-io/milvus/pull/33155
https://github.com/milvus-io/milvus/pull/33503
https://github.com/milvus-io/milvus/pull/32213

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-07-02 15:30:15 +08:00
wei liu
c344083f22
enhance: Optimize grow slice cost during query (#34254) 
issue: #32252
pr: #34253

This PR try to pre-allocate FieldData for Reduce operations in the Query
chain using typeutil.PrepareResultFieldData to avoid the overhead of
dynamically growing the slice during appendFieldData process.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-01 15:18:11 +08:00
cqy123456
0bd93de627
enhance: [cherry-pick]remove CheckVecIndexWithDataTypeExist function in pkg (#34104) 
issue: https://github.com/milvus-io/milvus/issues/22837
related pr: https://github.com/milvus-io/milvus/pull/34102

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-26 21:14:26 +08:00
jaime
7f636dbdf6
enhance: refine error message for invalid limit parameter (#34179) 
pr: #34178

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-06-26 20:46:06 +08:00
jaime
6423b6c718
enhance: move rocksmq from internal to pkg (#34165) 
pr:  https://github.com/milvus-io/milvus/pull/33881
issue:  https://github.com/milvus-io/milvus/issues/33956

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-06-26 13:36:05 +08:00
Patrick Weizhi Xu
1168e8c107
enhance: [2.4] support integral type for MV and skip MV if there is only one category (#34005) 
pr: #33161
issue: #29892

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-06-24 19:12:02 +08:00
wei liu
061a00c58f
enhance: Enable database level replica num and resource groups for loading collection (#33052) (#33981) 
pr: #33052

issue: #30040

This PR introduce two database level props:
1. database.replica.number
2. database.resource_groups

User can set those two database props by AlterDatabase API, then can
load collection without specified replica_num and resource groups. then
it will use database level load param when try to load collections.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-21 16:56:02 +08:00
wei liu
2d3c5191b8
enhance: Refine name rule check error msg (#33815) (#33900) 
pr: #33815

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-19 14:26:01 +08:00
wei liu
4513569207
enhance: add restful api to trigger component stop (#32076) (#33799) 
issue: #32698
pr: #32076
This PR add two rest api for component stop and status check:
1. `/management/stop?role=querynode` can stop the specified component
2. `/management/check/ready?role=rootcoord` can check whether the target
component is serviceable

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-17 21:53:59 +08:00