Gao
d7e926f1cb
enhance: [2.4] support recall estimation ( #38064 )
...
issue: #37899
pr: #38017
---------
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-12-11 20:40:48 +08:00
Xianhui Lin
a51647569b
enhance: [2.4]alterindex & altercollection supports altering properties ( #38111 )
...
enhance :
alterindex delete properties
We have introduced a new parameter deleteKeys to the alterindex
functionality, which allows for the deletion of properties within an
index. This enhancement provides users with the flexibility to manage
index properties more effectively by removing specific keys as needed.
altercollection delete properties
We have introduced a new parameter deleteKeys to the altercollection
functionality, which allows for the deletion of properties within an
collection. This enhancement provides users with the flexibility to
manage collection properties more effectively by removing specific keys
as needed.
3.support altercollectionfield
We currently support modifying the fieldparams of a field in a
collection using altercollectionfield, which only allows changes to the
max-length attribute.
Key Points:
New Parameter - deleteKeys: This new parameter enables the deletion of
specified properties from an index. By passing a list of keys to
deleteKeys, users can remove the corresponding properties from the
index.
Mutual Exclusivity: The deleteKeys parameter cannot be used in
conjunction with the extraParams parameter. Users must choose one
parameter to pass based on their requirement. If deleteKeys is provided,
it indicates an intent to delete properties; if extraParams is provided,
it signifies the addition or update of properties.
issue: https://github.com/milvus-io/milvus/issues/37436
pr: https://github.com/milvus-io/milvus/pull/37437
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-06 14:50:41 +08:00
wei liu
8646bba134
fix: grant ManualCompact api doesn't work ( #38096 ) ( #38168 )
...
issue: #38086
pr: #38096
cause ManualCompact api pass collection id in request, but RBAC requires
to check collection name, so grant ManualCompact api doesn't work.
This PR refine the ManualCompact api to accpet collection name in
request.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-05 16:00:42 +08:00
sthuang
febed0abb7
enhance: [2.4] add list aliases privilege into public role and fix typo ( #38208 )
...
cherry-pick from master: https://github.com/milvus-io/milvus/pull/38176 ,
https://github.com/milvus-io/milvus/pull/38195
related issue: https://github.com/milvus-io/milvus/issues/37031
---------
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-04 17:58:40 +08:00
sthuang
66f2dac5f5
fix: [2.4] fix grant/revoke v2 meta and unclear error messages ( #38146 )
...
cherry-pick from https://github.com/milvus-io/milvus/pull/38110 ,
https://github.com/milvus-io/milvus/pull/38130
related issue: https://github.com/milvus-io/milvus/issues/37031
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-03 22:06:41 +08:00
Gao
165afbba91
enhance: support retry search when topk is reduced and result not enough ( #37093 )
...
issue: #35576
pr: #35645
---------
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-11-28 10:12:37 +08:00
sthuang
d8f1af68e9
enhance: [2.4] RBAC built in privilege groups and grant v2 ( #37787 )
...
cherry-pick from master: https://github.com/milvus-io/milvus/pull/37720 ,
https://github.com/milvus-io/milvus/pull/37785
issue: https://github.com/milvus-io/milvus/issues/37031
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-25 11:24:54 +08:00
congqixia
8801322371
enhance: [2.4] Invalidate collection cache when release collection ( #37577 ) ( #37628 )
...
Cherry-pick from master
pr: #37577
Related to #37395
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-13 14:00:31 +08:00
sthuang
9e8b6ace6d
enhance: [2.4] RBAC custom privilege group ( #37560 )
...
Cherry-pick from master
pr: https://github.com/milvus-io/milvus/pull/37087 ,
https://github.com/milvus-io/milvus/pull/37558
issue: #37031
---------
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-11 14:20:29 +08:00
yihao.dai
380662153f
fix: [2.4] Revert "enhance: Support db for bulkinsert ( #37012 ) ( #37017 )" ( #37421 )
...
This reverts commit d6adc62765665d1555039c4d256a75d1144d49d0.
issue: https://github.com/milvus-io/milvus/issues/31273
pr: https://github.com/milvus-io/milvus/pull/37420
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-05 10:48:24 +08:00
yihao.dai
ca2057c57d
enhance: Tidy import options ( #37077 ) ( #37078 )
...
1. Tidy import options.
2. Tidy common import util functions.
issue: https://github.com/milvus-io/milvus/issues/34150
pr: https://github.com/milvus-io/milvus/pull/37077
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-25 14:35:45 +08:00
yihao.dai
d6adc62765
enhance: Support db for bulkinsert ( #37012 ) ( #37017 )
...
issue: https://github.com/milvus-io/milvus/issues/31273
pr: https://github.com/milvus-io/milvus/pull/37012
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-23 16:31:29 +08:00
XuanYang-cn
6b68ec55e1
enhance: [24]Track complex delete rates ( #36958 )
...
See also: #36953
pr: #36957
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-22 10:07:27 +08:00
congqixia
385bfc2639
enhance: [2.4] Unify InboundLabel case using metrics.Label ( #36613 ) ( #36616 )
...
Cherry pick from master
pr: #36613
Previous label case broken by #36107 , this PR make all inbound label
using label constants from metrics package.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-30 16:19:17 +08:00
congqixia
1cd8d1bd80
enhance: [2.4] Use stats Handler to record request/response size metrics ( #36107 ) ( #36118 )
...
Cherry-pick from master
pr: #36107
Related to #36102
This PR use newly added `grpcSizeStatsHandler` to reduce calling
`proto.Size` since the request & response size info is recorded by grpc
framework.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-10 17:13:08 +08:00
jaime
2c1fa50412
enhance: remove cooling off in rate limiter for read requests ( #35936 )
...
issue: #35934
pr: #35935
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-09-04 14:39:10 +08:00
SimFG
fc324b4254
feat: [2.4] add the rbac msg and send them to the replicate channel ( #35562 )
...
- issue: #35391
- pr: #35392
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-27 14:45:00 +08:00
congqixia
ab261d0f8b
feat: [2.4] Support field partial load collection ( #35416 ) ( #35696 )
...
Cherry-pick from master
pr: #35416
Related to #35415
---------
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-27 14:07:00 +08:00
wei liu
e2542a1bf5
enhance: Update protobuf-go to protobuf-go v2 ( #34394 ) ( #35555 )
...
issue: #34252
pr: #34394 #35072 #35084
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-21 18:50:58 +08:00
wei liu
fc344d1eae
fix: Fix proxy panic after Import related api failed ( #35540 ) ( #35559 )
...
issue: #35539
pr: #35540
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-19 19:42:55 +08:00
wei liu
248a6ea401
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup ( #35444 ) ( #35513 )
...
issue: #35443
pr: #35444
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-18 13:42:55 +08:00
aoiasd
a20cb727eb
enhance:[Cherry-pick] Check by proxy rate limiter when delete get data by query. ( #30891 ) ( #35262 )
...
relate: https://github.com/milvus-io/milvus/issues/30927
pr: https://github.com/milvus-io/milvus/pull/30891
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-08-13 14:32:21 +08:00
congqixia
d16320705e
enhance: [2.4] Add Segment Level in milvus segment info APIs ( #34763 ) ( #35023 )
...
Cherry-pick from master
pr: #34763
See also #34746
This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`
---------
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-29 10:11:52 +08:00
congqixia
079276c6ff
fix: [2.4] Unify hook singleton implementation in proxy ( #34888 )
...
Cherry-pick from master
pr: #34887
Related to #34885
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-26 18:07:53 +08:00
SimFG
4e95f3835a
fix: [2.4] execute method should return an error when result is a failure ( #34872 )
...
- issue: #34812
- pr: #34813
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-22 12:13:42 +08:00
yihao.dai
ffee6b9c27
enhance: Optimize ConsistencyLevel and GuaranteeTimestamp log ( #34738 ) ( #34824 )
...
/kind improvement
pr: https://github.com/milvus-io/milvus/pull/34738
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-22 11:32:04 +08:00
Patrick Weizhi Xu
3cf2e1831a
feat: [cherry-pick][2.4] support partition key isolation ( #34336 ) ( #34613 )
...
issue: #34332
pr: #34336
---------
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 104d0966b7df80a251437bed74f28fa47e138397)
2024-07-17 14:03:43 +08:00
aoiasd
9087b6f42e
enhance: [Cherry-Pick] support mark error as user error ( #33498 ) ( #34396 )
...
relate: https://github.com/milvus-io/milvus/issues/33492
pr: https://github.com/milvus-io/milvus/pull/33498
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-07-04 10:08:10 +08:00
SimFG
b3c5eb29ed
enhance: [2.4] the proxy metric in the query request ( #34356 )
...
/kind improvement
- issue: #33306
- pr: #33307
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-03 10:00:09 +08:00
wayblink
c62bf8a0b0
fix: [Cherry-pick]Pick major compaction fixs and optimizations ( #34360 )
...
This PR cherry-picks the following commits:
- fix: sync partitiion stats blocking balance task #33742
- fix: Fix meta prefix overlap bug #33830
- fix: Small fixs of major compaction #33929
- fix: Fix memory buffer error & some renaming #33850
- fix: sync part stats task cannot be finished #34027
- Add an option to enable/disable vector field clustering key #34097
- fix: fix error ignore in compactor #34169
- fix:load major compaction partial result #34052
- Use new stream segment reader in clustering compaction #34232
issue: #30633
pr: #33742 #33830 #33929 #33850 #34027 #34097 #34169 #34052 #34232
---------
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: Chun Han <116052805+MrPresent-Han@users.noreply.github.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-07-03 09:53:37 +08:00
yihao.dai
ed1dee9e38
enhance: Support L0 import ( #33514 ) ( #33712 )
...
issue: https://github.com/milvus-io/milvus/issues/33157
pr: https://github.com/milvus-io/milvus/pull/33514
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-06-08 11:17:52 +08:00
congqixia
7203ff8898
enhance: [2.4] Print UseDefaultConsistency param in read requests ( #33617 ) ( #33644 )
...
Cherry-pick from master
pr: #33617
`UseDefaultConsistency` param is crucial for debugging slow query
problems. It could be confusing when guarantee timestamp is 1 while this
param is not logged
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-06 14:19:56 +08:00
yihao.dai
ad4c1975bd
fix: Fix filtering by partition key fails for importing data ( #33274 ) ( #33277 )
...
Before executing the import, partition IDs should be reordered according
to partition names. Otherwise, the data might be hashed to the wrong
partition during import. This PR corrects this error.
issue: https://github.com/milvus-io/milvus/issues/33237
pr: https://github.com/milvus-io/milvus/pull/33274
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-23 11:25:40 +08:00
yihao.dai
6fc00e42d7
fix: Fix binlog import ( #33027 )
...
binlog import should only import into one partition.
issue: https://github.com/milvus-io/milvus/issues/32807
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-14 21:45:32 +08:00
wei liu
c35797c399
enhance: expose DescribeDatabase api in proxy ( #32732 )
...
issue: #32707
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-09 22:51:30 +08:00
wei liu
ba02d54a30
enhance: update shard leader cache when leader location changed ( #32470 )
...
issue: #32466
this PR enhance that when shard location changed, update proxy's shard
leader cache. in case of query node failover case, proxy can find
replica recover
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-08 10:05:29 +08:00
yihao.dai
53874ce245
fix: Fix cannot specify partition name in binlog import ( #32730 )
...
issue: https://github.com/milvus-io/milvus/issues/32807
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-07 17:19:30 +08:00
Bingyi Sun
fecd9c21ba
feat: LRU cache implementation ( #32567 )
...
issue: https://github.com/milvus-io/milvus/issues/32783
This pr is the implementation of lru cache on branch lru-dev.
Signed-off-by: sunby <sunbingyi1992@gmail.com>
Co-authored-by: chyezh <chyezh@outlook.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
Co-authored-by: Ted Xu <ted.xu@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
2024-05-06 20:29:30 +08:00
wei liu
07720f1a95
enhance: expose alter database api in proxy ( #32639 )
...
issue: #30040
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-28 18:15:31 +08:00
jaime
3b5cbed86a
enhance: add db label for deletion and bulkinsert operation ( #32611 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-04-26 15:53:29 +08:00
SimFG
8594b55ad5
enhance: add max insert request size and must use partition key configs ( #32433 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-19 10:31:20 +08:00
SimFG
31a29a2451
enhance: avoid the repeated metric info in the proxy ( #32380 )
...
issue: #30577
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-19 10:21:20 +08:00
chyezh
48fe977a9d
enhance: declarative resource group api ( #31930 )
...
issue: #30647
- Add declarative resource group api
- Add config for resource group management
- Resource group recovery enhancement
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-15 08:13:19 +08:00
SimFG
c012e6786f
feat: support rate limiter based on db and partition levels ( #31070 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
co-author: @jaime0815
---------
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-04-12 16:01:19 +08:00
zhenshan.cao
0bfe130991
fix: fix metric about ProxySearchVectors ( #32152 )
...
issue :https://github.com/milvus-io/milvus/issues/25639
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-04-11 16:13:17 +08:00
SimFG
90bed1caf9
enhance: add the related data size for the read apis ( #31816 )
...
issue: #30436
origin pr: #30438
related pr: #31772
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-10 15:07:17 +08:00
SimFG
a63cf8625c
enhance: support to invalid the database meta cache ( #32078 )
...
issue: #32077
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-10 06:45:18 +08:00
zhenshan.cao
089c805e0a
enhance:Refactor hybrid search ( #32020 )
...
issue: https://github.com/milvus-io/milvus/issues/25639
https://github.com/milvus-io/milvus/issues/31368
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-04-09 14:21:18 +08:00
congqixia
7e0d03420e
enhance: Remove duplicated err check code ( #32001 )
...
The `err != nil` check is duplicated here and shall be removed
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-08 21:59:17 +08:00
jaime
bd853be8c7
enhance: Add db label for some usual metrics ( #30956 )
...
issue: #31782
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-04-02 14:27:13 +08:00
