65 Commits

Author SHA1 Message Date
wei liu
5a4bafccc6
enhance: Add resource group api on Restful (#38885)
issue: #38739
enable to manage resource group by Restful
- CreateResourceGroup
- DescribeResourceGroup
- UpdateResourceGroup
- ListResourceGroup
- DropResourceGroup
- TransferReplica

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-01-08 19:40:57 +08:00
smellthemoon
a89ac861da
fix: not add index name in alter index properties (#38979)
#38967

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-01-08 15:02:55 +08:00
smellthemoon
6b105837b4
enhance: add some apis in Restful (#38733)
add some apis in Restful #38709 

- alter/drop collection properties
- alter/drop index properties
- alter collection field properties
- refresh load

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-31 11:28:51 +08:00
smellthemoon
134d06f7e6
fix: need pass parameter to listPrivilegeGroups (#38562)
issue: https://github.com/milvus-io/milvus/issues/38563

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-18 20:28:51 +08:00
Gao
994fc544e7
enhance: support iterative filter execution (#37363)
issue: #37360

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-12-11 11:32:44 +08:00
smellthemoon
0d4073bdd8
enhance: use require dbname for some db req (#38267)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-06 16:26:46 +08:00
aoiasd
c91bb5a655
enhance: report restful info to hook (#38112)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-12-05 19:02:41 +08:00
smellthemoon
e359725530
enhance: support db request in Restful api (#38140)
#38077

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-03 20:04:41 +08:00
smellthemoon
db453c063d
enhance: no need database check in restful sdk (#38078)
#38077 
remove the check for two reason
1. server will do the same to make sure use the correct database;
2. each req has an additional overhead of calling the proxy to check
database.

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-02 17:04:38 +08:00
smellthemoon
cb926688f7
enhance: support templates for expression in Restful api (#38040)
#36672

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-02 16:54:38 +08:00
sthuang
bd0811434b
fix: fix grant v2 service (#37945)
issue: https://github.com/milvus-io/milvus/issues/37031

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-22 19:42:32 +08:00
sthuang
19572f5b06
enhance: RBAC new grant/revoke privilege (#37785)
issue: https://github.com/milvus-io/milvus/issues/37031
also fix issues: https://github.com/milvus-io/milvus/issues/37843,
https://github.com/milvus-io/milvus/issues/37842,
https://github.com/milvus-io/milvus/issues/37887

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-21 22:20:34 +08:00
congqixia
83c902992c
enhance: Prevent generate "null" search params (#37812)
Preventing generating null search params in restful search request

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-19 19:54:32 +08:00
yihao.dai
b3fc53015d
fix: Avoid memory copy in RESTful search and query (#37674)
Custom `jsonRender` that encodes JSON data directly into the response
stream, it uses less memory since it does not buffer the entire JSON
structure before sending it, unlike `c.JSON` in `HTTPReturn`, which
serializes the JSON fully in memory before writing it to the response.
Benchmark testing shows that, using the custom render incurs no
performance loss and reduces memory consumption by nearly 50%.

issue: https://github.com/milvus-io/milvus/issues/37671

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-19 17:24:31 +08:00
congqixia
b0bd290a6e
enhance: Use internal json(sonic) to replace std json lib (#37708)
Related to #35020

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-18 10:46:31 +08:00
smellthemoon
a654487995
fix: some error in restful (#37487)
#37370

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-13 17:12:39 +08:00
Chun Han
2d29dcd30c
enhance:refine group_strict_size parameter(#37482) (#37483)
related: #37482

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-11-12 09:56:28 +08:00
aoiasd
12951f0abb
enhance: rename tokenizer to analyzer and check analyzer params (#37478)
relate: https://github.com/milvus-io/milvus/issues/35853

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-10 16:12:26 +08:00
sthuang
ff00a12805
enhance: RBAC custom privilege group ut coverage (#37558)
issue: https://github.com/milvus-io/milvus/issues/37031

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-09 20:40:25 +08:00
sthuang
70605cf5b3
enhance: Support custom privilege group for RBAC (#37087)
issue: #37031

---------

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-09 08:44:28 +08:00
yihao.dai
994f52fab8
fix: Revert "enhance: Support db for bulkinsert (#37012)" (#37420)
This reverts commit 6e90f9e8d90440716d596a7fe8fe1db465d529b7.

issue: https://github.com/milvus-io/milvus/issues/31273

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-07 17:02:25 +08:00
smellthemoon
86fd3200be
enhance: refactor createIndex in RESTful API (#37235)
Make the parameter input method consistent with miluvs-client.

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-07 14:18:30 +08:00
congqixia
9a9de3df5c
enhance: Pass rpc stats via gin.Context (#37439)
Related #37223

RPC stats worked in middleware but faild to get method & collection info

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-05 18:26:23 +08:00
congqixia
0f59bfdf30
enhance: Use middleware to observe restful v2 in/out rpc stats (#37223)
Related to #36102

Previous PR #36107 add grpc inteceptor to observe rpc stats. Using same
strategy, this pr add gin middleware to observer restful v2 rpc stats.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-29 11:22:24 +08:00
smellthemoon
84d48b498b
enhance: support upsert autoid==true in Restful API (#37072)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-25 14:33:39 +08:00
yihao.dai
6e90f9e8d9
enhance: Support db for bulkinsert (#37012)
issue: https://github.com/milvus-io/milvus/issues/31273

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-25 14:31:39 +08:00
Chun Han
e2f2fd55a5
enhance: avoid limiting ddl operations repeatedly(#37006) (#37010)
related: #37006

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-10-22 20:11:27 +08:00
smellthemoon
8669153ff2
fix: not add isClusteringKey when describe collection (#36930)
#36928

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-18 12:57:26 +08:00
Chun Han
eccc326e8b
enhance: report err when group_size is wrong(#36146) (#36908)
related: #36146

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-10-17 16:05:29 +08:00
smellthemoon
44d80c1355
fix: not return err if consistencyLevel is not set to a valid value (#36714)
https://github.com/milvus-io/milvus/issues/36444

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-16 13:23:22 +08:00
smellthemoon
c9752bd2e6
enhance: refactor createCollection in RESTful API (#36790)
1.  support isClusteringKey in restful api;
2. throw err if passed invalid 'enableDynamicField' params
3. parameters in indexparams are not processed properly, related with
#36365

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-15 10:29:22 +08:00
Buqian Zheng
16b533cbf0
feat: Restful support for BM25 function (#36713)
issue: https://github.com/milvus-io/milvus/issues/35853

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-13 17:41:21 +08:00
smellthemoon
5e1c392243
enhance: support null and default value in restful api (#35825)
#31728

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-11 21:23:20 +08:00
yihao.dai
0751c508de
enhance: Support new RESTful URL to describe import job (#36753)
Support the new RESTful URL for retrieving/describing import progress:
`/v2/vectordb/jobs/import/describe`.
Deprecate the old URL: `/v2/vectordb/jobs/import/get_progress`.

issue: https://github.com/milvus-io/milvus/issues/36752

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-11 16:07:21 +08:00
Chun Han
a25dc98794
enhance: support group_size and hybridsearch+groupby on httpv2 side(#36386) (#36461)
related: #36386

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-09-30 11:09:16 +08:00
smellthemoon
b1ac3f0df0
enhance: do not log out the full req (#36546)
![img_v3_02f3_61ac166c-df6f-4b5e-b06a-756e5f40a65g](https://github.com/user-attachments/assets/df668339-d0d8-4295-a2b6-52d89ad98766)

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-27 16:39:15 +08:00
smellthemoon
6e880d19a8
enhance: add search params in search request in restful (#36304)
https://github.com/milvus-io/milvus/issues/36321

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-23 16:51:12 +08:00
Chun Han
031ee6f155
enhance: support httpv1/v2 throttle and add it for httpV2(#35350) (#35470)
related: #35350

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-08-20 16:16:55 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394)
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-29 11:31:51 +08:00
Patrick Weizhi Xu
80c0ae3519
feat: [restful v2] add partition key isolation prop (#34710)
issue: https://github.com/milvus-io/milvus/issues/34332

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-07-22 11:29:59 +08:00
PowderLi
c16a704934
fix: [restful v2] count(*) & hook (#34369)
issue:  #31224 #34374

for query api:
1. param filter is not requried
2. param limit is useless while outputFields = [count(*)]

add hook about grpc call

---------

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-07-04 19:38:09 +08:00
aoiasd
0cf225fa75
enhance: access log support restful api (#33155)
relate: https://github.com/milvus-io/milvus/issues/31823

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-31 11:13:44 +08:00
PowderLi
ee73e62202
fix: [restful v2] search result be cut while nq > 1 (#33339)
issue: #33099 #32837 #32419

1. len(search result) may be nq * topk, we need return all rather than
topk
2. the  in restful response payload keep the same with milvus error code

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-05-27 14:25:41 +08:00
Buqian Zheng
1b67cecd65
enhance: add sparse float vector support to restful v2 (#33231)
issue: #29419
also re-enabled an e2e test using restful api, which is previously
disabled due to https://github.com/milvus-io/milvus/issues/32214.

In restful api, the accepted json formats of sparse float vector are:

* `{"indices": [1, 100, 1000], "values": [0.1, 0.2, 0.3]}`
* {"1": 0.1, "100": 0.2, "1000": 0.3}

for accepted indice and value range, see
https://milvus.io/docs/sparse_vector.md#FAQ

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-05-27 00:47:40 +08:00
PowderLi
b9d7145049
fix: [restful v2]role operations need dbName (#33283)
issue: #33220

use dbName as part of privilege entity, so
1. grant / revoke a privilege need dbName
2. we can describe the privileges of the role which belong to one
special database

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-05-23 09:51:45 +08:00
SimFG
b9b6343c88
enhance: check the auth in some rest v2 api (#33256)
/kind improvement
link master proto:
https://github.com/milvus-io/milvus-proto/blob/master/proto/milvus.proto

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-22 16:03:40 +08:00
congqixia
c0e62e6629
enhance: Use collection default consistency level for restv2 (#32956)
Set `UseDefaultConsistency` to true so that restv2 read API shall use
collection consistency level setting correctly.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-11 00:47:31 +08:00
SimFG
8963615b33
enhance: add the cost response for the rest api (#32620)
issue: https://github.com/milvus-io/milvus/issues/30436

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-08 11:53:29 +08:00
PowderLi
29922829da
fix: [restful v2] parameters of create collection (#32485)
issue: #31176

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-04-24 09:43:29 +08:00
PowderLi
a0a23d0666
fix: [restful]use default search parameter nq: 0 (#32355)
issue: #32225 #31978 #32360

1. v1 can only accept one vector, but v2 accept list of vectors #32225
2. cannot get dbName from AliasReq #31978
3. enhance: [restful v2]support config DB-Name in the http header #32360

---------

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-04-18 14:41:20 +08:00