1645 Commits

Author SHA1 Message Date
Patrick Weizhi Xu
3c7f73137e
fix: disallow expr when partition key isolation is enabled (#35031)
issue: #34336 

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 2889481ce9a14230e9c7f1c8f9c3c1decde77e03)
2024-07-29 14:21:50 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394)
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-29 11:31:51 +08:00
congqixia
783f9d9c33
fix: Unify hook singleton implementation in proxy (#34887)
Related to #34885

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-26 18:07:53 +08:00
congqixia
94d8f48b85
enhance: Use auto detection proto interface to fetch grpc size (#35024)
Related to #34763

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-26 14:23:47 +08:00
congqixia
4ee6c69217
enhance: Add Segment Level in milvus segment info APIs (#34763)
See also #34746

This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-26 10:01:46 +08:00
chyezh
d93c51d5f6
enhance: change GetVChannels rpc into GetPChannelInfo (#34940)
issue: #33285

Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-25 13:29:45 +08:00
congqixia
e2f40fc2a8
fix: Check legacy guarantee ts when skipping alloc ts (#34981)
See also #34980

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-25 10:17:45 +08:00
Jiquan Long
a2ac84bd64
feat: record the duration waiting in the proxy queue (#34744)
fix: https://github.com/milvus-io/milvus/issues/34743

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-07-23 14:23:52 +08:00
yihao.dai
8aab6cbfac
enhance: Organize the common modules of streamingNode and dataNode (#34773)
1. Move the common modules of streamingNode and dataNode to flushcommon
2. Add new GetVChannels interface for rootcoord

issue: https://github.com/milvus-io/milvus/issues/33285

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-22 11:33:51 +08:00
SimFG
88c38a3756
fix: execute method should return an error when result is a failure (#34813)
- issue: #34812

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-20 23:13:40 +08:00
zhagnlu
fbad77f22a
fix: fix ut test fail (#34832)
#34831

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-19 17:51:40 +08:00
congqixia
e4e18cb8c3
enhance: Add dynamic cgo pool for proxy CGO call (#34768)
Related to #34705

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-19 15:11:41 +08:00
yihao.dai
c8bf6c8a46
enhance: Optimize ConsistencyLevel and GuaranteeTimestamp log (#34738)
/kind improvement

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-18 15:17:41 +08:00
zhagnlu
f1b2f7b640
enhance: refactor bitmap index and internal hybrid index (#34450)
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-18 10:39:42 +08:00
balloon1995
7306d2d115
fix: fix metaCache cleanup issue when listPolicy failed (#34449)
issue: #34667

---------

Signed-off-by: balloon1995 <hszoe1995@outlook.com>
Co-authored-by: congqixia <congqi.xia@zilliz.com>
2024-07-16 10:03:38 +08:00
congqixia
b5225df21c
enhance: Use unified requery flag when set metrics (#34632)
See also #34601

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-15 23:03:38 +08:00
SimFG
d8e68cbb97
enhance: use the insert rate limit for the upsert request (#34615)
- issue: #34614

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-12 16:17:35 +08:00
aoiasd
5bb0d21e32
fix: Restful API use deprecate error code cause access log panic. (#34576)
relate: https://github.com/milvus-io/milvus/issues/34578

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-07-12 10:13:35 +08:00
Patrick Weizhi Xu
104d0966b7
feat: support partition key isolation (#34336)
issue: #34332

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-07-11 19:01:35 +08:00
smellthemoon
07b94b4615
enhance: support upsert autoid==true (#30342)
related with: #29258

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-07-11 16:53:35 +08:00
Gao
ae6d6f91e6
enhance: change autoindex default metric type (#34261)
issue: #34304 
cosine is more widely used in float vectors, and cosine and hamming
distance are 'metrics' which have good geometric properties

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-07-08 19:52:24 +08:00
jaime
60be454db0
enhance: add disk quota and max collections into db properties (#34368)
issue: #34385

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-05 18:22:17 +08:00
aoiasd
233b8486ec
fix: generate gin logs failed cause access logs panic (#34142)
relate: https://github.com/milvus-io/milvus/issues/34141

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-07-05 16:30:10 +08:00
zhagnlu
18f2458385
fix: fix bitmap supported type (#34350)
#34314

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-05 15:50:10 +08:00
jaime
21fc5f5d46
enhance: Remove datanode reporting TT based on MQ implementation (#34421)
issue: #34420

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-05 15:48:09 +08:00
PowderLi
c16a704934
fix: [restful v2] count(*) & hook (#34369)
issue:  #31224 #34374

for query api:
1. param filter is not requried
2. param limit is useless while outputFields = [count(*)]

add hook about grpc call

---------

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-07-04 19:38:09 +08:00
Chun Han
10b3ce24ba
enhance: use configed max topk for iterator when input topk exceeds (#34290)
related: #34290

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-07-04 14:24:19 +08:00
Buqian Zheng
fa8d641ce6
fix: update check for sparse hnsw index (#33713)
issue: #29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-07-02 21:56:09 +08:00
congqixia
b32dae1883
enhance: Refine max length exceeded error message (#34300)
This PR make varchar & string array field max length exceeded error
message clearer. Also fixed a minor issue that error string format and
argument number not match.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-01 21:16:07 +08:00
wei liu
b49862d4f3
enhance: Optimize grow slice cost during query (#34253)
issue: #32252 

This PR try to pre-allocate FieldData for Reduce operations in the Query
chain using typeutil.PrepareResultFieldData to avoid the overhead of
dynamically growing the slice during appendFieldData process.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-01 15:18:11 +08:00
aoiasd
186757e622
enhance: support mark error as user error (#33498)
relate: https://github.com/milvus-io/milvus/issues/33492

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-07-01 14:56:12 +08:00
Chun Han
96dcee5dff
fix:load major compaction partial result(#34051) (#34052)
related: #34051

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-06-28 14:04:05 +08:00
jaime
9c2eeff427
enhance: refine error message for invalid limit parameter (#34178)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-06-26 22:14:04 +08:00
cqy123456
51ebe95be5
enhance: remove CheckVecIndexWithDataTypeExist function in pkg and remove some cgo call (#34102)
issue: https://github.com/milvus-io/milvus/issues/22837
related pr: https://github.com/milvus-io/milvus/pull/34104

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-06-26 21:13:44 +08:00
jaime
9630974fbb
enhance: move rocksmq from internal to pkg module (#33881)
issue: #33956

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-06-25 21:18:15 +08:00
wayblink
f9a0f7bb25
Add an option to enable/disable vector field clustering key (#34097)
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-06-25 18:52:04 +08:00
congqixia
fd922d921a
enhance: Add nilness linter and fix some small issues (#34049)
Add `nilness` for govet linter and fixed some detected issues

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-24 14:52:03 +08:00
jaime
d08cb885ca
enhance: enable flush rate limiter of collection level (#33837)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-06-24 10:52:03 +08:00
Patrick Weizhi Xu
b961767005
enhance: support integral type for MV and skip MV if there is only one category (#33161)
issue: #29892

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-06-24 10:20:01 +08:00
cqy123456
298e50b834
enhance: check index with data type (#33880)
issue: https://github.com/milvus-io/milvus/issues/22837
related: https://github.com/milvus-io/milvus/pull/33878

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-06-19 10:15:59 +08:00
Gao
0d20303e54
fix: fix binary vector data size (#33750)
issue: https://github.com/milvus-io/milvus/issues/22837

- fix byte size wrong for binary vectors
- fix the expect/actual error msg

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-06-18 21:39:59 +08:00
Gao
a789c60380
enhance: autoindex for multi data type (#33868)
issue: #22837 

contain https://github.com/milvus-io/milvus/pull/33625
https://github.com/milvus-io/milvus/pull/33867
https://github.com/milvus-io/milvus/pull/33911 which already merged to
2.4 branch

Signed-off-by: chasingegg <chao.gao@zilliz.com>
Co-authored-by: foxspy <xianliang.li@zilliz.com>
2024-06-18 21:34:01 +08:00
wei liu
188ee433b9
enhance: Refine name rule check error msg (#33815)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-06-17 10:37:58 +08:00
wayblink
a1232fafda
feat: Major compaction (#33620)
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
2024-06-10 21:34:08 +08:00
yihao.dai
3540eee977
enhance: Support L0 import (#33514)
issue: https://github.com/milvus-io/milvus/issues/33157

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-06-07 14:17:20 +08:00
wei liu
0a3d456688
enhance: add restful api to trigger component stop (#32076)
issue: #32698
This PR add two rest api for component stop and status check:
1. `/management/stop?role=querynode` can stop the specified component
2. `/management/check/ready?role=rootcoord` can check whether the target
component is serviceable

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-06-07 10:35:54 +08:00
aoiasd
b9bafc76b4
enhance: access log support print with consistency level (#33503)
relate: https://github.com/milvus-io/milvus/issues/33563

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-06-05 17:13:50 +08:00
congqixia
25080bb455
enhance: Print UseDefaultConsistency param in read requests (#33617)
`UseDefaultConsistency` param is crucial for debugging slow query
problems. It could be confusing when guarantee timestamp is 1 while this
param is not logged

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-05 10:27:50 +08:00
aoiasd
0cf225fa75
enhance: access log support restful api (#33155)
relate: https://github.com/milvus-io/milvus/issues/31823

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-31 11:13:44 +08:00
Chun Han
416a2cf507
fix: query iterator lack results(#33137) (#33422)
related: #33137 
adding has_more_result_tag for various level's reduce to rectify
reduce_stop_for_best

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-05-30 17:51:44 +08:00