admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
20,266 Commits 137 Branches 188 Tags
Commit Graph

1487 Commits

Author SHA1 Message Date
congqixia
555b7a6aa7
enhance: [2.4] Remove duplicated schema helper creation in proxy (#35489) (#35502) 
Cherry-pick from master
pr: #35489
Related to PRs of #35415

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-16 14:32:54 +08:00
SimFG
4fae5407dd
enhance: [2.4] remove the invalid log (#35473) 
/kind improvement
pr: #35465

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-15 14:14:53 +08:00
aoiasd
a20cb727eb
enhance:[Cherry-pick] Check by proxy rate limiter when delete get data by query. (#30891) (#35262) 
relate: https://github.com/milvus-io/milvus/issues/30927
pr: https://github.com/milvus-io/milvus/pull/30891

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-08-13 14:32:21 +08:00
smellthemoon
23052f6ac2
fix: upsert use the previous pk in insert when autoid(#34672) (#35130) 
issue: #34668 
pr: #34672

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-08-12 18:58:25 +08:00
wei liu
a4fa9ced16
fix: delete by expr failed at retry progress (#35241) (#35268) 
issue: #35240
pr: #35241
delete by expr shard the same err object between channels, so if one
channel's query failed, it will fail all channel, which will break
channel level retry policy, and make delete operation failed.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-09 16:12:18 +08:00
zhagnlu
db9e4b898a
enhance: support bitmap index (#35336) 
pr: #32902
cherry-pick bitmap index from master

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-07 20:12:18 +08:00
Jiquan Long
b11db52160
fix: error message when input is not aligned (#35322) (#35323) 
fix: https://github.com/milvus-io/milvus/issues/35321
pr: https://github.com/milvus-io/milvus/pull/35322

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-08-07 09:56:28 +08:00
congqixia
2a1fc700a2
enhance: [2.4] Add in-memory cache for casbin enforcer result (#35272) 
Cherry-pick from master
pr: #35271
See also #35270

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-06 12:14:17 +08:00
congqixia
f8444b900f
enhance: [2.4] Support proxy/delegator qn client pooling (#35195) 
Cherry pick from master
pr: #35194
See also #35196
Add param item for proxy/delegator query node client pooling and
implement pooling logic

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-02 11:24:19 +08:00
Jiquan Long
86edca8c1b
fix: support auto index for array (#35095) 
/kind branch-feature
pr: #34450

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: Zhagnlu <lu.zhang@zilliz.com>
 2024-07-30 17:57:50 +08:00
Patrick Weizhi Xu
e2fc8853b6
fix: [2.4] disallow expr when partition key isolation is enabled (#35025) 
issue: #34336 
pr: #35031

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-07-29 14:23:49 +08:00
congqixia
d16320705e
enhance: [2.4] Add Segment Level in milvus segment info APIs (#34763) (#35023) 
Cherry-pick from master
pr: #34763
See also #34746

This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-29 10:11:52 +08:00
congqixia
079276c6ff
fix: [2.4] Unify hook singleton implementation in proxy (#34888) 
Cherry-pick from master
pr: #34887
Related to #34885

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-26 18:07:53 +08:00
congqixia
0dd9af8f13
fix: [2.4] Check legacy guarantee ts when skipping alloc ts (#34982) 
Cherry-pick from master
pr: #34981
See also #34980

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-25 10:27:47 +08:00
SimFG
4e95f3835a
fix: [2.4] execute method should return an error when result is a failure (#34872) 
- issue: #34812
- pr: #34813

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-22 12:13:42 +08:00
yihao.dai
ffee6b9c27
enhance: Optimize ConsistencyLevel and GuaranteeTimestamp log (#34738) (#34824) 
/kind improvement

pr: https://github.com/milvus-io/milvus/pull/34738

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-22 11:32:04 +08:00
congqixia
d1e0d4ea4a
enhance: [2.4] Add dynamic cgo pool for proxy CGO call (#34768) (#34842) 
Cherry-pick from master
pr: #34768
Related to #34705

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-21 18:05:41 +08:00
Jiquan Long
34873a7f76
feat: record the duration waiting in the proxy queue (#34744) (#34788) 
fix: https://github.com/milvus-io/milvus/issues/34743
pr: #34744

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-07-19 15:37:44 +08:00
Patrick Weizhi Xu
3cf2e1831a
feat: [cherry-pick][2.4] support partition key isolation (#34336) (#34613) 
issue: #34332
pr: #34336

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 104d0966b7df80a251437bed74f28fa47e138397)
 2024-07-17 14:03:43 +08:00
SimFG
9cf1f03e89
fix: [2.4] the lint-fix error (#34717) 
- issue: #31293
- pr: #34033
- related pr: #34699

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-16 17:37:39 +08:00
SimFG
d607a43ea6
fix: [2.4] metaCache cleanup issue when listPolicy failed (#34697) 
- issue:#34667
- pr: #34449

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-16 14:13:45 +08:00
smellthemoon
0fdb288de7
enhance: upsert support autoid(#30342) (#34633) 
pr: #30342 
issue: #29258

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
 2024-07-15 20:53:39 +08:00
SimFG
9bcab2ebee
enhance: [2.4] use the insert rate limit for the upsert request (#34616) 
- issue: #34614
- pr: #34615

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-12 16:19:36 +08:00
aoiasd
d6fc6a9f41
fix: [Cherry-pick] Restful API use deprecate error code cause access log panic. (#34579) 
Avoid panic when use deprecate error code and use merr code replace
deprecate common code.
relate: https://github.com/milvus-io/milvus/issues/34578
pr： https://github.com/milvus-io/milvus/pull/34576

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-07-11 11:58:51 +08:00
Gao
f6cd84161c
enhance: ensure autoindex default metric type compatibility (#34479) 
issue: #34304 
pr: #34261

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-07-08 19:48:14 +08:00
jaime
326370c1be
enhance: add disk quota and max collections into db properties (#34386) 
issue: https://github.com/milvus-io/milvus/issues/34385
pr: #34368

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-05 18:22:17 +08:00
PowderLi
ba2c232331
fix: [cherry-pick] [restful v2] count(*) & hook (#34433) 
issue: #31224 #34374
pr: #34369

for query api:

1. param filter is not requried
2. param limit is useless while outputFields = [count(*)]

add hook about grpc call

---------

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-07-05 09:52:10 +08:00
aoiasd
9087b6f42e
enhance: [Cherry-Pick] support mark error as user error (#33498) (#34396) 
relate: https://github.com/milvus-io/milvus/issues/33492
pr: https://github.com/milvus-io/milvus/pull/33498
---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-07-04 10:08:10 +08:00
Chun Han
014cb7b071
enhance: use configed max topk for iterator when input topk exceeds(#34292) (#34293) 
related: #34292
pr: https://github.com/milvus-io/milvus/pull/34290

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-07-04 09:58:09 +08:00
SimFG
b3c5eb29ed
enhance: [2.4] the proxy metric in the query request (#34356) 
/kind improvement
- issue: #33306
- pr: #33307

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-03 10:00:09 +08:00
wayblink
c62bf8a0b0
fix: [Cherry-pick]Pick major compaction fixs and optimizations (#34360) 
This PR cherry-picks the following commits:

- fix: sync partitiion stats blocking balance task #33742
- fix: Fix meta prefix overlap bug #33830
- fix: Small fixs of major compaction #33929 
- fix: Fix memory buffer error & some renaming #33850
- fix: sync part stats task cannot be finished #34027 
- Add an option to enable/disable vector field clustering key #34097
- fix: fix error ignore in compactor #34169
- fix:load major compaction partial result #34052
- Use new stream segment reader in clustering compaction #34232

issue: #30633
pr: #33742 #33830 #33929 #33850 #34027 #34097 #34169 #34052 #34232

---------

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: Chun Han <116052805+MrPresent-Han@users.noreply.github.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-07-03 09:53:37 +08:00
wayblink
99586066f5
feat: [cherry-pick] Major compaction (#34326) 
This PR cherry-picks the following commits:
fix: speed up segment lookup via channel name in datacoord (#33530)
needed by the next commit
  feat: Major compaction (#33620)

issue: #30633
pr: #33620

---------

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: yiwangdr <80064917+yiwangdr@users.noreply.github.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
 2024-07-02 18:29:01 +08:00
congqixia
975f3bbeab
enhance: [2.4] Refine max length exceeded error message (#34300) (#34323) 
Cherry-pick from master
pr: #34300
This PR make varchar & string array field max length exceeded error
message clearer. Also fixed a minor issue that error string format and
argument number not match.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-02 15:52:09 +08:00
aoiasd
bc8ca593bd
enhance: [Cherry-Pick] Update access log (#34295) 
Support restful api,print with consistency level and add new write
cache.
relate: https://github.com/milvus-io/milvus/issues/31823
pr: https://github.com/milvus-io/milvus/pull/33155
https://github.com/milvus-io/milvus/pull/33503
https://github.com/milvus-io/milvus/pull/32213

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-07-02 15:30:15 +08:00
wei liu
c344083f22
enhance: Optimize grow slice cost during query (#34254) 
issue: #32252
pr: #34253

This PR try to pre-allocate FieldData for Reduce operations in the Query
chain using typeutil.PrepareResultFieldData to avoid the overhead of
dynamically growing the slice during appendFieldData process.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-01 15:18:11 +08:00
cqy123456
0bd93de627
enhance: [cherry-pick]remove CheckVecIndexWithDataTypeExist function in pkg (#34104) 
issue: https://github.com/milvus-io/milvus/issues/22837
related pr: https://github.com/milvus-io/milvus/pull/34102

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-26 21:14:26 +08:00
jaime
7f636dbdf6
enhance: refine error message for invalid limit parameter (#34179) 
pr: #34178

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-06-26 20:46:06 +08:00
jaime
6423b6c718
enhance: move rocksmq from internal to pkg (#34165) 
pr:  https://github.com/milvus-io/milvus/pull/33881
issue:  https://github.com/milvus-io/milvus/issues/33956

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-06-26 13:36:05 +08:00
Patrick Weizhi Xu
1168e8c107
enhance: [2.4] support integral type for MV and skip MV if there is only one category (#34005) 
pr: #33161
issue: #29892

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-06-24 19:12:02 +08:00
wei liu
061a00c58f
enhance: Enable database level replica num and resource groups for loading collection (#33052) (#33981) 
pr: #33052

issue: #30040

This PR introduce two database level props:
1. database.replica.number
2. database.resource_groups

User can set those two database props by AlterDatabase API, then can
load collection without specified replica_num and resource groups. then
it will use database level load param when try to load collections.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-21 16:56:02 +08:00
wei liu
2d3c5191b8
enhance: Refine name rule check error msg (#33815) (#33900) 
pr: #33815

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-19 14:26:01 +08:00
wei liu
4513569207
enhance: add restful api to trigger component stop (#32076) (#33799) 
issue: #32698
pr: #32076
This PR add two rest api for component stop and status check:
1. `/management/stop?role=querynode` can stop the specified component
2. `/management/check/ready?role=rootcoord` can check whether the target
component is serviceable

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-17 21:53:59 +08:00
Gao
5fc1370f6f
enhance: [2.4] autoindex for multi data type (#33867) 
issue: #22837 
pr: https://github.com/milvus-io/milvus/pull/33868

- opensource autoindex support
- metric type check for different data types
- autoindex data type for search param

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-06-14 23:26:00 +08:00
cqy123456
562369627d
enhance: [cherry-pick]check index with data type in knowhere api (#33878) 
issue: https://github.com/milvus-io/milvus/issues/22837
related: https://github.com/milvus-io/milvus/pull/33880

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-14 19:45:58 +08:00
jaime
fd1c7b1a1c
enhance: enable flush rate limiter of collection level (#33864) 
pr: #33837

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-06-14 16:27:57 +08:00
Gao
7ef28927e6
fix: [2.4] fix binary vector data size (#33751) 
issue #22837 
pr: #33750

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-06-12 20:11:55 +08:00
SimFG
c331aa4ad3
enhance: [2.4] add the includeCurrentMsg param for the Seek method (#33743) 
/kind improvement

- issue: #33325
- pr: #33326

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-06-11 15:01:55 +08:00
yihao.dai
ed1dee9e38
enhance: Support L0 import (#33514) (#33712) 
issue: https://github.com/milvus-io/milvus/issues/33157

pr: https://github.com/milvus-io/milvus/pull/33514

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-08 11:17:52 +08:00
Buqian Zheng
39e341e83a
fix: [2.4] update check for sparse hnsw index (#33714) 
issue: #29419
pr: #33713

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-06-07 18:36:07 +08:00
congqixia
7203ff8898
enhance: [2.4] Print UseDefaultConsistency param in read requests (#33617) (#33644) 
Cherry-pick from master
pr: #33617 
`UseDefaultConsistency` param is crucial for debugging slow query
problems. It could be confusing when guarantee timestamp is 1 while this
param is not logged

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-06 14:19:56 +08:00