2921 Commits

Author SHA1 Message Date
sre-ci-robot
a63badab5d
[automated] Update Pytest image changes (#36298)
Update Pytest image changes
See changes:
2df9309b4a
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-09-18 19:31:12 +08:00
nico
2df9309b4a
test: update pymilvus version and skip some cases (#36283)
pr: #36281

Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-09-14 17:57:14 +08:00
sre-ci-robot
97ddf1e29d
[automated] Update Pytest image changes (#36136)
Update Pytest image changes
See changes:
33054d5723
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-09-12 17:29:12 +08:00
Buqian Zheng
089790a459
enhance: [2.4]Allow empty sparse row (#36061)
pr: #34700

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-09-12 10:13:09 +08:00
nico
33054d5723
test: update pymilvus version and test cases (#36032)
pr: #35867

Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-09-10 10:39:09 +08:00
wei liu
ceca666e2a
fix: Fix privilege group hasn't been register for validate (#35938)
issue: #35471
pr: #35937

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-05 18:09:11 +08:00
yihao.dai
b578064869
fix: Fix DB limiter nodes are mistakenly cleaned up (#35991) (#35992)
This issue only occurs for a short time right after a table is created.
To avoid this, we simply reduce the frequency of cleaning up invalid
limiter nodes.

issue: https://github.com/milvus-io/milvus/issues/35933

pr: https://github.com/milvus-io/milvus/pull/35991

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-05 17:31:05 +08:00
jaime
a044a590cc
enhance: add IT for rate limit using db properties (#35931)
issue: #35929
pr: #35930

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-09-04 14:35:05 +08:00
wei liu
da026b1e28
enhance: Add depguard rules to ban deprecated proto lib (#35140) (#35818)
See also #34394 #34252
pr: #35140

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: congqixia <congqi.xia@zilliz.com>
2024-08-30 14:13:01 +08:00
yellow-shine
8cd75d627c
enhance: [ci] organize values file in the form of milvus deployment o… (#35839)
https://github.com/milvus-io/milvus/issues/35842
https://github.com/milvus-io/milvus/pull/35832

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-30 13:07:15 +08:00
nico
498c9470de
test: update test cases (#35718)
pr: #35640

Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-08-28 19:23:01 +08:00
yellow-shine
e331288b78
enhance: [skip e2e] Fix test script always skip pip install (#35775)
https://github.com/milvus-io/milvus/pull/35766

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-28 17:05:06 +08:00
yellow-shine
03627150d3
enhance new jenkinsfile for nightly and e2e (#35715)
https://github.com/milvus-io/milvus/pull/35672

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-27 14:23:05 +08:00
XuanYang-cn
ff68eb53d7
fix: [skip e2e]unstable l0 it (#35613)
See also: #35617 
pr: #35612

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-26 18:53:04 +08:00
sre-ci-robot
216f33c529
[automated] Update Pytest image changes (#35639)
Update Pytest image changes
See changes:
d7d4525c2e
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-22 19:18:58 +08:00
wei liu
421a00bfe8
enhance: Refresh proxy cache after restore rbac meta (#35636)
issue: #35443
pr: #35635

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-22 19:09:01 +08:00
yanliang567
d7d4525c2e
test:[cherry-pick] Add upsert with autoid and update init collection method (#35624)
Related issue: https://github.com/milvus-io/milvus/issues/32653
pr:  #35596

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-08-22 14:00:57 +08:00
wei liu
e2542a1bf5
enhance: Update protobuf-go to protobuf-go v2 (#34394) (#35555)
issue: #34252
pr: #34394 #35072 #35084

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-21 18:50:58 +08:00
wei liu
4bf4cbad85
enhance: Mark query node as read only after suspend (#35492) (#35586)
issue: #34985 #35493
pr: #35492
after querynode has been suspended, it's not allow to load
segment/channel on it, which means the node is read only. to be
compatible with resource group design, after query node has been
suspend, we remove it from it's original resource group, make it a read
only query node in replica. then two things will happens:
1. it's original resource group will be lacking of query nodes, query
coord will assign new node to it.
2. querycoord will try to move out all segments/channels after querynode
has been suspended

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-20 19:00:56 +08:00
wei liu
4610dafb2e
enhance: make configure load param feature be compatible with old sdk(#35520) (#35573)
issue: #31570 #35521
pr: #35520 #35546

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-20 18:20:57 +08:00
Chun Han
cf8494ef45
enhance: support httpv1/v2 throttle and add it for httpV2(#35350) (#35504)
related: #35350
pr: https://github.com/milvus-io/milvus/pull/35470

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-08-20 16:32:56 +08:00
XuanYang-cn
4a5e6bc6f6
enhance: Add integration tests for l0 (#35430)
pr: #35429
See also: #34796

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-19 10:54:55 +08:00
wei liu
248a6ea401
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup (#35444) (#35513)
issue: #35443
pr: #35444

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-18 13:42:55 +08:00
smellthemoon
23052f6ac2
fix: upsert use the previous pk in insert when autoid(#34672) (#35130)
issue: #34668 
pr: #34672

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-12 18:58:25 +08:00
zhagnlu
db9e4b898a
enhance: support bitmap index (#35336)
pr: #32902
cherry-pick bitmap index from master

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-07 20:12:18 +08:00
wei liu
0201e00a2f
enhance: enable to set load config in cluster level (#35293)
issue: #35170
pr: #35169
This PR enable to set load configs in cluster level, such as replicas
and resource groups. then when load collections will use the load
config.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-07 12:38:21 +08:00
nico
18de5f56ad
test: update test cases (#35210)
pr: #35218

Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-08-07 09:32:17 +08:00
wei liu
2ac1bf7532
enhance: Enable setting the replica number and resource group during collection creation (#34403) (#34561)
issue: #30040
pr: #34403

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-06 15:06:17 +08:00
sre-ci-robot
fc61f66bb3
[automated] Update Pytest image changes (#35245)
Update Pytest image changes
See changes:
3a997e8056
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-05 09:24:33 +08:00
yellow-shine
3a997e8056
enhance: docker-compose first then try to use docker compose (#35227)
https://github.com/milvus-io/milvus/issues/35209
https://github.com/milvus-io/milvus/pull/35208

---------

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-02 19:37:09 +08:00
nico
9bebca472b
test: update pymilvus version (#35143)
pr: #34843

Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-08-01 11:04:13 +08:00
cai.zhang
c340f387cf
enhance: [cherry-pick] Change the fixed value to a ratio for clustering segment size (#35075)
issue: #34495 

master pr: #35076

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-31 10:32:00 +08:00
zhagnlu
866055527b
enhance: revert remove duplicated pk function (#35102)
pr: #35103

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-30 20:03:50 +08:00
congqixia
079276c6ff
fix: [2.4] Unify hook singleton implementation in proxy (#34888)
Cherry-pick from master
pr: #34887
Related to #34885

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-26 18:07:53 +08:00
wayblink
1e5c71d550
fix: [cherry-pick] fix dropped segment still visible after dropped by L2 single compaction (#35006)
bug: #35003

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-26 13:47:48 +08:00
zhagnlu
6f0f3034c6
fix: fix query count(*) concurrently (#35010)
pr: #35007

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-26 11:11:46 +08:00
zhagnlu
9eb42bbda0
fix: fix parse plan proto failed for search type (#34957)
pr: #34944

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-25 20:27:46 +08:00
cai.zhang
74adedf750
enhance: Optimized the GC logic to ensure that memory is released in time (#34950)
issue: #34703 

master pr: #34949

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-24 14:07:43 +08:00
Chun Han
ae1636c2be
fix: refine handling type for segment pruner(#34923) (#34926)
related: #34923
pr: https://github.com/milvus-io/milvus/pull/34925

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-07-24 12:05:44 +08:00
wayblink
8f3c126129
enhance: [cherry-pick] support l2 single compaction (#34929)
#34928
pr: #34935

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-24 11:47:50 +08:00
cai.zhang
4ed62e9dbb
enhance: [cherry-pick] Add integration test for clustering compaction (#34860)
issue: #34792 

master pr: #34881

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2024-07-22 17:49:42 +08:00
wayblink
21973a600d
enhance: [cherry-pick] refine clustering compaction basic it (#34794)
issue: #34792
pr: #34793

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-21 20:05:41 +08:00
yihao.dai
07bc1b6717
enhance: Seal by total growing segments size (#34692) (#34779)
Seals the largest growing segment if the total size of growing segments
of each shard exceeds the size threshold(default 4GB). Introducing this
policy can help keep the size of growing segments within a suitable
level, alleviating the pressure on the delegator.

issue: https://github.com/milvus-io/milvus/issues/34554

pr: https://github.com/milvus-io/milvus/pull/34692

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-19 18:25:50 +08:00
XuanYang-cn
edefc3cbb5
enhance: [skip e2e]Enable compaction it test (#34526) (#34720)
pr: #34526

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-07-16 18:19:38 +08:00
SimFG
13f129aa9c
enhance: [2.4] format the sdk code (#34699)
- issue: #31293
- pr: #34033

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-16 14:15:38 +08:00
elstic
40f6febf9b
test: [cherry-pick] Fix index creation mismatch for sparse vector columns in multivector case (#34620)
pr: https://github.com/milvus-io/milvus/pull/34618

Signed-off-by: elstic <hao.wang@zilliz.com>
2024-07-16 09:39:38 +08:00
smellthemoon
0fdb288de7
enhance: upsert support autoid(#30342) (#34633)
pr: #30342 
issue: #29258

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
2024-07-15 20:53:39 +08:00
wei liu
cf701a9bf0
enhance: Preserve fixed-size memory in delegator node for growing segment (#34600)
issue: #34595
pr: #34596
When consuming insert data on the delegator node, QueryCoord will move
out some sealed segments to manage its memory usage. After the growing
segment gets flushed, some sealed segments from other workers will be
moved back to the delegator node. To avoid the frequent movement of
segments, we estimate the maximum growing row count and preserve a
fixed-size memory in the delegator node.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-15 20:51:46 +08:00
zhuwenxing
f2d0517f96
test: [cherry-pick]add testcase for count query (#34471)
pr: https://github.com/milvus-io/milvus/pull/34453

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-07-08 12:54:12 +08:00
Gao
a60e2a65ff
enhance: change autoindex default metric type (#34277)
issue: #34304 
pr: #34261

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-07-08 10:52:14 +08:00