admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00
Code Issues Packages Projects Releases Wiki Activity
20,220 Commits 146 Branches 196 Tags
Commit Graph

20220 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
yihao.dai
7ef96ef648
enhance: Avoid unnecessary compaction (#35148) (#35149) 
Estimate the import segment size based on DiskSegmentMaxSize(2G) to
avoid unnecessary compaction after import completed.

issue: https://github.com/milvus-io/milvus/issues/35147

pr: https://github.com/milvus-io/milvus/pull/35148

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-06 10:30:20 +08:00
sre-ci-robot
afd12a15a0
[automated] Update gpu Builder image changes (#35285) 
Update gpu Builder image changes
See changes:
595aeae0f1
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-08-06 10:14:16 +08:00
yellow-shine
94bb687e6f
feat: [skip e2e] effective way to use cache (#35232) 
issue: https://github.com/milvus-io/milvus/issues/34876
https://github.com/milvus-io/milvus/pull/35109

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-08-06 10:12:26 +08:00
sre-ci-robot
8b5e02a464
[automated] Update cpu Builder image changes (#35286) 
Update cpu Builder image changes
See changes:
595aeae0f1
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-08-06 10:10:16 +08:00
Jiquan Long
e598233b6a
feat: put inverted index into ram (#35222) (#35223) 
fix: https://github.com/milvus-io/milvus/issues/35224
pr: https://github.com/milvus-io/milvus/pull/35222

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-08-06 10:08:16 +08:00
congqixia
1b304e982a
enhance: [2.4] Resolve chunk cache conflict cpp lint (#35278) 
Resolve cpp lint issue introduced in #35189

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-05 19:46:16 +08:00
sthuang
595aeae0f1
enhance: upgrade go version to 1.21.11 (#35258) 
issue: https://github.com/milvus-io/milvus/issues/35142

related pr in master: https://github.com/milvus-io/milvus/pull/35257

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2024-08-05 17:28:26 +08:00
yihao.dai
7f49476363
enhance: Refine import error log (#35067) (#35068) 
issue: https://github.com/milvus-io/milvus/issues/35060

pr: https://github.com/milvus-io/milvus/pull/35067

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-05 15:28:17 +08:00
wei liu
d48c690cb3
enhance: Avoid unnecesary syncTargetVersion func call after querycoord recover (#34954) (#35234) 
pr: #34954
before querycoord stop gracefully, we will save the current target to
meta store and recover it after querycoord start up, to speed the
querycoord's recovery time. but the target version hasn't been recovered
as expected, and it use latest timestamp as current target's version,
which has no effect to querycoord but an unnecessary syncTargetVersion
func call.

This PR recover the correct target version to avoid unnecessary
syncTargetVersion func call

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-05 10:18:16 +08:00
sre-ci-robot
fc61f66bb3
[automated] Update Pytest image changes (#35245) 
Update Pytest image changes
See changes:
3a997e8056
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-08-05 09:24:33 +08:00
sre-ci-robot
737f402478
[automated] Update cpu Builder image changes (#35250) 
Update cpu Builder image changes
See changes:
f90322a8ca
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-08-05 09:22:21 +08:00
cai.zhang
2534b30e39
enhance: [cherry-pick] Add monitoring metrics for task execution time in datacoord (#35141) 
issue: #35138 

master pr: #35139

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-02 19:46:16 +08:00
yellow-shine
3a997e8056
enhance: docker-compose first then try to use docker compose (#35227) 
https://github.com/milvus-io/milvus/issues/35209
https://github.com/milvus-io/milvus/pull/35208

---------

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-08-02 19:37:09 +08:00
Chun Han
58f7c35b75
enhance: add log for partition stats(#30376) (#35220) 
related: #30376
pr: https://github.com/milvus-io/milvus/pull/35219

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-02 19:34:21 +08:00
cqy123456
d9c1990045
enhance:[cherry-pick]update mmapchunkmanager config in milvus.yaml (#35186) 
issue: https://github.com/milvus-io/milvus/issues/32984
related pr: https://github.com/milvus-io/milvus/pull/35180

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-08-02 19:26:48 +08:00
yellow-shine
f90322a8ca
enhance: upgrade conan version (#35216) 
https://github.com/milvus-io/milvus/pull/35215

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-08-02 19:26:15 +08:00
Chun Han
f8676178f7
fix: querynode hang when failing to allocate disk space for mmap(#35184) (#35189) 
related: #35184
pr: https://github.com/milvus-io/milvus/pull/35187

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-02 19:20:16 +08:00
wei liu
2609e95cb5
enhance: add segment state on data coord bin log size metrics (#35153) (#35235) 
pr: #35153

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-02 18:28:14 +08:00
wei liu
11578772ef
fix: Set legacy level to l0 segment after qc restart (#35197) (#35211) 
issue: #35087
pr: #35197
after qc restarts, and target is not ready yet, if dist_handler try to
update segment dist, it will set legacy level to l0 segment, which may
cause l0 segment be moved to other node, cause search/query failed.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-02 18:22:15 +08:00
cai.zhang
756922ebec
fix: [cherry-pick] Maintain load idempotency even when building new indexes (#35179) 
issue: #34404 

master pr: #35178

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-02 17:28:15 +08:00
congqixia
b70ede4c9b
enhance: [2.4] Set WatchProgress for channel operation progress response (#35183) (#35231) 
Cherry-pick from master
pr: #35183

The watch progress is always zero in CheckChannelOperationProgress
response, which is meaningless and confusing. This PR set progress value
in rpc response to fix this problem.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-02 16:52:15 +08:00
wayblink
fc974c4db1
enhance: [cherry-pick] improve ut for compaction_policy_clustering (#35206) 
#34792
master pr: #35205

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-08-02 16:38:15 +08:00
wei liu
d767f8977a
enhance: Refine param init for MmapDirPath (#35181) (#35214) 
pr: #35181

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-02 16:30:15 +08:00
congqixia
f8444b900f
enhance: [2.4] Support proxy/delegator qn client pooling (#35195) 
Cherry pick from master
pr: #35194
See also #35196
Add param item for proxy/delegator query node client pooling and
implement pooling logic

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-02 11:24:19 +08:00
wayblink
8137c26330
enhance: [cherry-pick] unify time in clustering compaction task to unix (#35168) 
related: #34495 
pr: #35167

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-08-02 10:30:19 +08:00
Jiquan Long
ae3657b23e
enhance: add metric to record maxInsertRate and queryNodeMemoryHighWaterLevel (#35185) (#35188) 
pr: https://github.com/milvus-io/milvus/pull/35185
fix: https://github.com/milvus-io/milvus/issues/35177

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-08-02 10:16:15 +08:00
cai.zhang
5f3d41d6c6
enhance:[cherry-pick] Support calculate segment maxRows with diskann (#35155) 
issue: #34495 

master pr: #35076

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-02 09:56:14 +08:00
cai.zhang
f7184101e1
fix: [cherry-pick] Fix data race for clustering buffer writer (#35146) 
issue: #34495 

master pr: #35145

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-01 16:08:13 +08:00
wei liu
5f601fcc50
enhance: Reduce delegator memory overloaded factor to 0.1 (#35092) (#35164) 
pr: #35092

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-01 14:20:13 +08:00
nico
9bebca472b
test: update pymilvus version (#35143) 
pr: #34843

Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-08-01 11:04:13 +08:00
congqixia
824b26c209
enhance: [2.4] Pre-allocate space for reduce data structure (#35118) (#35137) 
Cherry-pick from master
pr: #35118 
Grow slice & map.growWork may cause a lot when segment number is large
for big K query. This PR pre-allocate space for reduce methods to avoid
this cost.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-01 09:59:50 +08:00
wayblink
37a5f91683
enhance: [cherry-pick] add commit time in partitionStats proto (#35124) 
fix: #35110
pr: #35125

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-31 21:22:04 +08:00
Gao
be0123863f
enhance: add channel num for queryHook optimization (#35105) 
pr: #35104

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-07-31 18:23:51 +08:00
wei liu
f8f2736cda
enhance: Skip manual stopped component during health check (#34953) (#35123) 
pr: #34953
after manual stop component by management restful api, `healthz` may
return unhealthy state. k8s may restart the pod to save the unhealthy
sate, and the manual stop operation will got unexpected result.

to solve this, we make `healthz` API skip the manual stopped component.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-31 18:01:51 +08:00
Patrick Weizhi Xu
85d8200acb
enhance: [2.4] improve error msg when expr invalid during isolation (#35115) 
issue: #34336 
pr: #35127

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-07-31 17:21:50 +08:00
XuanYang-cn
cf56a1452c
enhance: Add more ut for l0CompactionTask (#35101) 
See also: #34796
pr: #35100

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-31 17:19:50 +08:00
Patrick Weizhi Xu
0bec383342
fix: [2.4] disable sparse and binary in MV (#35121) 
issue: #35120 
pr: #35126

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-07-31 15:02:01 +08:00
congqixia
f5d0e55ebe
enhance: [2.4] Resolve conflict cpp format (#35129) 
See also #35073 #35119

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-31 12:15:50 +08:00
congqixia
94c86327ae
enhance: [2.4] Use clang-format-12 to perform cpp check (#35119) 
clang-format version is 12 after upgrading to ubuntu 22.04, this PR
change formating script to use clang-format-12 and fix existing problems

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-31 10:33:50 +08:00
cai.zhang
c340f387cf
enhance: [cherry-pick] Change the fixed value to a ratio for clustering segment size (#35075) 
issue: #34495 

master pr: #35076

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-31 10:32:00 +08:00
cai.zhang
4bbb7b8219
enhance: [cherry-pick] Skip updating the level that are already at the specified level (#35093) 
issue: #34495 

master pr: #35094

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-31 09:39:54 +08:00
congqixia
935a117396
enhance: [2.4] Support otlp http exporter (#35053) (#35073) 
Cherry-pick from master
pr: #35053
See also #35052

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 21:00:08 +08:00
congqixia
8991dc211e
enhance: [2.4] Fix go&cpp lint issues (#35107) 
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 20:25:55 +08:00
zhagnlu
866055527b
enhance: revert remove duplicated pk function (#35102) 
pr: #35103

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-30 20:03:50 +08:00
foxspy
410eb8a787
fix: [2.4] add prometheus dependency for milvus_monitor (#35079) 
issue: #35077 
pr: #35078

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
 2024-07-30 19:41:49 +08:00
Jiquan Long
86edca8c1b
fix: support auto index for array (#35095) 
/kind branch-feature
pr: #34450

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: Zhagnlu <lu.zhang@zilliz.com>
 2024-07-30 17:57:50 +08:00
Congqi Xia
d51bfcdd7f
Skip cases 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 16:10:38 +08:00
PowderLi
fa9d606aa8
enhance: add metrics for restful apis (#35083) 
issue: #34968
pr: #34969

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-07-30 14:13:49 +08:00
chyezh
b6c1aeb6d9
enhance: the datacoord gc should fast quitable (#35051) 
issue: #35049
pr: #35050

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-07-30 14:11:58 +08:00
yihao.dai
cc188fb322
enhance: Avoid panic due to nil schema (#35063) (#35064) 
/kind improvement

issue: https://github.com/milvus-io/milvus/discussions/25620

pr: https://github.com/milvus-io/milvus/pull/35063

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-29 20:07:49 +08:00