admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00
Code Issues Packages Projects Releases Wiki Activity
19,362 Commits 146 Branches 196 Tags
Commit Graph

19362 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
congqixia
5d3aa2a496
fix: [Cherry-pick] Check latest leader exists before using it (#31500) (#31546) 
Cherry-pick from master
pr: #31500
See also #31495

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-23 11:51:06 +08:00
congqixia
99774548f2
enhance: [Cherry-pick] Add AllPartitionsID const to replace InvalidPartitionID (#31438) (#31515) 
Cherry-pick from master
pr: #31438

"-1" as `InvalidPartitionID` previously used as All partition place
holder in delete cases. It's confusing and hard to maintain when a const
var has more than one meaning.

This PR add `AllPartitionsID` to replace these usages in delete
scenarios.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-22 16:37:08 +08:00
congqixia
b8d8805c87
fix: [Cherry-pick] Use mutex protection when accessing pks of l0 segments (#31505) (#31517) 
Cherry-pick from master
pr: #31505
See also #31504

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-22 16:07:08 +08:00
congqixia
9174958335
enhance: [2.4] Add scripts to use gotestsum to execute integration test (#31490) (#31510) 
Cherry-pick from master
pr: #31490
See also #31489

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-22 15:41:07 +08:00
congqixia
278233391f
fix: [2.4] Check nodeID before update channel checkpoint (#31473) (#31507) 
Cherry-pick from master
pr: #31473
See also #31470 #31506

This PR adds nodeID assignment verification before updating channel
checkpoints.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-22 15:39:06 +08:00
sre-ci-robot
957765042d
[automated] Update cpu Builder image changes (#31482) 
Update cpu Builder image changes
See changes:
6afbc8a60d
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-03-22 11:39:10 +08:00
yanliang567
193638a7fe
test: [cherry-pick]Add tests for more json forms support (#31503) 
related issue: #31051 
related pr: #31474

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-03-22 11:09:06 +08:00
wei liu
cac9a29eac
fix: [skip e2e] Unstable integration test TestNodeDown (#31471) 
issue: #31468 
pr: #31430 cherry-pick test part
This PR fix unstable integration test `TestNodeDown`

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-03-22 10:27:17 +08:00
zhuwenxing
c0f1af11d5
test: adjust index timing (#31493) 
pr: https://github.com/milvus-io/milvus/pull/31492

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-03-22 09:09:11 +08:00
yihao.dai
47868e9d34
enhance: Enhance the load balancing of import tasks (#31423) (#31475) 
Optimize the load balancing of import tasks.

issue: https://github.com/milvus-io/milvus/issues/31232

pr: https://github.com/milvus-io/milvus/pull/31423

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-03-21 19:59:07 +08:00
sammy.huang
6afbc8a60d
enhance: add openblas dependency (#31435) 
pr: #31437

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-03-21 15:59:12 +08:00
congqixia
df154ccfc7
fix: [Cherry-pick] Use correct ticker for gc work defer (#31456) (#31462) 
Cherry-pick from master
pr: #31456
See also #31362

Fix the second defer using same ticker var.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-21 14:29:07 +08:00
groot
a0535edb67
enhance: Support MinIO TLS connection (#31396) 
issue: https://github.com/milvus-io/milvus/issues/30709
pr: https://github.com/milvus-io/milvus/pull/31292

Signed-off-by: yhmo <yihua.mo@zilliz.com>
Co-authored-by: Chen Rao <chenrao317328@163.com>
 2024-03-21 11:15:20 +08:00
sammy.huang
ad75ce043c
enhance: change config for etcd in nightly one pod (#31464) 
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-03-21 11:09:06 +08:00
chyezh
775129bd4e
enhance: add resource usage estimate for segment interface (#31309) 
issue: #30931
pr: #31050

- move resource estimate function outside from segment loader.
- add load info and collection to base segment.
- add resource usage method for sealed segment.

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-03-21 09:27:07 +08:00
zhuwenxing
a07abba1a0
enhance:[skip e2e]remove some custom deploy config (#31441) 
pr: https://github.com/milvus-io/milvus/pull/31440

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-03-20 20:57:06 +08:00
zhuwenxing
9d61ef56c7
test: [cherry-pick]update restful v2 testcases (#31433) 
update restful v2 test cases
pr: https://github.com/milvus-io/milvus/pull/31404

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-03-20 20:55:12 +08:00
zhuwenxing
b1df1d1516
enhance: [cherry-pick][skip e2e]remove slow pre commit (#31424) 
pr: https://github.com/milvus-io/milvus/pull/31397
remove slow pre commit

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-03-20 20:53:23 +08:00
zhuwenxing
528824f929
enhance:[skip e2e]update one pod helm values (#31425) 
pr: https://github.com/milvus-io/milvus/pull/31400

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-03-20 20:51:26 +08:00
congqixia
c6019c4f9d
enhance: [Cherry-pick] Add metrics for querycoord current target cp lag (#31391) (#31420) 
Cherry-pick from master
pr: #31391 #31399
See also #31390

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-20 20:47:10 +08:00
sammy.huang
2dd16bd401
enhance: add some parameters in docker-compose (#31453) 
pr: #31454

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-03-20 20:45:11 +08:00
sre-ci-robot
60fe5e8ed1
[automated] Bump milvus version to v2.4.0-rc.1 (#31383) 
Bump milvus version to v2.4.0-rc.1
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-03-20 19:19:06 +08:00
sammy.huang
72cf60457a
enhance: [skip e2e]change cache path (#31446) 
pr: #31445

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-03-20 18:03:11 +08:00
wei liu
7abebf81a3
fix: Load segment task promote failed (#31431) 
issue: #30816
pr: #31430

pr #31319 introduce the logic that segment checker need to load level
zero segment which only exist in current target.

This PR fix load segment task promote failed when segment only belongs
to current target

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
v2.4.0-rc.1 		2024-03-20 15:11:05 +08:00
zhagnlu
6856ba1e69
fix: fix mmap failed when string field all value is empty (#31418) 
pr: #31406

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-03-20 14:45:10 +08:00
cai.zhang
7f201e803c
fix: Fix bug for get segment index state (#31428) 
issue: #31361 
master pr: #31427

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-03-20 13:01:13 +08:00
sammy.huang
19d649f0b7
enhance: [skip e2e] remove unused mergify (#31393) 
issue: #31417
the file mergify is not supposed to be used in this branch
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-03-20 10:21:07 +08:00
congqixia
5649b9a463
enhance: [2.4] Cache formatted key for param item (#31388) (#31401) 
Cherry-pick from master
pr: #31388 
See also #30806

`formatKey` may cost lots of CPU on string processing under high QPS
scenario, this PR adds a formattedKeys cache preventing string operation
in each param get value.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-19 19:25:10 +08:00
nico
a95e1df7db
enhance: [skip e2e]modify 2.4 branch nihgtly groovy (#31398) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-03-19 15:45:06 +08:00
wei liu
f4449d4ef4
fix: Wrong behavior of CurrentTargetFirst/NextTargetFirst in target manager (#31378) 
issue: #31162
pr: #31379

when give scope CurrentTargetFirst/NextTargetFirst, it's expected to
scan both current and next target.

This PR fixed wrong behavior of CurrentTargetFirst/NextTargetFirst in
target manager, which may cause unexpected task generated, and load
collection may stuck forever due to dirty leader view.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-03-19 11:41:05 +08:00
XuanYang-cn
8946aa10d4
enhance: [cherry-pick]Skip submmit empty l0 tasks in DC (#31338) 
pr: #31280

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-03-19 10:15:12 +08:00
wei liu
0dbc0f2c11
fix: Hybrid search doesn't expire shard leader cache (#31381) 
issue: #31351
pr: #31380
This PR fixed that search/hybrid_search doesn't expire shard leader
cache when send request to query node failed, which make every request
keep trying to connect a offline query node

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-03-19 10:05:05 +08:00
congqixia
3254a14319
fix: [Cherry-pick] Cleanup write buffer when flowgraph released (#31377) 
Cherry-pick from master
pr: #31376
See also #30137

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-18 23:47:06 +08:00
Xiaofan
b2b107a774
fix: [cherry-pick] get compaction failure when datanode is actually live (#31356) 
if get compaction result failed, then skip processing compaction 
pr: #31353
see also #31352

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-03-18 22:53:05 +08:00
yihao.dai
72acd1d676
feat: Add RBAC functionality to ImportV2 (#31298) (#31337) 
issue: https://github.com/milvus-io/milvus/issues/31282,
https://github.com/milvus-io/milvus/issues/28521

pr: https://github.com/milvus-io/milvus/pull/31298

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-03-18 22:29:05 +08:00
Bingyi Sun
e7b053817d
feat: Add global mmap enable configuration (#31267) (#31373) 
https://github.com/milvus-io/milvus/issues/31279
related pr: #31267

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-03-18 21:57:05 +08:00
congqixia
89cff29b6a
enhance: [Cherry-pick] Use different interval for gc scan (#31364) 
Cherry-pick from master
pr: #31363
See also #31362

This PR make datacoord garbage collection scan operation using differet
interval than other opeartion.

This interval is a newly added param item, which default value is 7*24
hours.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-18 21:43:05 +08:00
wei liu
7ee7d484cc
fix: Transfer l0 segment to new delegator after balance (#31332) 
issue: #30186
pr: #31319

during channel balance, after new delegator loaded, instead of syncing
l0 segment's location to new delegator, we should load l0 segment on new
delegator, and release the old l0 segment, then start to release old
delegator.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-03-18 20:49:04 +08:00
yihao.dai
1885d176cf
fix: Return specific error code when encountering incomplete requery results (#31343) (#31359) 
During requery, segments may change (e.g., due to compaction), so we
need to return specific error codes when encountering incomplete requery
results. Clients can then retry to avoid this issue.

issue: https://github.com/milvus-io/milvus/issues/29656

pr: https://github.com/milvus-io/milvus/pull/31343

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-03-18 19:43:06 +08:00
congqixia
25d1c0e04b
enhance: Bump milvus & milvus-proto to 2.4.0-rc.1 (#31371) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-18 19:07:05 +08:00
sammy.huang
b773581fde
enhance: fetch simdjson directly in the format of targz (#31370) 
pr: #31369

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-03-18 18:55:10 +08:00
congqixia
4e83ca46bc
fix: [cherry-pick] Report offline info in GetDataDistribition (#31349) 
Cherry pick from master
pr: #31347
See also #31345

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-18 17:05:05 +08:00
PowderLi
f863f58785
fix: [restful v2] add hybrid_search (#31344) 
master old pr: #31113
(will updated after 2.4-rc released)

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-03-18 16:51:06 +08:00
yihao.dai
3c3be49cf4
fix: Fix binlog import (#31310) (#31330) 
Fix binlog import functionality by removing the existing check and
refining the size retrieval process.

issue: https://github.com/milvus-io/milvus/issues/31221,
https://github.com/milvus-io/milvus/issues/28521

pr: https://github.com/milvus-io/milvus/pull/31310

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-03-18 15:41:05 +08:00
liliu-z
fdb3231151
enhance: Upgrade Knowhere (#31308) 
/kind improvement

Signed-off-by: Li Liu <li.liu@zilliz.com>
 2024-03-18 14:21:04 +08:00
wei liu
555d96ff05
fix: Unstable ut TestGetClientFailed (#31296) (#31335) 
issue: #31295
pr: #31335

This PR fix unstable ut TestGetClientFailed

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-03-18 13:51:04 +08:00
wei liu
3987cd69d7
fix: save current target after target observer stop (#31333) 
issue: #28491
pr: #31315

should save target to meta store after target observer stop, incase of
target changed

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-03-18 13:49:11 +08:00
cai.zhang
85a1b6c96e
enhance: When describing an index, fetch the index info in batches (#31305) 
issue: #29313
master pr: #31238

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-03-18 12:31:05 +08:00
zhagnlu
7e3ac66617
fix: fix wrong guaranteeTs for hybrid search (#31329) 
pr: #31302

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-03-18 12:29:04 +08:00
sammy.huang
506534c29d
enhance: [skip e2e]change Semantics from 2.3.x to 2.X.X (#31348) 
pr: #31350

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-03-18 12:03:04 +08:00