zhuwenxing
a07abba1a0
enhance:[skip e2e]remove some custom deploy config ( #31441 )
...
pr: https://github.com/milvus-io/milvus/pull/31440
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-03-20 20:57:06 +08:00
zhuwenxing
9d61ef56c7
test: [cherry-pick]update restful v2 testcases ( #31433 )
...
update restful v2 test cases
pr: https://github.com/milvus-io/milvus/pull/31404
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-03-20 20:55:12 +08:00
zhuwenxing
b1df1d1516
enhance: [cherry-pick][skip e2e]remove slow pre commit ( #31424 )
...
pr: https://github.com/milvus-io/milvus/pull/31397
remove slow pre commit
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-03-20 20:53:23 +08:00
zhuwenxing
528824f929
enhance:[skip e2e]update one pod helm values ( #31425 )
...
pr: https://github.com/milvus-io/milvus/pull/31400
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-03-20 20:51:26 +08:00
congqixia
c6019c4f9d
enhance: [Cherry-pick] Add metrics for querycoord current target cp lag ( #31391 ) ( #31420 )
...
Cherry-pick from master
pr: #31391 #31399
See also #31390
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-20 20:47:10 +08:00
sammy.huang
2dd16bd401
enhance: add some parameters in docker-compose ( #31453 )
...
pr: #31454
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-03-20 20:45:11 +08:00
sre-ci-robot
60fe5e8ed1
[automated] Bump milvus version to v2.4.0-rc.1 ( #31383 )
...
Bump milvus version to v2.4.0-rc.1
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-03-20 19:19:06 +08:00
sammy.huang
72cf60457a
enhance: [skip e2e]change cache path ( #31446 )
...
pr: #31445
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-03-20 18:03:11 +08:00
wei liu
7abebf81a3
fix: Load segment task promote failed ( #31431 )
...
issue: #30816
pr: #31430
pr #31319 introduce the logic that segment checker need to load level
zero segment which only exist in current target.
This PR fix load segment task promote failed when segment only belongs
to current target
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-20 15:11:05 +08:00
zhagnlu
6856ba1e69
fix: fix mmap failed when string field all value is empty ( #31418 )
...
pr: #31406
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-03-20 14:45:10 +08:00
cai.zhang
7f201e803c
fix: Fix bug for get segment index state ( #31428 )
...
issue: #31361
master pr: #31427
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-20 13:01:13 +08:00
sammy.huang
19d649f0b7
enhance: [skip e2e] remove unused mergify ( #31393 )
...
issue: #31417
the file mergify is not supposed to be used in this branch
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-03-20 10:21:07 +08:00
congqixia
5649b9a463
enhance: [2.4] Cache formatted key for param item ( #31388 ) ( #31401 )
...
Cherry-pick from master
pr: #31388
See also #30806
`formatKey` may cost lots of CPU on string processing under high QPS
scenario, this PR adds a formattedKeys cache preventing string operation
in each param get value.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-19 19:25:10 +08:00
nico
a95e1df7db
enhance: [skip e2e]modify 2.4 branch nihgtly groovy ( #31398 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-03-19 15:45:06 +08:00
wei liu
f4449d4ef4
fix: Wrong behavior of CurrentTargetFirst/NextTargetFirst in target manager ( #31378 )
...
issue: #31162
pr: #31379
when give scope CurrentTargetFirst/NextTargetFirst, it's expected to
scan both current and next target.
This PR fixed wrong behavior of CurrentTargetFirst/NextTargetFirst in
target manager, which may cause unexpected task generated, and load
collection may stuck forever due to dirty leader view.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-19 11:41:05 +08:00
XuanYang-cn
8946aa10d4
enhance: [cherry-pick]Skip submmit empty l0 tasks in DC ( #31338 )
...
pr: #31280
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-03-19 10:15:12 +08:00
wei liu
0dbc0f2c11
fix: Hybrid search doesn't expire shard leader cache ( #31381 )
...
issue: #31351
pr: #31380
This PR fixed that search/hybrid_search doesn't expire shard leader
cache when send request to query node failed, which make every request
keep trying to connect a offline query node
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-19 10:05:05 +08:00
congqixia
3254a14319
fix: [Cherry-pick] Cleanup write buffer when flowgraph released ( #31377 )
...
Cherry-pick from master
pr: #31376
See also #30137
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-18 23:47:06 +08:00
Xiaofan
b2b107a774
fix: [cherry-pick] get compaction failure when datanode is actually live ( #31356 )
...
if get compaction result failed, then skip processing compaction
pr: #31353
see also #31352
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-03-18 22:53:05 +08:00
yihao.dai
72acd1d676
feat: Add RBAC functionality to ImportV2 ( #31298 ) ( #31337 )
...
issue: https://github.com/milvus-io/milvus/issues/31282 ,
https://github.com/milvus-io/milvus/issues/28521
pr: https://github.com/milvus-io/milvus/pull/31298
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-18 22:29:05 +08:00
Bingyi Sun
e7b053817d
feat: Add global mmap enable configuration ( #31267 ) ( #31373 )
...
https://github.com/milvus-io/milvus/issues/31279
related pr: #31267
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-18 21:57:05 +08:00
congqixia
89cff29b6a
enhance: [Cherry-pick] Use different interval for gc scan ( #31364 )
...
Cherry-pick from master
pr: #31363
See also #31362
This PR make datacoord garbage collection scan operation using differet
interval than other opeartion.
This interval is a newly added param item, which default value is 7*24
hours.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-18 21:43:05 +08:00
wei liu
7ee7d484cc
fix: Transfer l0 segment to new delegator after balance ( #31332 )
...
issue: #30186
pr: #31319
during channel balance, after new delegator loaded, instead of syncing
l0 segment's location to new delegator, we should load l0 segment on new
delegator, and release the old l0 segment, then start to release old
delegator.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-18 20:49:04 +08:00
yihao.dai
1885d176cf
fix: Return specific error code when encountering incomplete requery results ( #31343 ) ( #31359 )
...
During requery, segments may change (e.g., due to compaction), so we
need to return specific error codes when encountering incomplete requery
results. Clients can then retry to avoid this issue.
issue: https://github.com/milvus-io/milvus/issues/29656
pr: https://github.com/milvus-io/milvus/pull/31343
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-18 19:43:06 +08:00
congqixia
25d1c0e04b
enhance: Bump milvus & milvus-proto to 2.4.0-rc.1 ( #31371 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-18 19:07:05 +08:00
sammy.huang
b773581fde
enhance: fetch simdjson directly in the format of targz ( #31370 )
...
pr: #31369
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-03-18 18:55:10 +08:00
congqixia
4e83ca46bc
fix: [cherry-pick] Report offline info in GetDataDistribition ( #31349 )
...
Cherry pick from master
pr: #31347
See also #31345
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-18 17:05:05 +08:00
PowderLi
f863f58785
fix: [restful v2] add hybrid_search ( #31344 )
...
master old pr: #31113
(will updated after 2.4-rc released)
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-03-18 16:51:06 +08:00
yihao.dai
3c3be49cf4
fix: Fix binlog import ( #31310 ) ( #31330 )
...
Fix binlog import functionality by removing the existing check and
refining the size retrieval process.
issue: https://github.com/milvus-io/milvus/issues/31221 ,
https://github.com/milvus-io/milvus/issues/28521
pr: https://github.com/milvus-io/milvus/pull/31310
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-18 15:41:05 +08:00
liliu-z
fdb3231151
enhance: Upgrade Knowhere ( #31308 )
...
/kind improvement
Signed-off-by: Li Liu <li.liu@zilliz.com>
2024-03-18 14:21:04 +08:00
wei liu
555d96ff05
fix: Unstable ut TestGetClientFailed ( #31296 ) ( #31335 )
...
issue: #31295
pr: #31335
This PR fix unstable ut TestGetClientFailed
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-18 13:51:04 +08:00
wei liu
3987cd69d7
fix: save current target after target observer stop ( #31333 )
...
issue: #28491
pr: #31315
should save target to meta store after target observer stop, incase of
target changed
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-18 13:49:11 +08:00
cai.zhang
85a1b6c96e
enhance: When describing an index, fetch the index info in batches ( #31305 )
...
issue: #29313
master pr: #31238
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-18 12:31:05 +08:00
zhagnlu
7e3ac66617
fix: fix wrong guaranteeTs for hybrid search ( #31329 )
...
pr: #31302
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-03-18 12:29:04 +08:00
sammy.huang
506534c29d
enhance: [skip e2e]change Semantics from 2.3.x to 2.X.X ( #31348 )
...
pr: #31350
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-03-18 12:03:04 +08:00
PowderLi
f2814f2eea
fix: [CHERRY-PICK] [restful v2] create a collection with vector field name ( #31318 )
...
fix: [restful v2] create a collection with vector field name
master pr: #31317
1. quickly create collection, create an index on the vector field #31149
2. valid consistencyLevel is needed, while create a collection
3. show collection's properties and aliases #31180
4. list aliase for one collection, default is the whole database
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-03-17 20:07:04 +08:00
Buqian Zheng
99e0d28c9e
fix: [2.4 sparse float vector] support the new streaming deserialize reader ( #31326 )
...
issue: https://github.com/milvus-io/milvus/issues/31324
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-03-16 23:58:22 -07:00
congqixia
08aba2e05f
fix: Remove QueryNodeEntitiesSize after segment/collection released ( #31290 )
...
See also #31289
This PR:
- Set collection level `QueryNodeEntitiesSize` to zero if all segment
released
- Delete `QueryNodeEntitiesSize` metrics value after collection ref is
zero
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-15 15:43:04 +08:00
chyezh
465fd474de
enhance: add syncutil type ContextCond and VersionedNotifier ( #30648 )
...
issue: #30647
- ContextCond is a broadcast-only condition variable which can be
canceled by context.
- VersionedNotifier is a version-based notifier-listener implementation,
which promise no change can be ignored.
Signed-off-by: chyezh <chyezh@outlook.com>
2024-03-15 15:41:04 +08:00
Gao
038c570ef3
enhance: upgrade folly to run on arm ( #31284 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-03-15 15:39:03 +08:00
Gao
6f1e9cd0f4
fix: fix cal_distance default implementation ( #31245 )
...
#31243
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-03-15 15:37:11 +08:00
elstic
93bb04f884
test: add some mmap cases ( #31211 )
...
issue : https://github.com/milvus-io/milvus/issues/30940
Signed-off-by: elstic <hao.wang@zilliz.com>
2024-03-15 15:25:05 +08:00
nico
9e7ce0a8f7
test: update test cases ( #31266 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-03-15 15:23:10 +08:00
yanliang567
8563c4a5ee
enhance: Update groupby tests ( #31297 )
...
related issue: #29883
skip running for now
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-03-15 15:21:03 +08:00
yihao.dai
c408a32db6
feat: Add disk quota checks for import V2 ( #31131 )
...
Return quota error when the files to be imported exceed the disk quota.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-15 14:43:03 +08:00
wei liu
d79aa58b37
enhance: Speed up target recovery after query coord restart ( #31240 )
...
issue: #28491
after querycoord restart, it will pull a new target, which include
channel and segment list. when segments loaded on querynode has reached
the target, the collection could provide search/query. but if segment
list changes by time, ater querycoord pull a new target, it will takes a
few minutes to catch up the target's segment distribution. and before
that, query/search will fail due to lack of segments.
This PR save the current loaded target to meta storein querycoord's stop
progress, and recover it when query coord starts, to speed up the target
recovery time.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-15 14:19:03 +08:00
aoiasd
b724753137
enhance: Add runtime config to paramtable ( #31006 )
...
relate: https://github.com/milvus-io/milvus/issues/30806
Avoid use string convert or format function when get some runtime
parameter
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-15 11:07:06 +08:00
yihao.dai
2b035ba2d4
enhance: Allow import tasks to retry for more errors ( #31268 )
...
Allow import tasks to retry for a wider range of errors, including all
gRPC errors and unexpected status codes from Milvus.
issue: https://github.com/milvus-io/milvus/issues/31227 ,
https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-15 11:05:04 +08:00
yihao.dai
811316d2ba
fix: Fix binlog import and refine error reporting ( #31241 )
...
1. Fix binlog import with partition key.
2. Refine binlog import error reportins.
3. Avoid division by zero when retrieving import progress.
issue: https://github.com/milvus-io/milvus/issues/31221 ,
https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-15 10:55:05 +08:00
chyezh
ff4237bb90
enhance: add hostname into node info ( #30673 )
...
issue: https://github.com/milvus-io/milvus/issues/30647
- Address may be reused in k8s environment. Using hostname can be
better.
Signed-off-by: chyezh <chyezh@outlook.com>
2024-03-15 10:45:06 +08:00
