sre-ci-robot
cce25ecdbc
[automated] Update Knowhere Commit ( #38635 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-21 00:50:46 +08:00
Xianhui Lin
90de37e60a
enhance: field stringtype maxlength raise to 1M ( #38592 )
...
enhance: field stringtype maxlength raise to 1M
issue: https://github.com/milvus-io/milvus/issues/37436
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-20 14:32:45 +08:00
XuanYang-cn
ca7ec23198
enhance: Use partitionID when delete by partitionKey ( #38231 )
...
When delete by partition_key, Milvus will generates L0 segments
globally. During L0 Compaction, those L0 segments will touch all
partitions collection wise. Due to the false-positive rate of segment
bloomfilters, L0 compactions will append false deltalogs to completed
irrelevant partitions, which causes *partition deletion amplification.
This PR uses partition_key to set targeted partitionID when producing
deleteMsgs into MsgStreams. This'll narrow down L0 segments scope to
partition level, and remove the false-positive influence
collection-wise.
However, due to DeleteMsg structure, we can only label one partition to
one deleteMsg, so this enhancement fails if user wants to delete over 2
partition_keys in one deletion.
See also: #34665
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-20 11:18:46 +08:00
XuanYang-cn
c0b855dc75
fix: ChannelManager concurret Release and Watch bug ( #38590 )
...
See also: #38589
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-19 22:50:47 +08:00
zhagnlu
8fcb33c21d
fix:fix delete record assert failed ( #38580 )
...
#38472
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-19 18:22:47 +08:00
congqixia
3d360c0624
fix: SyncSegments rpc always failed ( #38578 )
...
miss the patch due to code branching
previous pr: #38032
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
2024-12-19 15:40:45 +08:00
Zhen Ye
b537a72309
fix: interted index out of range ( #38577 )
...
issue: #38546 , #38486
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-19 15:20:47 +08:00
cai.zhang
306e5e6898
enhance: clean compaction task in compactionHandler ( #38170 )
...
issue: #35711
---------
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
2024-12-19 12:36:47 +08:00
jaime
78438ef41e
fix: revert optimize CPU usage for CheckHealth requests ( #35589 ) ( #38555 )
...
issue: #35563
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-19 00:38:45 +08:00
smellthemoon
134d06f7e6
fix: need pass parameter to listPrivilegeGroups ( #38562 )
...
issue: https://github.com/milvus-io/milvus/issues/38563
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-18 20:28:51 +08:00
sthuang
de8858931a
fix: rbac revoke check if there is same privilege in other privilege groups granted ( #38558 )
...
related: https://github.com/milvus-io/milvus/issues/38557
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-18 18:46:45 +08:00
cai.zhang
a348122758
fix: Support get segments from current segments view ( #38512 )
...
issue: #38511
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-18 18:00:54 +08:00
foxspy
06d410b70f
enhance: update knowhere version ( #38544 )
...
related: #37730
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-12-18 16:52:45 +08:00
zhagnlu
87056be748
fix: fix snapshot or size when query ( #38549 )
...
#38472
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-18 16:42:45 +08:00
Bingyi Sun
894c203f27
enhance: allow hyphen in partition name ( #38494 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-18 15:04:45 +08:00
zhagnlu
6ee94d00b9
fix:fix calculate arrow nest type and add ut ( #38527 )
...
#37767
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-18 11:54:44 +08:00
sre-ci-robot
ffd3c5d2f5
[automated] Update Knowhere Commit ( #38542 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-18 01:24:47 +08:00
cai.zhang
7a05b5bbea
fix: Fix stats task wrong RootPath when upload binlog ( #38539 )
...
issue: #38336
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-18 00:18:45 +08:00
yihao.dai
1aa31e2a9b
fix: Fix inaccurate general count ( #38524 )
...
Checking general count should only count healthy collections and
partitions.
issue: https://github.com/milvus-io/milvus/issues/37630
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-17 23:24:45 +08:00
jaime
4d5572d7ed
enhance: [skip e2e] refine display format ( #38537 )
...
issue: #36621
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-17 22:30:44 +08:00
Chun Han
decdfdae10
fix: growing-groupby-crush( #38533 ) ( #38538 )
...
related: #38533
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-12-17 21:05:12 +08:00
Bingyi Sun
f0096ec292
fix: Fix IsMmapSupported for scalar index ( #38135 )
...
https://github.com/milvus-io/milvus/issues/38134
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-17 20:30:44 +08:00
zhagnlu
9afcc5bc5c
fix:fix incorrect dir operations when create or load inverted index ( #38359 )
...
#37944
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-17 20:06:45 +08:00
aoiasd
93fba1d5ab
fix: access log remove new log not the old and skip rotate empty log ( #38309 )
...
relate: https://github.com/milvus-io/milvus/issues/38293
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-12-17 19:16:44 +08:00
jaime
29e620fa6d
fix: sync task still running after DataNode has stopped ( #38377 )
...
issue: #38319
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-17 18:06:44 +08:00
zhagnlu
d0a7e98a27
fix:remove incorrect assert for delete query ( #38509 )
...
#38472
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-17 17:48:44 +08:00
Zhen Ye
8916fbf122
fix: use new path for streamingnode recovery info ( #38516 )
...
issue: #38399
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-17 17:10:45 +08:00
yihao.dai
d3c174b0f1
enhance: Accelerate observe collection ( #38028 )
...
1. A collection should observe the channel only once.
2. A collection should check the CollectionLoadPercent for updates only
once.
3. Skip saving coll/partition meta if there are no changes, primarily to
accelerate collection observation after recovery.
issue: https://github.com/milvus-io/milvus/issues/37630
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-17 14:14:45 +08:00
yihao.dai
d4dab3c62f
enhance: Reduce segmentManager lock granularity ( #37836 )
...
Use a channel level key lock for segments in segmentManager.
issue: https://github.com/milvus-io/milvus/issues/37633 ,
https://github.com/milvus-io/milvus/issues/37630
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-17 14:12:52 +08:00
Bingyi Sun
dd4f33ae19
fix: Fix chunked segment can not warmup using mmap ( #38492 )
...
issue: #38410
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-17 13:42:45 +08:00
Ted Xu
33aecb0655
fix: build break on target test_cpp under OSX ( #38479 )
...
See: #38434
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-12-17 13:38:45 +08:00
yihao.dai
01fa2f8da5
fix: Fix compile error ( #38510 )
...
/kind improvement
related: #37559
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-17 12:16:19 +08:00
Zhen Ye
afac153c26
enhance: move the lifetime implementation out of server level lifetime ( #38442 )
...
issue: #38399
- move the lifetime implementation of common code out of the server
level lifetime implementation
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-17 11:42:44 +08:00
jaime
28fdbc4e30
enhance: optimize CPU usage for CheckHealth requests ( #35589 )
...
issue: #35563
1. Use an internal health checker to monitor the cluster's health state,
storing the latest state on the coordinator node. The CheckHealth
request retrieves the cluster's health from this latest state on the
proxy sides, which enhances cluster stability.
2. Each health check will assess all collections and channels, with
detailed failure messages temporarily saved in the latest state.
3. Use CheckHealth request instead of the heavy GetMetrics request on
the querynode and datanode
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-17 11:02:45 +08:00
SimFG
2afe2eaf3e
feat: support to replicate collection when the services contains the system tt msg ( #37559 )
...
- issue: #37105
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-17 09:08:46 +08:00
Bingyi Sun
3e2a2f278b
enhance: Handle rust error in c++ ( #38113 )
...
https://github.com/milvus-io/milvus/issues/37930
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-16 19:40:45 +08:00
wei liu
659847c11f
enhance: Remove load task limit in one round ( #38436 )
...
the task limit in assignSegment/assignChannel will works for both load
task and balance task.
this PR remove the load task limit, only limit balance task num in one
round.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-16 19:30:43 +08:00
congqixia
9c8c1b3bb7
enhance: Remove levelZeroMut totally ( #38473 )
...
The level zero mutex could be remove since all operations are guarded by
segment manager mutex
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-16 14:38:45 +08:00
wei liu
40f9db491e
fix: Fix SyncDistribution may cost too much time on retry ( #38454 )
...
issue: #38428
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-16 11:38:44 +08:00
Ted Xu
4919ccf543
enhance: eliminate compile warnings ( #38420 )
...
See: #38435
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-12-16 09:58:43 +08:00
zhagnlu
c3edc85359
fix: fix wrong size of arrow array for zero-copy mode ( #38449 )
...
#37767
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-15 18:14:43 +08:00
zhagnlu
01de0afc4e
enhance: refactor delete mvcc function ( #38066 )
...
#37413
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-15 18:02:43 +08:00
zhagnlu
6ea15265e1
enhance: add file info log when mmap failed. ( #38386 )
...
#37944
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-15 17:36:43 +08:00
SimFG
fa8ac09550
fix: the issue of replicate message exception when the ttMsgEnable config is changed dynamically ( #38178 )
...
- issue: #38177
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-14 23:24:51 +08:00
tinswzy
27229f7907
enhance: refine exists log print with ctx ( #38080 )
...
issue: #35917
Refines exists log print with ctx
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-14 22:36:44 +08:00
sre-ci-robot
1e274384cd
[automated] Update Knowhere Commit ( #38458 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-14 00:54:43 +08:00
Xianhui Lin
b416ff8c6c
enhance: altercollectionfield support mmap_enable convert to mmap.enable ( #38448 )
...
altercollectionfield support mmap_enable convert to mmap.enable
issue: https://github.com/milvus-io/milvus/issues/37436
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-13 20:02:43 +08:00
Chun Han
c1f9158996
fix: search-group-by failed to get data from multi-chunked-segment(##… ( #38383 )
...
related: #38343
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-12-13 16:54:43 +08:00
Ted Xu
3038383e36
fix: UT compile broken under osx ( #38432 )
...
See: #38434
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-12-13 16:24:43 +08:00
zhagnlu
efbfa1cc3e
fix:fix ut failed for debug ( #38384 )
...
#38382
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-13 14:38:43 +08:00
