Chun Han
3739446a33
enhance: refine array view to optimize memory usage( #38736 ) ( #38808 )
...
related: #38736
700m data, array_length=10
non-mmap_offsets_uint64: 2.0G
mmap_offsets_uint64: 1.1G
mmap_offsets_uint32: 880MB
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-01-07 13:26:55 +08:00
congqixia
72f5b85c05
enhance: Accelerate find_first by utilizing bitset simd methods ( #39004 )
...
Related to #39003
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-07 10:34:54 +08:00
zhagnlu
8165044b6d
fix: fix query incorrect in case of concurrent delete ( #38991 )
...
#38961
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-01-06 15:14:54 +08:00
Bingyi Sun
f0cddfd160
fix: Fix panic caused by removing directory ( #38622 )
...
https://github.com/milvus-io/milvus/issues/38604
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-01-06 10:54:54 +08:00
congqixia
7128e36eb0
enhance: Use mvcc timestamp as guarantee ts if set ( #38980 )
...
When MvccTimestamp is set, it could be used as guarantee timestamp
directly instead of new ts allocated by scheduler reducing the waiting
time when delegator has tsafe lag
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-05 19:02:54 +08:00
cai.zhang
bd280d6f98
fix: expression with capital AND and OR are not recognized ( #38927 )
...
fix https://github.com/milvus-io/milvus/issues/38864
AND, OR and NOT is not recognized in milvus parser
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-01-05 18:55:00 +08:00
cai.zhang
9672eee78a
enhance: Increase the buffer capacity of notifyIndexChan to support concurrency ( #38957 )
...
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-01-05 18:53:01 +08:00
sre-ci-robot
11bfc93683
[automated] Update Knowhere Commit ( #38993 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-01-04 01:16:53 +08:00
Bingyi Sun
c6a7f48123
enhance: Fix using wrong pool for warmup ( #38941 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-01-03 17:50:54 +08:00
foxspy
af08b5b311
enhance: Update Knowhere version ( #38942 )
...
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2025-01-03 14:28:53 +08:00
Spade A
4245c5bed1
fix: text match panics when enable_match is set be false ( #38950 )
...
fix: https://github.com/milvus-io/milvus/issues/38949
---------
Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
2025-01-03 14:20:55 +08:00
SimFG
d7623ab635
enhance: extract the task step execution process ( #38323 )
...
- issue: #38322
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-01-03 12:36:54 +08:00
Gao
d9ebb70b10
fix: search req transfer correctly ( #38955 )
...
issue: #37899
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2025-01-03 10:04:54 +08:00
Bingyi Sun
aa0a87eda7
fix: Block warmup submit if pool full in sync mode ( #38690 )
...
https://github.com/milvus-io/milvus/issues/38692
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-01-02 15:04:58 +08:00
Patrick Weizhi Xu
d3a5282eaa
enhance: add param for tuning max VARCHAR length and restore limit to 65535 ( #38884 )
...
issue: #38882
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 8e740e004151dd2c11918aad2857c1c8d1bd98f5)
2025-01-02 14:42:53 +08:00
smellthemoon
907fc24f85
enhance: support null expr ( #38772 )
...
#31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-01-02 14:16:54 +08:00
MimoN
accfacf012
fix: [skip e2e]update webui logic ( #38755 )
...
issue: #36621
* update collection detail page logic
* fix data page segments list
* remove useless js file
2024-12-31 18:25:00 +08:00
tinswzy
11f8f4a378
enhance: add intent ctx to rootcoord init ( #38439 )
...
issue: #35917
add intent ctx to rootcoord initialization process
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-31 18:22:53 +08:00
smellthemoon
6b105837b4
enhance: add some apis in Restful ( #38733 )
...
add some apis in Restful #38709
- alter/drop collection properties
- alter/drop index properties
- alter collection field properties
- refresh load
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-31 11:28:51 +08:00
Bingyi Sun
3822819942
enhance: Remove an undefined behavior in index writer ( #38657 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-31 10:42:52 +08:00
cai.zhang
af0315f190
fix: Release compaction task lock when return function ( #38856 )
...
issue: #38851
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-30 22:44:49 +08:00
Xiaofan
cb6eca8e91
fix: drop partition can not be successful if load failed ( #38793 )
...
fix #38649
when partition load failed, the partition drop will also fail due to the
wrong error message
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-12-30 19:42:52 +08:00
cai.zhang
ba3c2e6fb1
fix: Only generate the index_null_offset file when the field support null value ( #38833 )
...
issue: #38832
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2024-12-30 18:02:52 +08:00
Zhen Ye
07c1f43d95
fix: the retrieve plan on heap is used after free when reduce ( #38840 )
...
issue: #38731
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-30 17:50:51 +08:00
Zhen Ye
56c5b66619
fix: the close operation of rmq consumer is not sync ( #38734 )
...
issue: #38399
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-30 16:04:51 +08:00
Reid Chan
dd81a13821
fix: used wrong label for query received_nq metrics ( #37987 )
...
issue: #37982
Signed-off-by: Reid Chan <reidchan@apache.org>
2024-12-30 11:10:52 +08:00
Bingyi Sun
2557e3f2a9
enhance: Initialize field id to avoid negative number ( #38789 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-27 18:00:50 +08:00
Zhen Ye
4ba0ed3178
enhance: speed up streaming barrier timetick ( #38787 )
...
issue: #38399
- use last allocate but not last confirmed id to make barrier.
- move the barrier logic into the timetick allocator.
- try to sync up local allocator and remote allocator when first barrier
check not pass to speed up.
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-27 15:08:50 +08:00
congqixia
19052ef3e5
enhance: Add buffered writer to reduce fwrite syscall ( #38570 )
...
Related to previous PR #38157
If mmapped row is too small, frequent fwrite call still cost too much
cpu time for context switching. This PR add buffered write to avoid this
bad case with extra buffer per variable field.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-27 12:20:50 +08:00
XuanYang-cn
4df444ef25
fix: Aviod add negative missing count ( #38748 )
...
See also: #34665
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-26 18:58:56 +08:00
sthuang
94955e5292
fix: expand privilege group when list policy in rootcoord ( #38758 )
...
related: https://github.com/milvus-io/milvus/issues/38757
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-26 15:38:50 +08:00
Patrick Weizhi Xu
85f462be1a
enhance: speed up search iterator stage 1 ( #37947 )
...
issue: #37548
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-12-26 10:32:49 +08:00
wei liu
f49d618382
fix: Querycoord will trigger unexpected balance task after restart ( #38630 )
...
issue: #38606
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-25 19:30:48 +08:00
aoiasd
bc15ad24f2
fix: sealed segment get empty index params when brute force search for bm25 ( #38707 )
...
relate: https://github.com/milvus-io/milvus/issues/38236
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-12-25 19:06:51 +08:00
Gao
363d7f31ef
fix: report error when hints not supported ( #38717 )
...
issue: #38705
---------
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-12-25 19:02:56 +08:00
aoiasd
c7ea09a8be
enhance: return exception type name when segcore return unkonwn exception ( #38326 )
...
relate: https://github.com/milvus-io/milvus/issues/38265
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-12-25 18:58:50 +08:00
wei liu
25f0c82ceb
fix: Fix update loading collection's load config doesn't work ( #38595 )
...
issue: #38594
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-25 18:02:51 +08:00
sthuang
6bc799061e
fix: fix privilege group list and list collections ( #38684 )
...
related: #37031
* built-in privilege group privileges in listPrivilegeGroups() should be
the same as in milvus.yaml
* collections granted by collection level built-in privilege group
should be list in showCollections()
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-25 18:00:51 +08:00
wei liu
9c3f59dbbe
fix: Prevent balancer from overloading the same QueryNode ( #38719 )
...
issue: #38718
The balancer calculates the workload of executing tasks as an ongoing
score for target nodes. However, a logic issue arises when
GetSegmentTaskDelta or GetChannelTaskDelta is called with
collectionID=-1, which incorrectly returns zero.
Due to the incorrect global score, the executing task's workload is not
properly reflected for each collection. Consequently, each collection
submits its own balance task, leading to the balancer assigning
excessive tasks to the same QueryNode.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-25 16:36:49 +08:00
Ted Xu
acc8fb7af6
enhance: eliminate compile warnings (part2) ( #38535 )
...
See #38435
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-12-25 15:30:50 +08:00
Zhen Ye
5395ec19ad
enhance: add mem size for index file and fallback to multiply with serialized size ( #38716 )
...
issue: #38715
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-25 14:46:49 +08:00
XuanYang-cn
c731357538
enhance: Add missing delete metrics ( #38634 )
...
Add 2 counter metrics:
- Total delete entries from deltalog:
milvus_datanode_compaction_delete_count
- Total missing deletes: milvus_datanode_compaction_missing_delete_count
See also: #34665
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-25 10:24:50 +08:00
Zhen Ye
285289d5b0
enhance: implement kafka for wal ( #38598 )
...
issue: #38399
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-25 10:22:50 +08:00
Zhen Ye
05d9f8562f
fix: fix complie error generated by conflict pr ( #38729 )
...
issue: #38399
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-25 10:14:49 +08:00
Zhen Ye
69a9fd6ead
enhance: enable rmq for streaming ( #38669 )
...
issue: #38399
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-24 20:24:48 +08:00
yihao.dai
2b53b0905e
fix: Fix 0 read count during import ( #38694 )
...
issue: https://github.com/milvus-io/milvus/issues/38693
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-24 20:06:49 +08:00
MimoN
c458dec206
fix: [skip e2e]update webui collection and query pages ( #38700 )
...
issue: #36621
* enhance collection detail page list columns
* enhance query page segments state detail logic
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2024-12-24 17:36:48 +08:00
congqixia
636e107e44
fix: Remove sync task after finished ( #38681 )
...
Related to #38680
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-24 13:34:48 +08:00
Xianhui Lin
d718bbb4de
fix: fix altercollectionfield interceptor dbname ( #38662 )
...
fix altercollectionfield interceptor dbname
issue: https://github.com/milvus-io/milvus/issues/37436
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-23 18:24:48 +08:00
cai.zhang
bb5f38e574
enhance: Return collection not loaded rather than not found on querynode ( #38593 )
...
issue: #38586
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-23 14:36:48 +08:00
