admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-01 16:35:29 +08:00
Code Issues Packages Projects Releases Wiki Activity
20,584 Commits 138 Branches 188 Tags
Commit Graph

9290 Commits

Author SHA1 Message Date
Chun Han
bfd9d86fe9
feat: support groupby size on go-layer(#33544) (#33845) 
related: #33544

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-27 14:21:00 +08:00
aoiasd
fe83805d56
fix: loss data bug for deprecated querynode DoubleBuffer (#35128) 
relate: https://github.com/milvus-io/milvus/issues/31548

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-08-27 14:10:59 +08:00
XuanYang-cn
f12e368a76
fix: Fill nil schema so that Milvus can watch channel for those upgraded from 2.2 to 2.4 #35695 (#35694) 
See also: [#35701 ](https://github.com/milvus-io/milvus/issues/35701)

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-08-27 10:36:59 +08:00
SimFG
3e1052f889
enhance: use the msg position obj when getting replicate channel position (#35606) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-27 10:28:59 +08:00
jaime
7d3c0d748c
fix: the incorrect number of partitions in rootcoord meta (#35600) 
issue: #35698

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-08-26 15:20:58 +08:00
XuanYang-cn
82743c5c50
fix: Clear channelcp meta and metrics ASAP (#35658) 
See also: #35588

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-08-26 12:22:57 +08:00
sre-ci-robot
6ddfd02f01
[automated] Update Knowhere Commit (#35688) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-08-26 01:04:57 +08:00
Patrick Weizhi Xu
4236c3bafb
enhance: enable MV on binary vectors (#35668) 
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-08-25 16:27:09 +08:00
smellthemoon
098c10922b
enhance: add some check when create collection (#35629) 
not support nullable==true in vector type. check it when create
collection.

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-08-25 16:23:05 +08:00
Xiaofan
0dc5e89007
enhance: reduce the log level of frequent log (#35652) 
fix #35651

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-08-25 16:20:57 +08:00
cai.zhang
615a653988
fix: Fix offset out of range for creating Trie index (#35553) 
issue: #35550

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-25 15:50:57 +08:00
yihao.dai
f2b83d316b
enhance: Support memory mode chunk cache (#35347) 
Chunk cache supports loading raw vectors into memory.

issue: https://github.com/milvus-io/milvus/issues/35273

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-25 15:42:58 +08:00
Xiaofan
50fcfe8ef1
enhance: add nan and inf check (#35683) 
fix #35594
add float check on files

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-08-25 15:22:57 +08:00
zhagnlu
42f7800b5b
enhance: add bitmap offset cache to speed up retrieve raw data (#35498) 
#35458

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-24 01:40:58 +08:00
Zhen Ye
75da36d1aa
enhance: enable asan for milvus (#35627) 
issue: #35626

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-23 21:06:58 +08:00
congqixia
59387f0b3a
fix: Use correct release func when datanode unwatch a channel (#35655) 
See also #35654

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-23 14:09:03 +08:00
Gao
e8e3544a11
enhance: add hit segment num metrics for queryHook (#35577) 
issue: #35576

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-08-23 12:49:02 +08:00
Zhen Ye
a773836b89
enhance: optimize milvus core building (#35610) 
issue: #35549,#35611,#35633

- remove milvus_segcore milvus_indexbuilder..., add libmilvus_core
- core building only link once
- move opendal compilation into cmake
- fix odr

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-23 12:35:02 +08:00
congqixia
ab532ae199
enhance: Add back BF lazy load logic for datanode watch channel (#35646) 
Add back lazy loading statslog when watch dml channel on datanode.

Related to #22994 #27675

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-22 19:42:57 +08:00
wei liu
5c245d51c4
enhance: Refresh proxy cache after restore rbac meta (#35635) 
issue: #35443

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-22 19:09:01 +08:00
congqixia
9fff07936a
fix: Re-read value after once initialization (#35642) 
Related to #35641
See also #35271

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-22 17:24:58 +08:00
congqixia
582d2eec79
enhance: Move datanode/indexnode manager to session pkg (#35634) 
Related to #28861

Move session manager, worker manager to session package. Also renaming
each manager to corresponding node name(datanode, indexnode).

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-22 16:02:56 +08:00
zhagnlu
3107701fe8
enhance: optimize retrieve on dynamic field (#35580) 
#35514

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-08-22 14:24:56 +08:00
congqixia
c992a61a23
enhance: Separate allocator pkg in datacoord (#35622) 
Related to #28861

Move allocator interface and implementation into separate package. Also
update some unittest logic.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-22 10:06:56 +08:00
presburger
024eccbde0
enhance: add pkg-config for knowhere (#35433) 
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
 2024-08-22 09:56:56 +08:00
OxalisCu
ed4eaffc9d
enhance: add csv support for bulkinsert (#34938) 
See this issue for details: #34937

---------

Signed-off-by: OxalisCu <2127298698@qq.com>
 2024-08-21 17:47:01 +08:00
SimFG
731d45abbe
enhance: provide more general configuration to control mmap behavior (#35359) 
- issue: #35273

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-21 00:22:54 +08:00
Ted Xu
41646c8439
feat: integrate new deltalog format (#35522) 
See #34123

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-08-20 19:06:56 +08:00
aoiasd
bc2bbdc82f
enhance: Forbid access log print error message with line break (#34491) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-08-20 17:36:55 +08:00
smellthemoon
80a7c78f28
enhance: import supports null in parquet and json formats (#35558) 
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-08-20 16:50:55 +08:00
congqixia
2fbc628994
feat: Support field partial load collection (#35416) 
Related to #35415

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-20 16:49:02 +08:00
Chun Han
031ee6f155
enhance: support httpv1/v2 throttle and add it for httpV2(#35350) (#35470) 
related: #35350

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-20 16:16:55 +08:00
congqixia
3491608256
fix: Match int8_t and int16_t in Array::get_data (#35579) 
Related to #35578

Previously int16/int8 bitmap index may read int32 array as int16, which
may cause build index with half of the data(if array is full) and half
zeros. This causes BITMAP index lost information.

This PR matches int8_t & int16_t while `get_data` when building index.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-20 16:10:56 +08:00
wei liu
e09dc3be58
enhance: Mark query node as read only after suspend (#35492) 
issue: #34985 #35493
after querynode has been suspended, it's not allow to load
segment/channel on it, which means the node is read only. to be
compatible with resource group design, after query node has been
suspend, we remove it from it's original resource group, make it a read
only query node in replica. then two things will happens:
1. it's original resource group will be lacking of query nodes, query
coord will assign new node to it.
2. querycoord will try to move out all segments/channels after querynode
has been suspended

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-20 14:02:54 +08:00
nish112022
3948bd4e79
fix: Added check for validating varchar,array max length (#35499) 
issue : https://github.com/milvus-io/milvus/issues/34150

This is for numpy,parquet,json readers.

---------

Signed-off-by: Nischay Yadav <nischay.yadav@ibm.com>
 2024-08-20 11:42:55 +08:00
wei liu
22ced010cd
enhance: make configure load param feature be compatible with old sdk (#35520) 
issue: #31570 #35521

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-20 10:30:55 +08:00
Zhen Ye
4d69898cb2
enhance: support single pchannel level transaction (#35289) 
issue: #33285

- support transaction on single wal.
- last confirmed message id can still be used when enable transaction.
- add fence operation for segment allocation interceptor.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-19 21:22:56 +08:00
yihao.dai
227ecd3901
enhance: Remove the check for channel cp lag when generating compaction plan (#35383) 
issue: https://github.com/milvus-io/milvus/issues/35382

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-19 19:40:55 +08:00
Chun Han
337e065902
fix: querynode hang when failing to allocate disk space for mmap(#35184) (#35187) 
related: #35184

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-08-19 15:30:55 +08:00
wei liu
14ef88f423
fix: Fix proxy panic after Import related api failed (#35540) 
issue: #35539

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-19 15:22:55 +08:00
XuanYang-cn
967f38672a
enhance: Add integration tests for l0 (#35429) 
See also: #34796

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-08-19 10:56:54 +08:00
smellthemoon
80dbe87759
enhance: support null value in index (#35238) 
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-08-16 15:30:54 +08:00
congqixia
f87af9bc54
enhance: Exclude L0 segment from readable snapshot (#35507) 
L0 segments now do not contain insert data and may cause confusion for
query hook optimizer if counted as sealed segment number.

This PR add segment level flag in segment entry and exclude L0 segments
while get readable segment snaphsot

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-16 15:28:53 +08:00
CharlesFeng
5e9097c2dd
fix: err degenerated to a new variable (#35413) 
https://github.com/milvus-io/milvus/issues/35412

Signed-off-by: fengjun2016 <jornfeng@gmail.com>
 2024-08-16 14:24:54 +08:00
wei liu
a570567644
enhance: Enable ReadOnly/ReadWrite/Admin Privilege Group to simplify RBAC grant progress (#35472) 
issue: #35471

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-16 14:18:54 +08:00
Buqian Zheng
f4a91e135b
enhance: Allow empty sparse row (#34700) 
issue: #29419

* If a sparse vector with 0 non-zero value is inserted, no ANN search on
this sparse vector field will return it as a result. User may retrieve
this row via scalar query or ANN search on another vector field though.
* If the user uses an empty sparse vector as the query vector for a ANN
search, no neighbor will be returned.

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-08-16 14:14:54 +08:00
wei liu
1d49358f82
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup (#35444) 
issue: #35443

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-16 10:10:53 +08:00
Alexander Guzhva
b896143965
enhance: Improve bitset performance for AVX512 (#35479) 
see #35478

optimized functions take 20%+ less time to run

Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
 2024-08-16 07:44:53 +08:00
cai.zhang
1bbf7a3c0e
enhance: Optimize the use of locks and avoid double flush clustering buffer writer (#35486) 
issue: #35436

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-16 02:24:58 +08:00
congqixia
fe20366b5c
enhance: Remove duplicated schema helper creation in proxy (#35489) 
Related to PRs of #35415

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-15 19:18:53 +08:00