admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-01 16:35:29 +08:00
Code Issues Packages Projects Releases Wiki Activity
21,725 Commits 138 Branches 188 Tags
Commit Graph

1471 Commits

Author SHA1 Message Date
sthuang
c4ae9f4ece
feat: introduce third-party milvus-storage (#39418) 
related: https://github.com/milvus-io/milvus/issues/39173

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2025-01-24 17:21:13 +08:00
Cai Yudong
5730b69e56
feat: Enable more VECTOR_INT8 unittest (#39569) 
Issue: #38666

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2025-01-24 17:03:07 +08:00
zhagnlu
8117d59f85
fix:fix GetValueFromConfig for bool type (#39526) 
#39525

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-01-24 16:17:05 +08:00
congqixia
844df76cc0
enhance: Rectify run_clang_format grep command (#39534) 
Previously the grep with regex does not work and failed to match lots of
.cpp files

This PR:
- use "-E" flag to use regex match
- commit the fixed result of current cpp code

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-01-23 17:07:05 +08:00
Spade A
547c686027
fix: fix assignment operator in AssertInfo to comparison operator (#39347) 
fix: #39346

Remove the problem line as it's redundant.

---------

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-01-23 14:23:18 +08:00
Cai Yudong
341d6c1eb7
feat: Update segcore for VECTOR_INT8 (#39415) 
Issue: #38666

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2025-01-21 11:03:03 +08:00
congqixia
45d49df89b
fix: Skip load extra indexes for sorted segment pk field (#39389) 
Related to #39339

Extra indexes can be ignored for most cases since sorted pk column
already provided indexing features

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-01-20 18:13:15 +08:00
Gao
1a680c29e2
fix: correct remote centroids path in clustering compaction (#39398) 
issue: https://github.com/milvus-io/milvus/issues/39353
The path was modified unintentionally, change it back.

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2025-01-20 10:59:10 +08:00
congqixia
7cac87caca
fix: Skip erase field if index build on PK field (#39370) 
Related to #39339

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-01-17 20:31:02 +08:00
Spade A
0461ddf776
fix: phrase match does not support offset input (#39338) 
fix: #39337

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-01-16 22:05:01 +08:00
Gao
75d7978a18
enhance: pass partition key scalar info if enable for vector mem index (#39123) 
issue: #34332

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2025-01-16 14:33:03 +08:00
Spade A
8c4ba70a4c
fix: enable to build index with single segment (#39233) 
fix https://github.com/milvus-io/milvus/issues/39232

---------

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-01-16 11:01:06 +08:00
congqixia
eb63334312
enhance: Add try-catch and return CStatus for NewCollection (#39279) 
Related to #28795

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-01-15 19:17:01 +08:00
Cai Yudong
5bf1b2b929
feat: Support Int8Vector in go (#38990) 
Issue: #38666

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2025-01-14 20:43:06 +08:00
congqixia
da1b786ef8
enhance: Utilize "find0" in segment.find_first (#39229) 
Related to #39003

Previous PR #39004 has to clone & flip bitset due to bitset does not
support find0 operator. #39176 added this feature so clone & flip could
be removed now.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-01-14 14:14:58 +08:00
Zhen Ye
3e788f0fbd
enhance: record memory size (uncompressed) item for index (#38770) 
issue: #38715

- Current milvus use a serialized index size(compressed) for estimate
resource for loading.
- Add a new field `MemSize` (before compressing) for index to estimate
resource.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-01-14 10:33:06 +08:00
Alexander Guzhva
3447ff7310
enhance: [bitset] extend op_find() to be able to search both 0 and 1 (#39176) 
issue: #39124 

`bitset::find_first()` and `bitset::find_next()` now accept one more
parameter, which allows to search for `0` bit instead of `1` bit

Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
 2025-01-14 09:50:58 +08:00
Bingyi Sun
a00ba861a4
fix: Fix in filter search result is empty if pk type is varchar (#39106) 
https://github.com/milvus-io/milvus/issues/39107

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-01-13 16:14:58 +08:00
smellthemoon
accc9e7fbf
fix: fail to get empty index num rows (#39155) 
#39125

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2025-01-13 16:04:58 +08:00
Zhen Ye
5f94954bb4
fix: data race when accessing field_ when retrieving (#39151) 
issue: #39148

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-01-13 11:23:04 +08:00
Buqian Zheng
640a49ffb6
fix: fix chunk cache madvise when sparse raw data is mmaped (#39145) 
instead of marking as not supported,
`ChunkedSparseFloatColumn::DataByteSize` can simply use the impl of
super class.

issue: https://github.com/milvus-io/milvus/issues/39158

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2025-01-13 10:34:57 +08:00
Cai Yudong
2a02bbe3ee
enhance: Use template to remove unittest duplication (#39144) 
Issue: #38666

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2025-01-13 09:58:57 +08:00
Spade A
032292a432
feat: support phrase match query (#38869) 
The relevant issue: https://github.com/milvus-io/milvus/issues/38930

---------

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-01-12 20:24:58 +08:00
Cai Yudong
d6206ad2de
fix: Remove duplicated Macro definition (#39076) 
Issue: #39102

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2025-01-09 15:26:56 +08:00
Spade A
8abf6c9149
fix: build text index when loading field data (#39070) 
fix: https://github.com/milvus-io/milvus/issues/39053
may fix https://github.com/milvus-io/milvus/issues/38644 which could be
caused by https://github.com/milvus-io/milvus/issues/39053

---------

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-01-09 15:24:56 +08:00
Gao
f0dae81494
fix: set iterative filter hint to false when no expr specified (#39033) 
issue: https://github.com/milvus-io/milvus/issues/39013

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2025-01-08 12:56:56 +08:00
Ted Xu
3dc95153b7
fix: build break under debug mode (#38790) 
See #38435

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-01-07 17:36:56 +08:00
congqixia
182cac03e5
enhance: Use bitset or instead of bitwise set (#39037) 
Related to #39003

Copying bitset value bit by bit is slow and CPU heavy, this PR utilizes
bitset operator "|=" to accelerate this procedure

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-01-07 15:02:56 +08:00
Cai Yudong
84f8047a86
fix: Fix Milvus build error (#39008) 
Issue: #39005

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2025-01-07 14:22:56 +08:00
Chun Han
3739446a33
enhance: refine array view to optimize memory usage(#38736) (#38808) 
related: #38736

700m data, array_length=10
non-mmap_offsets_uint64: 2.0G
mmap_offsets_uint64: 1.1G
mmap_offsets_uint32: 880MB

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-01-07 13:26:55 +08:00
congqixia
72f5b85c05
enhance: Accelerate find_first by utilizing bitset simd methods (#39004) 
Related to #39003

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-01-07 10:34:54 +08:00
zhagnlu
8165044b6d
fix: fix query incorrect in case of concurrent delete (#38991) 
#38961

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-01-06 15:14:54 +08:00
Bingyi Sun
f0cddfd160
fix: Fix panic caused by removing directory (#38622) 
https://github.com/milvus-io/milvus/issues/38604

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-01-06 10:54:54 +08:00
Spade A
4245c5bed1
fix: text match panics when enable_match is set be false (#38950) 
fix: https://github.com/milvus-io/milvus/issues/38949

---------

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-01-03 14:20:55 +08:00
Bingyi Sun
aa0a87eda7
fix: Block warmup submit if pool full in sync mode (#38690) 
https://github.com/milvus-io/milvus/issues/38692

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-01-02 15:04:58 +08:00
smellthemoon
907fc24f85
enhance: support null expr (#38772) 
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2025-01-02 14:16:54 +08:00
cai.zhang
ba3c2e6fb1
fix: Only generate the index_null_offset file when the field support null value (#38833) 
issue: #38832

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2024-12-30 18:02:52 +08:00
congqixia
19052ef3e5
enhance: Add buffered writer to reduce fwrite syscall (#38570) 
Related to previous PR #38157

If mmapped row is too small, frequent fwrite call still cost too much
cpu time for context switching. This PR add buffered write to avoid this
bad case with extra buffer per variable field.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-12-27 12:20:50 +08:00
Patrick Weizhi Xu
85f462be1a
enhance: speed up search iterator stage 1 (#37947) 
issue: #37548

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-12-26 10:32:49 +08:00
aoiasd
bc15ad24f2
fix: sealed segment get empty index params when brute force search for bm25 (#38707) 
relate: https://github.com/milvus-io/milvus/issues/38236

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-12-25 19:06:51 +08:00
Gao
363d7f31ef
fix: report error when hints not supported (#38717) 
issue: #38705

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-12-25 19:02:56 +08:00
aoiasd
c7ea09a8be
enhance: return exception type name when segcore return unkonwn exception (#38326) 
relate: https://github.com/milvus-io/milvus/issues/38265

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-12-25 18:58:50 +08:00
Ted Xu
acc8fb7af6
enhance: eliminate compile warnings (part2) (#38535) 
See #38435

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-12-25 15:30:50 +08:00
zhagnlu
8fcb33c21d
fix:fix delete record assert failed (#38580) 
#38472

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-12-19 18:22:47 +08:00
Zhen Ye
b537a72309
fix: interted index out of range (#38577) 
issue: #38546, #38486

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-12-19 15:20:47 +08:00
zhagnlu
87056be748
fix: fix snapshot or size when query (#38549) 
#38472

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-12-18 16:42:45 +08:00
Chun Han
decdfdae10
fix: growing-groupby-crush(#38533) (#38538) 
related: #38533

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-12-17 21:05:12 +08:00
Bingyi Sun
f0096ec292
fix: Fix IsMmapSupported for scalar index (#38135) 
https://github.com/milvus-io/milvus/issues/38134

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-12-17 20:30:44 +08:00
zhagnlu
9afcc5bc5c
fix:fix incorrect dir operations when create or load inverted index (#38359) 
#37944

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-12-17 20:06:45 +08:00
zhagnlu
d0a7e98a27
fix:remove incorrect assert for delete query (#38509) 
#38472

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-12-17 17:48:44 +08:00