admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-04 01:42:15 +08:00
Code Issues Packages Projects Releases Wiki Activity
21,816 Commits 138 Branches 188 Tags
Commit Graph

1814 Commits

Author SHA1 Message Date
congqixia
7ccde3300e
fix: Use text_log prefix for TextMatchIndex null offset file (#39935) 
Related to #39933

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-17 20:17:25 +08:00
zhagnlu
8a9f02ef71
enhance: optimize expr performace for some points (#39695) 
1. skip get expr arguments which deserialize proto for every batch
execute.
2. replace unordered_set with sort array that has better performace for
small set.

#39688

Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-02-16 20:32:14 +08:00
sre-ci-robot
61cc22354e
[automated] Update Knowhere Commit (#39898) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2025-02-16 01:32:13 +08:00
Bingyi Sun
b59555057d
feat: support json index (#36750) 
https://github.com/milvus-io/milvus/issues/35528

This PR adds json index support for json and dynamic fields. Now you can
only do unary query like 'a["b"] > 1' using this index. We will support
more filter type later.

basic usage:
```
collection.create_index("json_field", {"index_type": "INVERTED",
    "params": {"json_cast_type": DataType.STRING, "json_path":
'json_field["a"]["b"]'}})
```

There are some limits to use this index:
1. If a record does not have the json path you specify, it will be
ignored and there will not be an error.
2. If a value of the json path fails to be cast to the type you specify,
it will be ignored and there will not be an error.
3. A specific json path can have only one json index.
4. If you try to create more than one json indexes for one json field,
sdk(pymilvus<=2.4.7) may return immediately because of internal
implementation. This will be fixed in a later version.

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-02-15 14:06:15 +08:00
Spade A
f7d9587720
enhance: add tantivy collector for i64 (#39850) 
issue: #39852

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-02-14 15:50:15 +08:00
sre-ci-robot
ba03a435fb
[automated] Update Knowhere Commit (#39878) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2025-02-14 15:18:21 +08:00
aoiasd
24d2bbc441
enhance: unmashall ts msg in dispatcher instead in msgstream (#38656) 
relate: https://github.com/milvus-io/milvus/issues/38655

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-02-14 12:04:13 +08:00
cai.zhang
9e6e477c5d
fix: Fix modulo for long type (#39722) 
issue: #39640

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-02-11 20:04:46 +08:00
Bingyi Sun
c13fc8cd19
enhance: update tantivy version (#39253) 
https://github.com/milvus-io/milvus/issues/39254

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-02-08 14:08:43 +08:00
sre-ci-robot
ba312427f2
[automated] Update Knowhere Commit (#39696) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2025-02-08 01:36:43 +08:00
sparknack
2d9bef44d4
fix: sparse: add inverted_index_algo and dim_max_score_ratio config (#39358) 
issue: #39332

Signed-off-by: Shawn Wang <shawn.wang@zilliz.com>
 2025-02-07 16:40:44 +08:00
Gao
c1794cc490
enhance: update knowhere version and IsAdditionalScalarSupported interface (#39573) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2025-02-05 19:51:10 +08:00
sthuang
c4ae9f4ece
feat: introduce third-party milvus-storage (#39418) 
related: https://github.com/milvus-io/milvus/issues/39173

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2025-01-24 17:21:13 +08:00
Cai Yudong
5730b69e56
feat: Enable more VECTOR_INT8 unittest (#39569) 
Issue: #38666

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2025-01-24 17:03:07 +08:00
zhagnlu
8117d59f85
fix:fix GetValueFromConfig for bool type (#39526) 
#39525

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2025-01-24 16:17:05 +08:00
congqixia
844df76cc0
enhance: Rectify run_clang_format grep command (#39534) 
Previously the grep with regex does not work and failed to match lots of
.cpp files

This PR:
- use "-E" flag to use regex match
- commit the fixed result of current cpp code

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-01-23 17:07:05 +08:00
Spade A
547c686027
fix: fix assignment operator in AssertInfo to comparison operator (#39347) 
fix: #39346

Remove the problem line as it's redundant.

---------

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-01-23 14:23:18 +08:00
Cai Yudong
341d6c1eb7
feat: Update segcore for VECTOR_INT8 (#39415) 
Issue: #38666

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2025-01-21 11:03:03 +08:00
Bingyi Sun
140c5a0a75
enhance: add unit test for string pk (#39329) 
https://github.com/milvus-io/milvus/issues/39107

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-01-20 19:03:04 +08:00
congqixia
45d49df89b
fix: Skip load extra indexes for sorted segment pk field (#39389) 
Related to #39339

Extra indexes can be ignored for most cases since sorted pk column
already provided indexing features

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-01-20 18:13:15 +08:00
Bingyi Sun
cb959cd1f9
enhance: upgrade rust version to 1.83 (#39295) 
#39254

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-01-20 11:15:03 +08:00
Gao
1a680c29e2
fix: correct remote centroids path in clustering compaction (#39398) 
issue: https://github.com/milvus-io/milvus/issues/39353
The path was modified unintentionally, change it back.

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2025-01-20 10:59:10 +08:00
sre-ci-robot
fdb968d0ea
[automated] Update Knowhere Commit (#39420) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2025-01-20 01:17:02 +08:00
Cai Yudong
5b35fc700d
enhance: [skip-e2e] Use template to remove duplicate unittest (#39396) 
Issue: #38666

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2025-01-18 10:33:01 +08:00
congqixia
7cac87caca
fix: Skip erase field if index build on PK field (#39370) 
Related to #39339

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-01-17 20:31:02 +08:00
Cai Yudong
64feeb0e2b
enhance: Rename API GenDataset to GenFieldData in unittest (#39386) 
Issue: #38666

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2025-01-17 15:55:03 +08:00
Ted Xu
9209a70bb6
fix: clang format broken under osx (#38427) 
See: #38434

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-01-17 10:43:03 +08:00
Spade A
0461ddf776
fix: phrase match does not support offset input (#39338) 
fix: #39337

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-01-16 22:05:01 +08:00
Gao
75d7978a18
enhance: pass partition key scalar info if enable for vector mem index (#39123) 
issue: #34332

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2025-01-16 14:33:03 +08:00
Spade A
8c4ba70a4c
fix: enable to build index with single segment (#39233) 
fix https://github.com/milvus-io/milvus/issues/39232

---------

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-01-16 11:01:06 +08:00
congqixia
eb63334312
enhance: Add try-catch and return CStatus for NewCollection (#39279) 
Related to #28795

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-01-15 19:17:01 +08:00
sre-ci-robot
55dcac375c
[automated] Update Knowhere Commit (#39263) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2025-01-15 02:52:59 +08:00
Cai Yudong
5bf1b2b929
feat: Support Int8Vector in go (#38990) 
Issue: #38666

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2025-01-14 20:43:06 +08:00
congqixia
da1b786ef8
enhance: Utilize "find0" in segment.find_first (#39229) 
Related to #39003

Previous PR #39004 has to clone & flip bitset due to bitset does not
support find0 operator. #39176 added this feature so clone & flip could
be removed now.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-01-14 14:14:58 +08:00
Zhen Ye
3e788f0fbd
enhance: record memory size (uncompressed) item for index (#38770) 
issue: #38715

- Current milvus use a serialized index size(compressed) for estimate
resource for loading.
- Add a new field `MemSize` (before compressing) for index to estimate
resource.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-01-14 10:33:06 +08:00
Buqian Zheng
5e38f01e5b
enhance: update knowhere version (#39212) 
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2025-01-14 10:21:05 +08:00
Alexander Guzhva
3447ff7310
enhance: [bitset] extend op_find() to be able to search both 0 and 1 (#39176) 
issue: #39124 

`bitset::find_first()` and `bitset::find_next()` now accept one more
parameter, which allows to search for `0` bit instead of `1` bit

Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
 2025-01-14 09:50:58 +08:00
Bingyi Sun
a00ba861a4
fix: Fix in filter search result is empty if pk type is varchar (#39106) 
https://github.com/milvus-io/milvus/issues/39107

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2025-01-13 16:14:58 +08:00
smellthemoon
accc9e7fbf
fix: fail to get empty index num rows (#39155) 
#39125

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2025-01-13 16:04:58 +08:00
Zhen Ye
5f94954bb4
fix: data race when accessing field_ when retrieving (#39151) 
issue: #39148

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-01-13 11:23:04 +08:00
Buqian Zheng
640a49ffb6
fix: fix chunk cache madvise when sparse raw data is mmaped (#39145) 
instead of marking as not supported,
`ChunkedSparseFloatColumn::DataByteSize` can simply use the impl of
super class.

issue: https://github.com/milvus-io/milvus/issues/39158

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2025-01-13 10:34:57 +08:00
Cai Yudong
2a02bbe3ee
enhance: Use template to remove unittest duplication (#39144) 
Issue: #38666

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2025-01-13 09:58:57 +08:00
Spade A
032292a432
feat: support phrase match query (#38869) 
The relevant issue: https://github.com/milvus-io/milvus/issues/38930

---------

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-01-12 20:24:58 +08:00
Cai Yudong
d6206ad2de
fix: Remove duplicated Macro definition (#39076) 
Issue: #39102

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2025-01-09 15:26:56 +08:00
Spade A
8abf6c9149
fix: build text index when loading field data (#39070) 
fix: https://github.com/milvus-io/milvus/issues/39053
may fix https://github.com/milvus-io/milvus/issues/38644 which could be
caused by https://github.com/milvus-io/milvus/issues/39053

---------

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-01-09 15:24:56 +08:00
Gao
f0dae81494
fix: set iterative filter hint to false when no expr specified (#39033) 
issue: https://github.com/milvus-io/milvus/issues/39013

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2025-01-08 12:56:56 +08:00
Ted Xu
3dc95153b7
fix: build break under debug mode (#38790) 
See #38435

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2025-01-07 17:36:56 +08:00
congqixia
182cac03e5
enhance: Use bitset or instead of bitwise set (#39037) 
Related to #39003

Copying bitset value bit by bit is slow and CPU heavy, this PR utilizes
bitset operator "|=" to accelerate this procedure

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-01-07 15:02:56 +08:00
Cai Yudong
84f8047a86
fix: Fix Milvus build error (#39008) 
Issue: #39005

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2025-01-07 14:22:56 +08:00
Chun Han
3739446a33
enhance: refine array view to optimize memory usage(#38736) (#38808) 
related: #38736

700m data, array_length=10
non-mmap_offsets_uint64: 2.0G
mmap_offsets_uint64: 1.1G
mmap_offsets_uint32: 880MB

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2025-01-07 13:26:55 +08:00