admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-03 01:12:25 +08:00
Code Issues Packages Projects Releases Wiki Activity
21,801 Commits 138 Branches 188 Tags
Commit Graph

69 Commits

Author SHA1 Message Date
Spade A
f7d9587720
enhance: add tantivy collector for i64 (#39850) 
issue: #39852

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-02-14 15:50:15 +08:00
cai.zhang
9e6e477c5d
fix: Fix modulo for long type (#39722) 
issue: #39640

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-02-11 20:04:46 +08:00
Spade A
547c686027
fix: fix assignment operator in AssertInfo to comparison operator (#39347) 
fix: #39346

Remove the problem line as it's redundant.

---------

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-01-23 14:23:18 +08:00
Spade A
0461ddf776
fix: phrase match does not support offset input (#39338) 
fix: #39337

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-01-16 22:05:01 +08:00
Spade A
032292a432
feat: support phrase match query (#38869) 
The relevant issue: https://github.com/milvus-io/milvus/issues/38930

---------

Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
 2025-01-12 20:24:58 +08:00
congqixia
182cac03e5
enhance: Use bitset or instead of bitwise set (#39037) 
Related to #39003

Copying bitset value bit by bit is slow and CPU heavy, this PR utilizes
bitset operator "|=" to accelerate this procedure

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-01-07 15:02:56 +08:00
smellthemoon
907fc24f85
enhance: support null expr (#38772) 
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2025-01-02 14:16:54 +08:00
Ted Xu
acc8fb7af6
enhance: eliminate compile warnings (part2) (#38535) 
See #38435

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-12-25 15:30:50 +08:00
Chun Han
decdfdae10
fix: growing-groupby-crush(#38533) (#38538) 
related: #38533

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-12-17 21:05:12 +08:00
Ted Xu
4919ccf543
enhance: eliminate compile warnings (#38420) 
See: #38435

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-12-16 09:58:43 +08:00
Chun Han
c1f9158996
fix: search-group-by failed to get data from multi-chunked-segment(##… (#38383) 
related: #38343

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-12-13 16:54:43 +08:00
cqy123456
b14a0c4bf5
fix:GrowingDataGetter get the wrong string data (#38015) 
issue: https://github.com/milvus-io/milvus/issues/37994
2.4 pr: https://github.com/milvus-io/milvus/pull/37995

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-12-12 14:50:42 +08:00
Gao
994fc544e7
enhance: support iterative filter execution (#37363) 
issue: #37360

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-12-11 11:32:44 +08:00
Xianhui Lin
6d0a4fdb31
fix: Fix bug for Search fails with filter expression contains underscore (#38085) 
Enhance the matching for elements within the UnaryRangeArray
https://github.com/milvus-io/milvus/issues/38068

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2024-12-05 10:18:40 +08:00
Bingyi Sun
90064cd47b
fix: Fix variable redeclaration in term filter (#38045) 
https://github.com/milvus-io/milvus/issues/38046

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-12-02 15:10:38 +08:00
zhagnlu
62af24c1a1
fix: change search latency metric from us unit to ms unit (#37806) 
#37805

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-11-24 17:26:33 +08:00
Bingyi Sun
d1596297d9
fix: Fix query failure with inverted index (#37686) 
https://github.com/milvus-io/milvus/issues/37649

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-11-15 10:28:31 +08:00
smellthemoon
3389a6b500
enhance: support null in text match index (#37517) 
#37508

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-11-13 11:08:29 +08:00
Chun Han
2d29dcd30c
enhance:refine group_strict_size parameter(#37482) (#37483) 
related: #37482

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-11-12 09:56:28 +08:00
Bingyi Sun
40ba5a3414
fix: fix chunked segment term filter expression and add ut (#37392) 
issue: https://github.com/milvus-io/milvus/issues/37143

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-11-07 11:04:19 -08:00
Bingyi Sun
bd04cac4b3
fix: fix group by on chunked segment (#37292) 
https://github.com/milvus-io/milvus/issues/37244

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-11-05 17:12:23 +08:00
Bingyi Sun
cd2655c861
fix: fix wrong method is called to fetch variable valid data (#37304) 
issue: https://github.com/milvus-io/milvus/issues/37147

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-11-01 01:52:20 +08:00
zhenshan.cao
63843dce33
fix: Fix conan gdal building problem (#37338) 
issue:https://github.com/milvus-io/milvus/issues/27576

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-10-31 21:04:16 +08:00
Hao Tan
67c4340565
feat: Geospatial Data Type and GIS Function Support for milvus server (#35990) 
issue:https://github.com/milvus-io/milvus/issues/27576

# Main Goals
1. Create and describe collections with geospatial fields, enabling both
client and server to recognize and process geo fields.
2. Insert geospatial data as payload values in the insert binlog, and
print the values for verification.
3. Load segments containing geospatial data into memory.
4. Ensure query outputs can display geospatial data.
5. Support filtering on GIS functions for geospatial columns.

# Solution
1. **Add Type**: Modify the Milvus core by adding a Geospatial type in
both the C++ and Go code layers, defining the Geospatial data structure
and the corresponding interfaces.
2. **Dependency Libraries**: Introduce necessary geospatial data
processing libraries. In the C++ source code, use Conan package
management to include the GDAL library. In the Go source code, add the
go-geom library to the go.mod file.
3. **Protocol Interface**: Revise the Milvus protocol to provide
mechanisms for Geospatial message serialization and deserialization.
4. **Data Pipeline**: Facilitate interaction between the client and
proxy using the WKT format for geospatial data. The proxy will convert
all data into WKB format for downstream processing, providing column
data interfaces, segment encapsulation, segment loading, payload
writing, and cache block management.
5. **Query Operators**: Implement simple display and support for filter
queries. Initially, focus on filtering based on spatial relationships
for a single column of geospatial literal values, providing parsing and
execution for query expressions.
6. **Client Modification**: Enable the client to handle user input for
geospatial data and facilitate end-to-end testing.Check the modification
in pymilvus.

---------

Signed-off-by: tasty-gumi <1021989072@qq.com>
 2024-10-31 20:58:20 +08:00
smellthemoon
b8492498ac
fix: mask with valid data when preCheckOverflow (#37221) 
#37175

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-10-31 10:44:26 +08:00
Bingyi Sun
b81f162f6a
fix: fix several bugs and refactor some codes related with chunked segment (#37168) 
issue: https://github.com/milvus-io/milvus/issues/37147

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-10-28 14:17:30 +08:00
Yinzuo Jiang
3628593d20
feat: Implement custom function module in milvus expr (#36560) 
OSPP 2024 project:
https://summer-ospp.ac.cn/org/prodetail/247410235?list=org&navpage=org

Solutions:

- parser (planparserv2)
    - add CallExpr in planparserv2/Plan.g4
    - update parser_visitor and show_visitor
- grpc protobuf
    - add CallExpr in plan.proto
- execution (`core/src/exec`)
- add `CallExpr` `ValueExpr` and `ColumnExpr` (both logical and
physical) for function call and function parameters
- function factory (`core/src/exec/expression/function`)
    - create a global hashmap when starting milvus (see server.go)
- the global hashmap stores function signatures and their function
pointers, the CallExpr in execution engine can get the function pointer
by function signature.
- custom functions
    - empty(string)
    - starts_with(string, string)
- add cpp/go unittests and E2E tests

closes: #36559

Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
 2024-10-25 15:25:30 +08:00
smellthemoon
6ef014d931
fix: get correct size when sealed segment chunked (#37062) 
#37019

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-10-25 12:01:31 +08:00
Gao
ad2df904c6
fix: correctly set ExecTermArrayVariableInField bitset result (#37111) 
issue: https://github.com/milvus-io/milvus/issues/37110

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-10-24 18:52:02 -07:00
smellthemoon
eb3e4583ec
enhance: all op(Null) is false in expr (#35527) 
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-10-17 21:14:30 +08:00
Bingyi Sun
a75bb85f3a
feat: support chunked column for sealed segment (#35764) 
This PR splits sealed segment to chunked data to avoid unnecessary
memory copy and save memory usage when loading segments so that loading
can be accelerated.

To support rollback to previous version, we add an option
`multipleChunkedEnable` which is false by default.

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-10-12 15:04:52 +08:00
zhagnlu
b1e678dcba
fix: fix json in [] expr bug (#36721) 
#36718

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-10-11 01:11:20 +08:00
zhagnlu
0799d927c6
fix:fix term expr overflow bug (#36525) 
#36520

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-09-26 15:01:14 +08:00
zhagnlu
489087d18b
enhance: refactor executor framework V2 (#35251) 
#32636

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-09-13 20:57:09 +08:00
Jiquan Long
89bf226f0b
feat: support keyword text match (#35923) 
fix: #35922

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-09-10 15:11:08 +08:00
Jiquan Long
11325d9ed5
fix: binary arith expression on inverted index (#35945) 
issue: https://github.com/milvus-io/milvus/issues/35946

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-09-05 20:01:05 +08:00
Zhen Ye
9b96841ae9
fix: wrong construction in evalctx (#35772) 
issue: #35771

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-28 19:21:00 +08:00
Zhen Ye
a773836b89
enhance: optimize milvus core building (#35610) 
issue: #35549,#35611,#35633

- remove milvus_segcore milvus_indexbuilder..., add libmilvus_core
- core building only link once
- move opendal compilation into cmake
- fix odr

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-08-23 12:35:02 +08:00
zhagnlu
c19fe95154
fix: support string match for hybrid and bitmap index (#35294) 
#34841

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-07 09:54:22 +08:00
zhagnlu
16dd53e7cf
enhance: remove timestamp_filter after retrieve (#35207) 
#35226

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-08-02 19:32:46 +08:00
smellthemoon
475c333fa2
enhance: add valid_data in span (#35030) 
#31728

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-08-02 15:40:14 +08:00
zhenshan.cao
aa247f192d
enhance: remove unused code for StorageV2 (#35132) 
issue: https://github.com/milvus-io/milvus/issues/34168

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-08-01 12:08:13 +08:00
congqixia
de8a266d8a
enhance: Enable linux code checker (#35084) 
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 15:53:51 +08:00
zhagnlu
f77f5364b2
fix: disable use_index when some array expr (#34894) 
#34797

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-29 00:17:46 +08:00
zhagnlu
804ec24c02
fix:fix retrieve raw data from bitmap array index (#34848) 
#34795

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-27 01:53:47 +08:00
zhagnlu
3030e4625e
enhance: refactor variable column to reduce memory cost (#33875) 
#33874

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-06-30 20:16:06 +08:00
zhagnlu
03a3f50892
enhance: add skip using array index when some situation (#33947) 
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-06-23 21:26:02 +08:00
zhagnlu
0d7ea8ec42
enhance: Enhance and correct exception module (#33705) 
#33704

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-06-23 21:22:01 +08:00
cqy123456
32f685ff12
enhance: growing segment support mmap (#32633) 
issue: https://github.com/milvus-io/milvus/issues/32984

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-18 14:42:00 +08:00
Jiquan Long
ecf2bcee42
enhance: speed up array-equal operator via inverted index (#33633) 
fix: #33632

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-06-11 14:13:54 +08:00