62 Commits

Author SHA1 Message Date
cai.zhang
ad68bc6204
fix: [2.4] Decode unicode for json key in expression (#38652)
issue: #38626

master pr: #38651

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-23 13:54:51 +08:00
cai.zhang
651a56e3dd
enhance: [2.4]Update the template expression proto to improve transmission efficiency (#37485)
issue: #36672 

master pr: #37484

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-07 12:12:25 +08:00
cai.zhang
4ae5337343
enhance: [2.4] Refine error message for contains array (#37443)
issue: #36221 

master pr: #37383

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-07 10:40:25 +08:00
cai.zhang
4fb86eb17d
fix: [2.4] Fix the bug where some expressions do not correctly parse the value (#37342)
issue: #37274

master pr: #37341

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-03 18:42:23 +08:00
cai.zhang
05c40522ce
enhance: [cherry-pick ]Enhance the expression template to support AND and OR operations (#37217)
issue: #36672

master pr: #37033

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-10-29 15:38:40 +08:00
cai.zhang
9c0f59488a
feat: [cherry-pick]The expression supports filling elements through templates (#37058)
issue: #36672 

master pr: #37033 

milvus-proto pr: https://github.com/milvus-io/milvus-proto/pull/332

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-10-28 15:17:30 +08:00
cai.zhang
f7302a78fb
enhance: [cherry-pick]Convert unincode to ascii to improving expression parsing efficiency (#36676)
issue: #36672 

master pr: #36675

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-10-09 09:23:24 +08:00
congqixia
ab31728b2c
enhance: [2.4] Check ANN field loaded in proxy (#36172) (#36194)
Cherry-pick from master
pr: #36172
Related to #35415

Currently ANN field is loaded in Parital load is not performed in proxy,
this cause error message not clear and no error returned when collection
is empty

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-12 16:05:08 +08:00
congqixia
c8926aee23
enhance: [2.4] Support dynamic field in SchemaHelper (#35461) (#35469)
Cherry-pick from master
pr: #35461
Related to #35415

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-15 10:02:54 +08:00
Jiquan Long
a00688c00a
fix: comparision operations between incompatible operands (#35264) (#35307)
fix: https://github.com/milvus-io/milvus/issues/34139
pr: https://github.com/milvus-io/milvus/pull/35264

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-08-07 11:24:16 +08:00
jaime
597362e00c
fix: memory leak while parsing query plan (#34932)
issue: #34930
pr: #34931

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-24 14:47:43 +08:00
Xiaofan
36f1ea93a5
enhance: optimize plan parser pool to avoid unnessary recycle (#32869)
fix #32868
plan parser takes too much cpu on high qps,this pr try to avoid create
lexer and parser too freequent

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-11 10:51:31 +08:00
Cai Yudong
00438f408f
enhance: Unify data type check APIs for go (#31887)
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-04-07 14:27:22 +08:00
Chun Han
c3264ca3e3
feat: support segment pruner (#31003)
related: #30376
2024-03-22 13:57:06 +08:00
Buqian Zheng
3c80083f51
feat: [Sparse Float Vector] add sparse vector support to milvus components (#30630)
add sparse float vector support to different milvus components,
including proxy, data node to receive and write sparse float vectors to
binlog, query node to handle search requests, index node to build index
for sparse float column, etc.

https://github.com/milvus-io/milvus/issues/29419

---------

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-03-13 14:32:54 -07:00
cai.zhang
de2c95d00c
enhance: Constraint dynamic field as key-value format (#31183)
issue: #31051

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-12 12:45:03 +08:00
cai.zhang
6a83f16871
feat: Support for multiple forms of JSON (#31052)
issue: #31051

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-11 19:55:02 +08:00
congqixia
9b3005f1be
enhance: Avoid create schema helper for each read task (#30981)
See also #30806

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-04 19:39:00 +08:00
cai.zhang
1aa97a5c21
enhance: Support more relational operators for binary expressions (#30902)
issue: #30677

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-01 16:57:00 +08:00
Jiquan Long
dcfc35316f
fix: underscore matching not work (#30826)
issue: https://github.com/milvus-io/milvus/issues/30721

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-02-29 10:41:00 +08:00
aoiasd
c863b82476
enhance: Return parse expression failed error with reason (#30548)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-02-20 16:02:52 +08:00
Jiquan Long
e549148a19
enhance: full-support for wildcard pattern matching (#30288)
issue: #29988 
This pr adds full-support for wildcard pattern matching from end to end.
Before this pr, the users can only use prefix match in their expression,
for example, "like 'prefix%'". With this pr, more flexible syntax can be
combined.

To do so, this pr makes these changes:
- 1. support regex query both on index and raw data;
- 2. translate the pattern matching to regex query, so that it can be
handled by the regex query logic;
- 3. loose the limit of the expression parsing, which allows general
pattern matching syntax;

With the support of regex query in segcore backend, we can also add
mysql-like `REGEXP` syntax later easily.

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-02-01 12:37:04 +08:00
Xu Tong
e429965f32
Add float16 approve for multi-type part (#28427)
issue:https://github.com/milvus-io/milvus/issues/22837

Add bfloat16 vector, add the index part of float16 vector.

Signed-off-by: Writer-X <1256866856@qq.com>
2024-01-11 15:48:51 +08:00
cai.zhang
31f442915b
fix: Fix bug for parsing expression that include quotes (#28416)
issue: #28365 
Fix bug for parsing error when a string enclosed in single quotes in an
expression contains multiple double quotes.
such as:
```
expr = "tag == '\"blue\"'"
```

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-23 17:18:32 +08:00
Bingyi Sun
d7145e2c06
enhance: Update golangci_lint version (#28535)
Update golangci lint and fix some warnings

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-11-21 10:04:21 +08:00
yihao.dai
f9c630247d
Construct plan directly when search with vector output (#27928)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-26 19:30:10 +08:00
congqixia
5d558623fe
Add revive sub-lints and fix existing problems (#27495)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-07 20:53:38 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
cai.zhang
a362bb1457
Support array datatype (#26369)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-09-19 14:23:23 +08:00
congqixia
cc9974979f
Add staticcheck linter and fix existing problems (#27174)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-19 10:05:22 +08:00
Xu Tong
9166011c4a
Add float16 vector (#25852)
Signed-off-by: Writer-X <1256866856@qq.com>
2023-09-08 10:03:16 +08:00
cai.zhang
82b0e39abe
Fix bug for concurrent parsing expr with strings (#26721)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-08-31 17:39:02 +08:00
Jiquan Long
36e29ef859
Auto generate parser code (#26716)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-31 11:13:01 +08:00
Jiquan Long
d9aa8ff545
Fix logical expression (#26513)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-22 15:44:22 +08:00
Jiquan Long
2a517d2da1
Fix expression incompatible between parser and executor (#26493)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-21 19:36:20 +08:00
cai.zhang
94846995bf
Refine error message for field not exist (#26330)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-08-21 11:52:20 +08:00
cai.zhang
90b12d04b4
Don't panic when receive unsupported expr (#26414)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-08-17 15:12:17 +08:00
cai.zhang
a0198ce8ae
Support json contains feature (#25384)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-08-11 17:09:30 +08:00
Jiquan Long
6578a27fe7
Support empty expression (#26153)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-10 08:37:15 +08:00
cai.zhang
3d78a452d7
Support escape string (#24848)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-07-25 10:29:01 +08:00
cai.zhang
f12574aaf3
Support JSON_CONTAINS or json_contains (#24814)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-06-13 10:50:37 +08:00
cai.zhang
853b7d0054
Support json contains for json field (#24720)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-06-09 16:10:36 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
xige-16
732fe54775
Support partition Key (#24047)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-06 10:24:34 +08:00
cai.zhang
dc02c5b064
Fix bug for supporting single quotes (#24464)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-29 09:55:27 +08:00
cai.zhang
6804bda8f5
Support single quotes with string (#24386)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-25 15:47:26 +08:00
cai.zhang
d16e18fd34
Add output fields for search/query results (#24302)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-23 10:19:26 +08:00
cai.zhang
008285f849
Support dynamic schema for create collection (#24176)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-18 09:33:24 +08:00
cai.zhang
ccd685013a
Json key must be enclosed in double quotes (#24113)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-16 16:19:22 +08:00
cai.zhang
9a0a41d2a8
Support access json key without quotation marks (#24062)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-14 22:51:21 +08:00