admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2025-12-28 22:45:26 +08:00
Code Issues Packages Projects Releases Wiki Activity
milvus/internal/parser/planparserv2/generated/Plan.interp
Spade A f6f716bcfd
feat: impl StructArray -- support embedding searches embeddings in embedding list with element level filter expression (#45830) 
issue: https://github.com/milvus-io/milvus/issues/42148

For a vector field inside a STRUCT, since a STRUCT can only appear as
the element type of an ARRAY field, the vector field in STRUCT is
effectively an array of vectors, i.e. an embedding list.
Milvus already supports searching embedding lists with metrics whose
names start with the prefix MAX_SIM_.

This PR allows Milvus to search embeddings inside an embedding list
using the same metrics as normal embedding fields. Each embedding in the
list is treated as an independent vector and participates in ANN search.

Further, since STRUCT may contain scalar fields that are highly related
to the embedding field, this PR introduces an element-level filter
expression to refine search results.
The grammar of the element-level filter is:

element_filter(structFieldName, $[subFieldName] == 3)

where $[subFieldName] refers to the value of subFieldName in each
element of the STRUCT array structFieldName.

It can be combined with existing filter expressions, for example:

"varcharField == 'aaa' && element_filter(struct_field, $[struct_int] ==
3)"

A full example:
```
struct_schema = milvus_client.create_struct_field_schema()
struct_schema.add_field("struct_str", DataType.VARCHAR, max_length=65535)
struct_schema.add_field("struct_int", DataType.INT32)
struct_schema.add_field("struct_float_vec", DataType.FLOAT_VECTOR, dim=EMBEDDING_DIM)

schema.add_field(
    "struct_field",
    datatype=DataType.ARRAY,
    element_type=DataType.STRUCT,
    struct_schema=struct_schema,
    max_capacity=1000,
)
...

filter = "varcharField == 'aaa' && element_filter(struct_field, $[struct_int] == 3 && $[struct_str] == 'abc')"
res = milvus_client.search(
    COLLECTION_NAME,
    data=query_embeddings,
    limit=10,
    anns_field="struct_field[struct_float_vec]",
    filter=filter,
    output_fields=["struct_field[struct_int]", "varcharField"],
)

```
TODO:
1. When an `element_filter` expression is used, a regular filter
expression must also be present. Remove this restriction.
2. Implement `element_filter` expressions in the `query`.

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-12-15 12:01:15 +08:00

149 lines
9.4 KiB
Plaintext
Raw Blame History

token literal names:
null
'('
')'
'['
','
']'
'{'
'}'
'<'
'<='
'>'
'>='
'=='
'!='
null
null
null
null
null
null
null
null
'='
'+'
'-'
'*'
'/'
'%'
'**'
'<<'
'>>'
'&'
'|'
'^'
null
null
null
null
'~'
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
'$meta'
null
null
null
null
null
token symbolic names:
null
null
null
null
null
null
LBRACE
RBRACE
LT
LE
GT
GE
EQ
NE
LIKE
EXISTS
TEXTMATCH
PHRASEMATCH
RANDOMSAMPLE
INTERVAL
ISO
MINIMUM_SHOULD_MATCH
ASSIGN
ADD
SUB
MUL
DIV
MOD
POW
SHL
SHR
BAND
BOR
BXOR
AND
OR
ISNULL
ISNOTNULL
BNOT
NOT
IN
EmptyArray
JSONContains
JSONContainsAll
JSONContainsAny
ArrayContains
ArrayContainsAll
ArrayContainsAny
ArrayLength
ElementFilter
STEuqals
STTouches
STOverlaps
STCrosses
STContains
STIntersects
STWithin
STDWithin
STIsValid
BooleanConstant
IntegerConstant
FloatingConstant
Identifier
Meta
StringLiteral
JSONIdentifier
StructSubFieldIdentifier
Whitespace
Newline
rule names:
expr
textMatchOption
atn:
[4, 1, 68, 252, 2, 0, 7, 0, 2, 1, 7, 1, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 3, 0, 10, 8, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 3, 0, 22, 8, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 5, 0, 42, 8, 0, 10, 0, 12, 0, 45, 9, 0, 1, 0, 3, 0, 48, 8, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 3, 0, 62, 8, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 3, 0, 72, 8, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 5, 0, 174, 8, 0, 10, 0, 12, 0, 177, 9, 0, 1, 0, 3, 0, 180, 8, 0, 3, 0, 182, 8, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 3, 0, 189, 8, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 3, 0, 205, 8, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 5, 0, 243, 8, 0, 10, 0, 12, 0, 246, 9, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 2, 0, 2, 0, 15, 1, 0, 23, 24, 1, 0, 8, 13, 1, 0, 62, 63, 2, 0, 23, 24, 38, 39, 2, 0, 42, 42, 45, 45, 2, 0, 43, 43, 46, 46, 2, 0, 44, 44, 47, 47, 2, 0, 62, 62, 65, 65, 1, 0, 25, 27, 1, 0, 29, 30, 1, 0, 8, 9, 2, 0, 62, 62, 65, 66, 1, 0, 10, 11, 1, 0, 8, 11, 1, 0, 12, 13, 308, 0, 188, 1, 0, 0, 0, 2, 247, 1, 0, 0, 0, 4, 5, 6, 0, -1, 0, 5, 9, 5, 62, 0, 0, 6, 7, 7, 0, 0, 0, 7, 8, 5, 19, 0, 0, 8, 10, 5, 64, 0, 0, 9, 6, 1, 0, 0, 0, 9, 10, 1, 0, 0, 0, 10, 11, 1, 0, 0, 0, 11, 12, 7, 1, 0, 0, 12, 13, 5, 20, 0, 0, 13, 189, 5, 64, 0, 0, 14, 15, 5, 20, 0, 0, 15, 16, 5, 64, 0, 0, 16, 17, 7, 1, 0, 0, 17, 21, 5, 62, 0, 0, 18, 19, 7, 0, 0, 0, 19, 20, 5, 19, 0, 0, 20, 22, 5, 64, 0, 0, 21, 18, 1, 0, 0, 0, 21, 22, 1, 0, 0, 0, 22, 189, 1, 0, 0, 0, 23, 189, 5, 60, 0, 0, 24, 189, 5, 61, 0, 0, 25, 189, 5, 59, 0, 0, 26, 189, 5, 64, 0, 0, 27, 189, 7, 2, 0, 0, 28, 189, 5, 65, 0, 0, 29, 189, 5, 66, 0, 0, 30, 31, 5, 6, 0, 0, 31, 32, 5, 62, 0, 0, 32, 189, 5, 7, 0, 0, 33, 34, 5, 1, 0, 0, 34, 35, 3, 0, 0, 0, 35, 36, 5, 2, 0, 0, 36, 189, 1, 0, 0, 0, 37, 38, 5, 3, 0, 0, 38, 43, 3, 0, 0, 0, 39, 40, 5, 4, 0, 0, 40, 42, 3, 0, 0, 0, 41, 39, 1, 0, 0, 0, 42, 45, 1, 0, 0, 0, 43, 41, 1, 0, 0, 0, 43, 44, 1, 0, 0, 0, 44, 47, 1, 0, 0, 0, 45, 43, 1, 0, 0, 0, 46, 48, 5, 4, 0, 0, 47, 46, 1, 0, 0, 0, 47, 48, 1, 0, 0, 0, 48, 49, 1, 0, 0, 0, 49, 50, 5, 5, 0, 0, 50, 189, 1, 0, 0, 0, 51, 189, 5, 41, 0, 0, 52, 53, 5, 15, 0, 0, 53, 189, 3, 0, 0, 37, 54, 55, 5, 16, 0, 0, 55, 56, 5, 1, 0, 0, 56, 57, 5, 62, 0, 0, 57, 58, 5, 4, 0, 0, 58, 61, 5, 64, 0, 0, 59, 60, 5, 4, 0, 0, 60, 62, 3, 2, 1, 0, 61, 59, 1, 0, 0, 0, 61, 62, 1, 0, 0, 0, 62, 63, 1, 0, 0, 0, 63, 189, 5, 2, 0, 0, 64, 65, 5, 17, 0, 0, 65, 66, 5, 1, 0, 0, 66, 67, 5, 62, 0, 0, 67, 68, 5, 4, 0, 0, 68, 71, 5, 64, 0, 0, 69, 70, 5, 4, 0, 0, 70, 72, 3, 0, 0, 0, 71, 69, 1, 0, 0, 0, 71, 72, 1, 0, 0, 0, 72, 73, 1, 0, 0, 0, 73, 189, 5, 2, 0, 0, 74, 75, 5, 18, 0, 0, 75, 76, 5, 1, 0, 0, 76, 77, 3, 0, 0, 0, 77, 78, 5, 2, 0, 0, 78, 189, 1, 0, 0, 0, 79, 80, 5, 49, 0, 0, 80, 81, 5, 1, 0, 0, 81, 82, 5, 62, 0, 0, 82, 83, 5, 4, 0, 0, 83, 84, 3, 0, 0, 0, 84, 85, 5, 2, 0, 0, 85, 189, 1, 0, 0, 0, 86, 87, 7, 3, 0, 0, 87, 189, 3, 0, 0, 30, 88, 89, 7, 4, 0, 0, 89, 90, 5, 1, 0, 0, 90, 91, 3, 0, 0, 0, 91, 92, 5, 4, 0, 0, 92, 93, 3, 0, 0, 0, 93, 94, 5, 2, 0, 0, 94, 189, 1, 0, 0, 0, 95, 96, 7, 5, 0, 0, 96, 97, 5, 1, 0, 0, 97, 98, 3, 0, 0, 0, 98, 99, 5, 4, 0, 0, 99, 100, 3, 0, 0, 0, 100, 101, 5, 2, 0, 0, 101, 189, 1, 0, 0, 0, 102, 103, 7, 6, 0, 0, 103, 104, 5, 1, 0, 0, 104, 105, 3, 0, 0, 0, 105, 106, 5, 4, 0, 0, 106, 107, 3, 0, 0, 0, 107, 108, 5, 2, 0, 0, 108, 189, 1, 0, 0, 0, 109, 110, 5, 50, 0, 0, 110, 111, 5, 1, 0, 0, 111, 112, 5, 62, 0, 0, 112, 113, 5, 4, 0, 0, 113, 114, 5, 64, 0, 0, 114, 189, 5, 2, 0, 0, 115, 116, 5, 51, 0, 0, 116, 117, 5, 1, 0, 0, 117, 118, 5, 62, 0, 0, 118, 119, 5, 4, 0, 0, 119, 120, 5, 64, 0, 0, 120, 189, 5, 2, 0, 0, 121, 122, 5, 52, 0, 0, 122, 123, 5, 1, 0, 0, 123, 124, 5, 62, 0, 0, 124, 125, 5, 4, 0, 0, 125, 126, 5, 64, 0, 0, 126, 189, 5, 2, 0, 0, 127, 128, 5, 53, 0, 0, 128, 129, 5, 1, 0, 0, 129, 130, 5, 62, 0, 0, 130, 131, 5, 4, 0, 0, 131, 132, 5, 64, 0, 0, 132, 189, 5, 2, 0, 0, 133, 134, 5, 54, 0, 0, 134, 135, 5, 1, 0, 0, 135, 136, 5, 62, 0, 0, 136, 137, 5, 4, 0, 0, 137, 138, 5, 64, 0, 0, 138, 189, 5, 2, 0, 0, 139, 140, 5, 55, 0, 0, 140, 141, 5, 1, 0, 0, 141, 142, 5, 62, 0, 0, 142, 143, 5, 4, 0, 0, 143, 144, 5, 64, 0, 0, 144, 189, 5, 2, 0, 0, 145, 146, 5, 56, 0, 0, 146, 147, 5, 1, 0, 0, 147, 148, 5, 62, 0, 0, 148, 149, 5, 4, 0, 0, 149, 150, 5, 64, 0, 0, 150, 189, 5, 2, 0, 0, 151, 152, 5, 57, 0, 0, 152, 153, 5, 1, 0, 0, 153, 154, 5, 62, 0, 0, 154, 155, 5, 4, 0, 0, 155, 156, 5, 64, 0, 0, 156, 157, 5, 4, 0, 0, 157, 158, 3, 0, 0, 0, 158, 159, 5, 2, 0, 0, 159, 189, 1, 0, 0, 0, 160, 161, 5, 58, 0, 0, 161, 162, 5, 1, 0, 0, 162, 163, 5, 62, 0, 0, 163, 189, 5, 2, 0, 0, 164, 165, 5, 48, 0, 0, 165, 166, 5, 1, 0, 0, 166, 167, 7, 7, 0, 0, 167, 189, 5, 2, 0, 0, 168, 169, 5, 62, 0, 0, 169, 181, 5, 1, 0, 0, 170, 175, 3, 0, 0, 0, 171, 172, 5, 4, 0, 0, 172, 174, 3, 0, 0, 0, 173, 171, 1, 0, 0, 0, 174, 177, 1, 0, 0, 0, 175, 173, 1, 0, 0, 0, 175, 176, 1, 0, 0, 0, 176, 179, 1, 0, 0, 0, 177, 175, 1, 0, 0, 0, 178, 180, 5, 4, 0, 0, 179, 178, 1, 0, 0, 0, 179, 180, 1, 0, 0, 0, 180, 182, 1, 0, 0, 0, 181, 170, 1, 0, 0, 0, 181, 182, 1, 0, 0, 0, 182, 183, 1, 0, 0, 0, 183, 189, 5, 2, 0, 0, 184, 185, 7, 7, 0, 0, 185, 189, 5, 36, 0, 0, 186, 187, 7, 7, 0, 0, 187, 189, 5, 37, 0, 0, 188, 4, 1, 0, 0, 0, 188, 14, 1, 0, 0, 0, 188, 23, 1, 0, 0, 0, 188, 24, 1, 0, 0, 0, 188, 25, 1, 0, 0, 0, 188, 26, 1, 0, 0, 0, 188, 27, 1, 0, 0, 0, 188, 28, 1, 0, 0, 0, 188, 29, 1, 0, 0, 0, 188, 30, 1, 0, 0, 0, 188, 33, 1, 0, 0, 0, 188, 37, 1, 0, 0, 0, 188, 51, 1, 0, 0, 0, 188, 52, 1, 0, 0, 0, 188, 54, 1, 0, 0, 0, 188, 64, 1, 0, 0, 0, 188, 74, 1, 0, 0, 0, 188, 79, 1, 0, 0, 0, 188, 86, 1, 0, 0, 0, 188, 88, 1, 0, 0, 0, 188, 95, 1, 0, 0, 0, 188, 102, 1, 0, 0, 0, 188, 109, 1, 0, 0, 0, 188, 115, 1, 0, 0, 0, 188, 121, 1, 0, 0, 0, 188, 127, 1, 0, 0, 0, 188, 133, 1, 0, 0, 0, 188, 139, 1, 0, 0, 0, 188, 145, 1, 0, 0, 0, 188, 151, 1, 0, 0, 0, 188, 160, 1, 0, 0, 0, 188, 164, 1, 0, 0, 0, 188, 168, 1, 0, 0, 0, 188, 184, 1, 0, 0, 0, 188, 186, 1, 0, 0, 0, 189, 244, 1, 0, 0, 0, 190, 191, 10, 31, 0, 0, 191, 192, 5, 28, 0, 0, 192, 243, 3, 0, 0, 32, 193, 194, 10, 29, 0, 0, 194, 195, 7, 8, 0, 0, 195, 243, 3, 0, 0, 30, 196, 197, 10, 28, 0, 0, 197, 198, 7, 0, 0, 0, 198, 243, 3, 0, 0, 29, 199, 200, 10, 27, 0, 0, 200, 201, 7, 9, 0, 0, 201, 243, 3, 0, 0, 28, 202, 204, 10, 26, 0, 0, 203, 205, 5, 39, 0, 0, 204, 203, 1, 0, 0, 0, 204, 205, 1, 0, 0, 0, 205, 206, 1, 0, 0, 0, 206, 207, 5, 40, 0, 0, 207, 243, 3, 0, 0, 27, 208, 209, 10, 11, 0, 0, 209, 210, 7, 10, 0, 0, 210, 211, 7, 11, 0, 0, 211, 212, 7, 10, 0, 0, 212, 243, 3, 0, 0, 12, 213, 214, 10, 10, 0, 0, 214, 215, 7, 12, 0, 0, 215, 216, 7, 11, 0, 0, 216, 217, 7, 12, 0, 0, 217, 243, 3, 0, 0, 11, 218, 219, 10, 9, 0, 0, 219, 220, 7, 13, 0, 0, 220, 243, 3, 0, 0, 10, 221, 222, 10, 8, 0, 0, 222, 223, 7, 14, 0, 0, 223, 243, 3, 0, 0, 9, 224, 225, 10, 7, 0, 0, 225, 226, 5, 31, 0, 0, 226, 243, 3, 0, 0, 8, 227, 228, 10, 6, 0, 0, 228, 229, 5, 33, 0, 0, 229, 243, 3, 0, 0, 7, 230, 231, 10, 5, 0, 0, 231, 232, 5, 32, 0, 0, 232, 243, 3, 0, 0, 6, 233, 234, 10, 4, 0, 0, 234, 235, 5, 34, 0, 0, 235, 243, 3, 0, 0, 5, 236, 237, 10, 3, 0, 0, 237, 238, 5, 35, 0, 0, 238, 243, 3, 0, 0, 4, 239, 240, 10, 36, 0, 0, 240, 241, 5, 14, 0, 0, 241, 243, 5, 64, 0, 0, 242, 190, 1, 0, 0, 0, 242, 193, 1, 0, 0, 0, 242, 196, 1, 0, 0, 0, 242, 199, 1, 0, 0, 0, 242, 202, 1, 0, 0, 0, 242, 208, 1, 0, 0, 0, 242, 213, 1, 0, 0, 0, 242, 218, 1, 0, 0, 0, 242, 221, 1, 0, 0, 0, 242, 224, 1, 0, 0, 0, 242, 227, 1, 0, 0, 0, 242, 230, 1, 0, 0, 0, 242, 233, 1, 0, 0, 0, 242, 236, 1, 0, 0, 0, 242, 239, 1, 0, 0, 0, 243, 246, 1, 0, 0, 0, 244, 242, 1, 0, 0, 0, 244, 245, 1, 0, 0, 0, 245, 1, 1, 0, 0, 0, 246, 244, 1, 0, 0, 0, 247, 248, 5, 21, 0, 0, 248, 249, 5, 22, 0, 0, 249, 250, 5, 60, 0, 0, 250, 3, 1, 0, 0, 0, 13, 9, 21, 43, 47, 61, 71, 175, 179, 181, 188, 204, 242, 244]
Reference in New Issue View Git Blame Copy Permalink