Hao Tan 67c4340565
feat: Geospatial Data Type and GIS Function Support for milvus server (#35990)
issue:https://github.com/milvus-io/milvus/issues/27576

# Main Goals
1. Create and describe collections with geospatial fields, enabling both
client and server to recognize and process geo fields.
2. Insert geospatial data as payload values in the insert binlog, and
print the values for verification.
3. Load segments containing geospatial data into memory.
4. Ensure query outputs can display geospatial data.
5. Support filtering on GIS functions for geospatial columns.

# Solution
1. **Add Type**: Modify the Milvus core by adding a Geospatial type in
both the C++ and Go code layers, defining the Geospatial data structure
and the corresponding interfaces.
2. **Dependency Libraries**: Introduce necessary geospatial data
processing libraries. In the C++ source code, use Conan package
management to include the GDAL library. In the Go source code, add the
go-geom library to the go.mod file.
3. **Protocol Interface**: Revise the Milvus protocol to provide
mechanisms for Geospatial message serialization and deserialization.
4. **Data Pipeline**: Facilitate interaction between the client and
proxy using the WKT format for geospatial data. The proxy will convert
all data into WKB format for downstream processing, providing column
data interfaces, segment encapsulation, segment loading, payload
writing, and cache block management.
5. **Query Operators**: Implement simple display and support for filter
queries. Initially, focus on filtering based on spatial relationships
for a single column of geospatial literal values, providing parsing and
execution for query expressions.
6. **Client Modification**: Enable the client to handle user input for
geospatial data and facilitate end-to-end testing.Check the modification
in pymilvus.

---------

Signed-off-by: tasty-gumi <1021989072@qq.com>
2024-10-31 20:58:20 +08:00

124 lines
6.9 KiB
Plaintext

token literal names:
null
'('
')'
'['
','
']'
'{'
'}'
'<'
'<='
'>'
'>='
'=='
'!='
null
null
null
'+'
'-'
'*'
'/'
'%'
'**'
'<<'
'>>'
'&'
'|'
'^'
null
null
'~'
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
null
'$meta'
null
null
null
null
token symbolic names:
null
null
null
null
null
null
LBRACE
RBRACE
LT
LE
GT
GE
EQ
NE
LIKE
EXISTS
TEXTMATCH
ADD
SUB
MUL
DIV
MOD
POW
SHL
SHR
BAND
BOR
BXOR
AND
OR
BNOT
NOT
IN
EmptyArray
JSONContains
JSONContainsAll
JSONContainsAny
ArrayContains
ArrayContainsAll
ArrayContainsAny
ArrayLength
STEuqals
STTouches
STOverlaps
STCrosses
STContains
STIntersects
STWithin
BooleanConstant
IntegerConstant
FloatingConstant
Identifier
Meta
StringLiteral
JSONIdentifier
Whitespace
Newline
rule names:
expr
atn:
[4, 1, 56, 184, 2, 0, 7, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 5, 0, 21, 8, 0, 10, 0, 12, 0, 24, 9, 0, 1, 0, 3, 0, 27, 8, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 5, 0, 112, 8, 0, 10, 0, 12, 0, 115, 9, 0, 1, 0, 3, 0, 118, 8, 0, 3, 0, 120, 8, 0, 1, 0, 1, 0, 1, 0, 3, 0, 125, 8, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 3, 0, 141, 8, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 5, 0, 179, 8, 0, 10, 0, 12, 0, 182, 9, 0, 1, 0, 0, 1, 0, 1, 0, 0, 13, 1, 0, 51, 52, 2, 0, 17, 18, 30, 31, 2, 0, 34, 34, 37, 37, 2, 0, 35, 35, 38, 38, 2, 0, 36, 36, 39, 39, 2, 0, 51, 51, 54, 54, 1, 0, 19, 21, 1, 0, 17, 18, 1, 0, 23, 24, 1, 0, 8, 9, 1, 0, 10, 11, 1, 0, 8, 11, 1, 0, 12, 13, 227, 0, 124, 1, 0, 0, 0, 2, 3, 6, 0, -1, 0, 3, 125, 5, 49, 0, 0, 4, 125, 5, 50, 0, 0, 5, 125, 5, 48, 0, 0, 6, 125, 5, 53, 0, 0, 7, 125, 7, 0, 0, 0, 8, 125, 5, 54, 0, 0, 9, 10, 5, 6, 0, 0, 10, 11, 5, 51, 0, 0, 11, 125, 5, 7, 0, 0, 12, 13, 5, 1, 0, 0, 13, 14, 3, 0, 0, 0, 14, 15, 5, 2, 0, 0, 15, 125, 1, 0, 0, 0, 16, 17, 5, 3, 0, 0, 17, 22, 3, 0, 0, 0, 18, 19, 5, 4, 0, 0, 19, 21, 3, 0, 0, 0, 20, 18, 1, 0, 0, 0, 21, 24, 1, 0, 0, 0, 22, 20, 1, 0, 0, 0, 22, 23, 1, 0, 0, 0, 23, 26, 1, 0, 0, 0, 24, 22, 1, 0, 0, 0, 25, 27, 5, 4, 0, 0, 26, 25, 1, 0, 0, 0, 26, 27, 1, 0, 0, 0, 27, 28, 1, 0, 0, 0, 28, 29, 5, 5, 0, 0, 29, 125, 1, 0, 0, 0, 30, 125, 5, 33, 0, 0, 31, 32, 5, 16, 0, 0, 32, 33, 5, 1, 0, 0, 33, 34, 5, 51, 0, 0, 34, 35, 5, 4, 0, 0, 35, 36, 5, 53, 0, 0, 36, 125, 5, 2, 0, 0, 37, 38, 7, 1, 0, 0, 38, 125, 3, 0, 0, 27, 39, 40, 7, 2, 0, 0, 40, 41, 5, 1, 0, 0, 41, 42, 3, 0, 0, 0, 42, 43, 5, 4, 0, 0, 43, 44, 3, 0, 0, 0, 44, 45, 5, 2, 0, 0, 45, 125, 1, 0, 0, 0, 46, 47, 7, 3, 0, 0, 47, 48, 5, 1, 0, 0, 48, 49, 3, 0, 0, 0, 49, 50, 5, 4, 0, 0, 50, 51, 3, 0, 0, 0, 51, 52, 5, 2, 0, 0, 52, 125, 1, 0, 0, 0, 53, 54, 7, 4, 0, 0, 54, 55, 5, 1, 0, 0, 55, 56, 3, 0, 0, 0, 56, 57, 5, 4, 0, 0, 57, 58, 3, 0, 0, 0, 58, 59, 5, 2, 0, 0, 59, 125, 1, 0, 0, 0, 60, 61, 5, 41, 0, 0, 61, 62, 5, 1, 0, 0, 62, 63, 5, 51, 0, 0, 63, 64, 5, 4, 0, 0, 64, 65, 5, 53, 0, 0, 65, 125, 5, 2, 0, 0, 66, 67, 5, 42, 0, 0, 67, 68, 5, 1, 0, 0, 68, 69, 5, 51, 0, 0, 69, 70, 5, 4, 0, 0, 70, 71, 5, 53, 0, 0, 71, 125, 5, 2, 0, 0, 72, 73, 5, 43, 0, 0, 73, 74, 5, 1, 0, 0, 74, 75, 5, 51, 0, 0, 75, 76, 5, 4, 0, 0, 76, 77, 5, 53, 0, 0, 77, 125, 5, 2, 0, 0, 78, 79, 5, 44, 0, 0, 79, 80, 5, 1, 0, 0, 80, 81, 5, 51, 0, 0, 81, 82, 5, 4, 0, 0, 82, 83, 5, 53, 0, 0, 83, 125, 5, 2, 0, 0, 84, 85, 5, 45, 0, 0, 85, 86, 5, 1, 0, 0, 86, 87, 5, 51, 0, 0, 87, 88, 5, 4, 0, 0, 88, 89, 5, 53, 0, 0, 89, 125, 5, 2, 0, 0, 90, 91, 5, 46, 0, 0, 91, 92, 5, 1, 0, 0, 92, 93, 5, 51, 0, 0, 93, 94, 5, 4, 0, 0, 94, 95, 5, 53, 0, 0, 95, 125, 5, 2, 0, 0, 96, 97, 5, 47, 0, 0, 97, 98, 5, 1, 0, 0, 98, 99, 5, 51, 0, 0, 99, 100, 5, 4, 0, 0, 100, 101, 5, 53, 0, 0, 101, 125, 5, 2, 0, 0, 102, 103, 5, 40, 0, 0, 103, 104, 5, 1, 0, 0, 104, 105, 7, 5, 0, 0, 105, 125, 5, 2, 0, 0, 106, 107, 5, 51, 0, 0, 107, 119, 5, 1, 0, 0, 108, 113, 3, 0, 0, 0, 109, 110, 5, 4, 0, 0, 110, 112, 3, 0, 0, 0, 111, 109, 1, 0, 0, 0, 112, 115, 1, 0, 0, 0, 113, 111, 1, 0, 0, 0, 113, 114, 1, 0, 0, 0, 114, 117, 1, 0, 0, 0, 115, 113, 1, 0, 0, 0, 116, 118, 5, 4, 0, 0, 117, 116, 1, 0, 0, 0, 117, 118, 1, 0, 0, 0, 118, 120, 1, 0, 0, 0, 119, 108, 1, 0, 0, 0, 119, 120, 1, 0, 0, 0, 120, 121, 1, 0, 0, 0, 121, 125, 5, 2, 0, 0, 122, 123, 5, 15, 0, 0, 123, 125, 3, 0, 0, 1, 124, 2, 1, 0, 0, 0, 124, 4, 1, 0, 0, 0, 124, 5, 1, 0, 0, 0, 124, 6, 1, 0, 0, 0, 124, 7, 1, 0, 0, 0, 124, 8, 1, 0, 0, 0, 124, 9, 1, 0, 0, 0, 124, 12, 1, 0, 0, 0, 124, 16, 1, 0, 0, 0, 124, 30, 1, 0, 0, 0, 124, 31, 1, 0, 0, 0, 124, 37, 1, 0, 0, 0, 124, 39, 1, 0, 0, 0, 124, 46, 1, 0, 0, 0, 124, 53, 1, 0, 0, 0, 124, 60, 1, 0, 0, 0, 124, 66, 1, 0, 0, 0, 124, 72, 1, 0, 0, 0, 124, 78, 1, 0, 0, 0, 124, 84, 1, 0, 0, 0, 124, 90, 1, 0, 0, 0, 124, 96, 1, 0, 0, 0, 124, 102, 1, 0, 0, 0, 124, 106, 1, 0, 0, 0, 124, 122, 1, 0, 0, 0, 125, 180, 1, 0, 0, 0, 126, 127, 10, 28, 0, 0, 127, 128, 5, 22, 0, 0, 128, 179, 3, 0, 0, 29, 129, 130, 10, 26, 0, 0, 130, 131, 7, 6, 0, 0, 131, 179, 3, 0, 0, 27, 132, 133, 10, 25, 0, 0, 133, 134, 7, 7, 0, 0, 134, 179, 3, 0, 0, 26, 135, 136, 10, 24, 0, 0, 136, 137, 7, 8, 0, 0, 137, 179, 3, 0, 0, 25, 138, 140, 10, 23, 0, 0, 139, 141, 5, 31, 0, 0, 140, 139, 1, 0, 0, 0, 140, 141, 1, 0, 0, 0, 141, 142, 1, 0, 0, 0, 142, 143, 5, 32, 0, 0, 143, 179, 3, 0, 0, 24, 144, 145, 10, 10, 0, 0, 145, 146, 7, 9, 0, 0, 146, 147, 7, 5, 0, 0, 147, 148, 7, 9, 0, 0, 148, 179, 3, 0, 0, 11, 149, 150, 10, 9, 0, 0, 150, 151, 7, 10, 0, 0, 151, 152, 7, 5, 0, 0, 152, 153, 7, 10, 0, 0, 153, 179, 3, 0, 0, 10, 154, 155, 10, 8, 0, 0, 155, 156, 7, 11, 0, 0, 156, 179, 3, 0, 0, 9, 157, 158, 10, 7, 0, 0, 158, 159, 7, 12, 0, 0, 159, 179, 3, 0, 0, 8, 160, 161, 10, 6, 0, 0, 161, 162, 5, 25, 0, 0, 162, 179, 3, 0, 0, 7, 163, 164, 10, 5, 0, 0, 164, 165, 5, 27, 0, 0, 165, 179, 3, 0, 0, 6, 166, 167, 10, 4, 0, 0, 167, 168, 5, 26, 0, 0, 168, 179, 3, 0, 0, 5, 169, 170, 10, 3, 0, 0, 170, 171, 5, 28, 0, 0, 171, 179, 3, 0, 0, 4, 172, 173, 10, 2, 0, 0, 173, 174, 5, 29, 0, 0, 174, 179, 3, 0, 0, 3, 175, 176, 10, 30, 0, 0, 176, 177, 5, 14, 0, 0, 177, 179, 5, 53, 0, 0, 178, 126, 1, 0, 0, 0, 178, 129, 1, 0, 0, 0, 178, 132, 1, 0, 0, 0, 178, 135, 1, 0, 0, 0, 178, 138, 1, 0, 0, 0, 178, 144, 1, 0, 0, 0, 178, 149, 1, 0, 0, 0, 178, 154, 1, 0, 0, 0, 178, 157, 1, 0, 0, 0, 178, 160, 1, 0, 0, 0, 178, 163, 1, 0, 0, 0, 178, 166, 1, 0, 0, 0, 178, 169, 1, 0, 0, 0, 178, 172, 1, 0, 0, 0, 178, 175, 1, 0, 0, 0, 179, 182, 1, 0, 0, 0, 180, 178, 1, 0, 0, 0, 180, 181, 1, 0, 0, 0, 181, 1, 1, 0, 0, 0, 182, 180, 1, 0, 0, 0, 9, 22, 26, 113, 117, 119, 124, 140, 178, 180]