admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2025-12-07 09:38:39 +08:00
Code Issues Packages Projects Releases Wiki Activity
milvus/docs/design_docs/segcore/visitor.md
Yinzuo Jiang 3628593d20
feat: Implement custom function module in milvus expr (#36560) 
OSPP 2024 project:
https://summer-ospp.ac.cn/org/prodetail/247410235?list=org&navpage=org

Solutions:

- parser (planparserv2)
    - add CallExpr in planparserv2/Plan.g4
    - update parser_visitor and show_visitor
- grpc protobuf
    - add CallExpr in plan.proto
- execution (`core/src/exec`)
- add `CallExpr` `ValueExpr` and `ColumnExpr` (both logical and
physical) for function call and function parameters
- function factory (`core/src/exec/expression/function`)
    - create a global hashmap when starting milvus (see server.go)
- the global hashmap stores function signatures and their function
pointers, the CallExpr in execution engine can get the function pointer
by function signature.
- custom functions
    - empty(string)
    - starts_with(string, string)
- add cpp/go unittests and E2E tests

closes: #36559

Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-10-25 15:25:30 +08:00

862 B
Raw Blame History

Visitor Pattern

Visitor Pattern is used in segcore for parse and execute Execution Plan.

  1. Inside ${internal/core}/src/query/PlanNode.h, contains physical plan for vector search:
    1. FloatVectorANNS FloatVector search execution node
    2. BinaryVectorANNS BinaryVector search execution node
  2. ${internal/core}/src/query/Expr.h contains physical plan for scalar expression:
    1. TermExpr support operation like col in [1, 2, 3]
    2. RangeExpr support constant compare with data column like a >= 5 1 < b < 2
    3. CompareExpr support compare with different columns, like a < b
    4. LogicalBinaryExpr support and/or
    5. LogicalUnaryExpr support not

Currently, under ${internal/core/src/query} directory, there are the following visitors:

  1. ExecPlanNodeVistor physical plan executor only supports ANNS node for now