149 Commits

Author SHA1 Message Date
neza2017
f80dbab6ec
watch proxynode (#5424)
watch proxy node and,
send InvalidataCollectionMetaCache to each proxynode

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

Resolves: #5423
2021-05-26 12:14:30 +00:00
Xiangyu Wang
e021f5e670
Support to use json to construct dummy request (#5430)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-26 08:57:32 +00:00
zhenshan.cao
6766169878
Refactor repack logic for insertion (#5399)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-05-25 11:53:15 +00:00
neza2017
c28c34e852
let master use session.ServerID as nodeID (#5385)
let master use session.ServerID as nodeID

also see #5386 

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-25 07:06:05 +00:00
Xiangyu Wang
88b42304ef
Add dummy RPC (#5376)
Adding a dummy RPC makes it easier to develop new RPCs in the future.
When developing a new function, the SDK is often implemented last. 
At this time, this is helpful for debugging.

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-25 06:44:43 +00:00
yukun
49443e8a33
Add retrieve taskscheduler implementation (#5353)
See also: #5257 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-05-24 10:19:43 +00:00
dragondriver
457e964e41
Add channelsTimeTicker to proxy (#5339)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-05-24 01:49:28 +00:00
neza2017
e326eaad1e
master client,reconnect if grpc failed (#5360)
master client,reconnect if grpc failed

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-24 01:44:49 +00:00
godchen
0eee7e5252
Change session (#5329)
Change session.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-21 11:28:52 +00:00
yukun
156d5b9f73
Add RetrieveTask PostExecute function (#5336)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>

See also: #5257
2021-05-21 08:39:28 +00:00
sunby
c1efa53779
Remove redundant fields in SegmentInfo and related structure (#5280)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-05-21 06:51:25 +00:00
zhenshan.cao
70241a8bf9
Cancel local timestamp caching logic (#5327)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-05-21 05:11:21 +00:00
dragondriver
f6ab3c81df
Add channelsMgr to proxy (#5323)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-05-21 01:52:20 +00:00
yukun
f18dfb4ff6
Add RetrieveTask implementation (#5313)
Resolves: #5257 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-05-20 07:02:31 +00:00
yukun
a7bb701f73
Add Retrieve method grpc definition (#5294)
See also: #5253 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-05-19 10:45:15 +00:00
godchen
e224d1e725
Add SessionManager (#5288)
Add SessionManager.
Resolves: #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-19 10:36:05 +00:00
godchen
b74afd7a0e
Add service registration (#5189)
Add service registration.
Part of Issue #5174.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-14 02:05:18 +00:00
dragondriver
64ab4d740b
Add row_count to partition statistics (#5162)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-05-10 09:39:08 +00:00
dragondriver
b7977698a8
Change the codec style of expr plan from text to binary (#5129)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-05-07 11:27:17 +00:00
dragondriver
e5d4963ba0
Distinguish the dsl and expr in proxy and query node (#5118)
Pass a parameter `IsExpr` from proxy to query node, so query 
node can create search plan according to the variable `IsExpr`.
It makes `segcore` unnecessary to distinguish `dsl` or `expr`
using dynamic exception. When `IsExpr` is set to true, query
node will call `CreatePlanByExpr` according to the passed
information about expression. Otherwise query node will keep
still to use `CreatePlan` according to the `dsl` information. At
the same time, this pr adds some unittests to `CreatePlanByExpr`,
these unittests translate already exist case with `dsl` to case with
`expr`.

Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-05-07 07:20:47 +00:00
FluorineDog
396b3f33e9
Support TermExpr, NotExpr, LogicalExpr (#5096)
1. Support Term, like `A in [1, 2, 3]`
2. Support Not, like `! A < 3`
3. Support logical combination, like `A < 3 && B > 5 or C == 0`

Type: Feature

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-04-30 07:19:52 +00:00
FluorineDog
88f5642603
Add plan proto and support basic boolean expr parser (#5088)
**What type of PR is this?**
- [x] Feature

**What this PR does / why we need it:**
This PR supports boolean expression as DSL.
1. The goal of this PR is to support predicates
    like `A > 3 && not B < 5 or C in [1, 2, 3]`. 
2. Defines `plan.proto`, as Intermediate Representation (IR) 
    used between go and cpp. 
3. Support expr parser, convert predicate expr to IR
    in proxynode, while doing static check there
4. Support IR to AST in cpp, enable the execution
2021-04-29 08:48:06 +00:00
godchen
004598f796
Remove field in desc index interface. (#5080)
* remove field in desc index interface

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* trigger GitHub actions

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-28 09:48:38 +00:00
godchen
94f0966225
Add get index build progress interface implementation (#5067)
Add get index build progress interface implementation.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-28 03:15:28 +00:00
godchen
060f7ca0d2
Add get index build progress proto (#5049)
Add get index build progress proto.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-27 07:46:45 +00:00
godchen
4f7e393bfd
Remove useless config (#4997)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-23 10:10:46 +08:00
bigsheeper
dadb02db75
Remove unused params in query node (#4987)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-23 10:07:45 +08:00
Xiangyu Wang
82ccd4cec0
Rename module (#4988)
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
sunby
fad94fb576
Remove unsed configs (#4990)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-04-22 03:04:59 +00:00
dragondriver
dc7db73251 Add copyright annotations to proxynode and proxyservice
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-04-19 10:09:43 +08:00
XuanYang-cn
e281f6a4a1 Remove useless configs in config files
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-04-13 10:04:39 +08:00
sunby
751c9d4e81 Refactor meta.go
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-04-09 09:55:04 +08:00
dragondriver
57831b9978 Add unittest to insertChannelsMap in proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-04-08 15:41:28 +08:00
dragondriver
846fd8a154 Add unittest to conf adapter in proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-04-08 09:51:04 +08:00
dragondriver
bfb7ca0df6 Add unittest to time tick logic in proxy service
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-04-07 10:06:17 +08:00
dragondriver
decc80a525 Add unittest to proxy service
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-04-06 14:12:57 +08:00
zhenshan.cao
c6950eb7eb Add log for allocator
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-04-01 13:37:18 +08:00
xige-16
c10b36f188 fix getting wrong offset when segment preinsert
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-31 16:16:58 +08:00
zhenshan.cao
efbf4452c5 Add log for port changed or proto unmarshal error
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-03-30 20:19:30 +08:00
sunby
ff8ebd5f42 Fix bug of estimating record size
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-03-29 15:14:51 +08:00
cai.zhang
c4a58d259a Add deploy mode env variable
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-03-27 11:56:57 +08:00
dragondriver
41e1975611 Parallelize the processing of loading binlog and saving index files
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-03-26 11:19:02 +08:00
quicksilver
7ad9b36207 Update reviewdog/action-hadolint github action version to v1.16.1
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-03-25 14:41:46 +08:00
dragondriver
1b743e5c6c Parallelize the processing of SearchTask.PostExecute
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-03-25 10:14:09 +08:00
zhenshan.cao
21ffc660b0 Get rid of memory panic
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-03-23 19:55:00 +08:00
dragondriver
49c6eeb052 Add useful debug log in proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-03-22 19:28:43 +08:00
zhenshan.cao
c2734fa55f Fix bug and enchance system
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-03-22 16:36:10 +08:00
groot
2280791128 Add memory message stream for search
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-03-19 20:16:04 +08:00
bigsheeper
ba65c38d40 Fix Deadline Exceeded
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-03-19 17:50:26 +08:00
zhenshan.cao
ec8ad89900 Delete uncessary print
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-03-15 19:47:13 +08:00