60 Commits

Author SHA1 Message Date
sunby
bf3a6dc3c6 Remove collection in segment manager when dropping collection
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-06 19:23:31 +08:00
sunby
a56f8f7b9e Add runtime stats in master
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-25 18:32:03 +08:00
sunby
a03ab05c6c Add DescribeIndex and DescribeIndexProgress RPC
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-25 15:39:56 +08:00
bigsheeper
4ecdea698f Refactor query node and query serviceMain changes:1. Add ddBuffer and save binLog.
2. Trans to insertData.
3. Change dataFields data to Data, dim to Dim4. Add float vector and binary vector.5. Deserialize data and convert to InsertData.6. Move all data into InsertData.7. Add insert buffer and hash string.
8. Add minIOkV in insertBuffer node.
9. Init write node insertBuffer maxSize from writeNode.yaml.
10. Add ddBuffer.
11. Add ddBuffer binLog and minio.
12. Add ddNode unittest.
13. Remove redundant call.
14. Increase test time.
15. Delete ddl const, use request's timestamp instead.

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-24 15:38:29 +08:00
xige-16
f5198bcaee Add LoadIndexService in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-24 14:51:55 +08:00
sunby
add31188ba Add CreateIndex RPC
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-24 14:50:16 +08:00
sunby
5d58606349 Move Status to SegIDAssignment
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-24 10:11:34 +08:00
cai.zhang
b582cf7b8e Add the interface of index in proto
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-21 19:28:54 +08:00
sunby
06620935ab Move expire timestamp to SegIDAssignment
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-21 10:51:37 +08:00
sunby
c71cd40f68 Refactor segment manager
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-19 12:55:24 +08:00
zhenshan.cao
7bbbc14637 Fix bug: address already used
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-07 15:22:20 +08:00
xige-16
37d7526d31 Fix search test failure
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-07 14:37:42 +08:00
neza2017
aeca8f85a5 Run cwrapper unittest in ci
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 17:39:58 +08:00
cai.zhang
806a97505c Add query node for CI progress
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-05 16:12:45 +08:00
FluorineDog
6412ebc0d4 Add support of metric type in schema, enable binary vector, fix segfault
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-05 06:46:01 +08:00
zhenshan.cao
0b6fe0a8cf Refactor paramtable
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-04 14:18:17 +08:00
godchen
7ab5b5d80d Add MinIO kv implements
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-03 19:00:11 +08:00
neza2017
cec903da19 Add parquet payload
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-02 18:31:56 +08:00
zhenshan.cao
23aebe7e2a Fix proxy unittest
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-01 19:53:53 +08:00
neza2017
17cc7bc337 Use MetaRootPath instead
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-01 17:03:39 +08:00
cai.zhang
85a544b79b Add dockerfile for each component
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-28 19:06:48 +08:00
cai.zhang
115d0732af Update docker-compose file for unittest
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-28 16:43:39 +08:00
dragondriver
eb6c0e0a28 Fix the bug when reduce the multiple search result from query nodes
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-28 15:57:49 +08:00
godchen
92bf09c7e6 Add ddstream and scheduler implements
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-11-28 15:26:39 +08:00
dragondriver
235e723eb6 Read proxy id from param table instead of hardcode
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-27 11:17:12 +08:00
XuanYang-cn
3e605ae25f Rename reader to querynode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-27 10:56:53 +08:00
neza2017
32445dd02b Add cpu profile
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-26 16:01:31 +08:00
cai.zhang
57da66ba72 Update read config logic in docker
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-26 15:18:36 +08:00
zhenshan.cao
045fc3d808 Fix segment fault
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-26 14:29:01 +08:00
XuanYang-cn
0a16a9a651 Add logic of getting milvus distributed config from master by grpc
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-26 13:21:06 +08:00
dragondriver
e36b565ec0 Normalize the naming of variable and function in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-21 16:31:38 +08:00
FluorineDog
9f35dee2df Support range executor
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-21 15:56:13 +08:00
neza2017
73e3ff09f5 Refactor variable name in Master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-21 15:53:01 +08:00
bigsheeper
dd3654f953 Fix query node param table
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-20 17:10:24 +08:00
quicksilver
1728a4f419 Update .devcontainer.json
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-11-20 15:02:26 +08:00
neza2017
cda47a9252 Remove unused code
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-20 09:13:29 +08:00
neza2017
c3de667bf5 Implement segment management in master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-18 19:46:18 +08:00
bigsheeper
99ea7e9f05 Update param table
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-18 19:33:56 +08:00
sunby
9e4ce8454e Implement segment management in master
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-18 15:04:17 +08:00
XuanYang-cn
fd6d0c78e5 Refactor master scheduler: show_collections
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-14 15:37:07 +08:00
zhenshan.cao
e94138816f Add tsLoop for master
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-14 15:26:14 +08:00
neza2017
ce969e9568 Update segment
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-14 14:39:52 +08:00
XuanYang-cn
27032cc126 Refactor master scheduler: describe_collection
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-14 13:06:41 +08:00
godchen
4a3ca1632b Add master partition task implementation
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-11-13 17:20:13 +08:00
XuanYang-cn
1b48e84c35 Refactor master scheduler: has_collection
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-13 15:44:02 +08:00
cai.zhang
0b478b614a Change Id to ID in go files according to golint check
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-13 15:17:18 +08:00
XuanYang-cn
88cb274644 Refactor master scheduler: drop_collection
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-13 12:46:24 +08:00
cai.zhang
b68322d469 Open golangci-lint check
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-12 17:58:05 +08:00
neza2017
14d4a400dd Load etcd port from etcd
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-12 17:15:01 +08:00
cai.zhang
1cd2fd9954 Add the standard of golangci-lint
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-12 12:04:12 +08:00