49 Commits

Author SHA1 Message Date
bigsheeper
4ecdea698f Refactor query node and query serviceMain changes:1. Add ddBuffer and save binLog.
2. Trans to insertData.
3. Change dataFields data to Data, dim to Dim4. Add float vector and binary vector.5. Deserialize data and convert to InsertData.6. Move all data into InsertData.7. Add insert buffer and hash string.
8. Add minIOkV in insertBuffer node.
9. Init write node insertBuffer maxSize from writeNode.yaml.
10. Add ddBuffer.
11. Add ddBuffer binLog and minio.
12. Add ddNode unittest.
13. Remove redundant call.
14. Increase test time.
15. Delete ddl const, use request's timestamp instead.

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-24 15:38:29 +08:00
xige-16
f5198bcaee Add LoadIndexService in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-24 14:51:55 +08:00
sunby
add31188ba Add CreateIndex RPC
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-24 14:50:16 +08:00
sunby
0b9cffaf87 Remove index_name in IndexRequest and remove indexType in FieldIndexMeta
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-23 11:34:35 +08:00
sunby
2aa483c922 Add index meta to meta table
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-22 20:16:32 +08:00
sunby
16071d92b5 Modify metatable close segment interface.
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-10 13:35:09 +08:00
zhenshan.cao
7bbbc14637 Fix bug: address already used
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-07 15:22:20 +08:00
xige-16
37d7526d31 Fix search test failure
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-07 14:37:42 +08:00
neza2017
aeca8f85a5 Run cwrapper unittest in ci
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 17:39:58 +08:00
cai.zhang
806a97505c Add query node for CI progress
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-05 16:12:45 +08:00
FluorineDog
6412ebc0d4 Add support of metric type in schema, enable binary vector, fix segfault
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-05 06:46:01 +08:00
zhenshan.cao
0b6fe0a8cf Refactor paramtable
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-04 14:18:17 +08:00
godchen
7ab5b5d80d Add MinIO kv implements
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-03 19:00:11 +08:00
neza2017
32445dd02b Add cpu profile
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-26 16:01:31 +08:00
cai.zhang
57da66ba72 Update read config logic in docker
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-26 15:18:36 +08:00
zhenshan.cao
045fc3d808 Fix segment fault
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-26 14:29:01 +08:00
XuanYang-cn
0a16a9a651 Add logic of getting milvus distributed config from master by grpc
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-26 13:21:06 +08:00
xige-16
ae78aa1ac1 Warp cpp reduce and repack searchResultMsg to specified pulsar channel
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-11-26 11:12:33 +08:00
neza2017
ae26a81550 Change to use Marshal as text
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-26 10:13:23 +08:00
sunby
f4c643f1bd Add number limitation of partition and field. And Change default partition tag to "_default"
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-25 18:39:05 +08:00
bigsheeper
84f3d974fa Delete pulsar address test and refactor master param table
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-23 18:04:09 +08:00
cai.zhang
17c3a70d64 update ParameterTable
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-21 15:06:46 +08:00
bigsheeper
dd3654f953 Fix query node param table
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-20 17:10:24 +08:00
quicksilver
1728a4f419 Update .devcontainer.json
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-11-20 15:02:26 +08:00
neza2017
cda47a9252 Remove unused code
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-20 09:13:29 +08:00
zhenshan.cao
50d59d91ee Add segmentIDAssigner for proxy
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-19 21:02:31 +08:00
neza2017
c3de667bf5 Implement segment management in master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-18 19:46:18 +08:00
bigsheeper
99ea7e9f05 Update param table
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-18 19:33:56 +08:00
sunby
9e4ce8454e Implement segment management in master
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-18 15:04:17 +08:00
dragondriver
b32e55d5f0 Add timeout logic to task in Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-17 20:00:23 +08:00
quicksilver
72ed65d7f8 Check builder images
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-11-17 18:36:03 +08:00
godchen
4f04be4a28 Add master dd task unittest
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-11-17 17:17:10 +08:00
cai.zhang
5bcabffdaf Replace conf struct with ParamsTable
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-16 21:10:43 +08:00
zhenshan.cao
9f948d3095 Remove mockmaster
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-14 16:40:10 +08:00
quicksilver
00b4b67a1f Update the version of build environment
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-11-14 16:01:12 +08:00
neza2017
e349508e78 Fix list-collections
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-14 15:57:39 +08:00
neza2017
ce969e9568 Update segment
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-14 14:39:52 +08:00
neza2017
74f8c45a23 Add ListCollection for meta table
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-14 11:50:57 +08:00
dragondriver
c2382adf04 Initialize the task scheduler of Proxy
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-14 11:24:49 +08:00
cai.zhang
0b478b614a Change Id to ID in go files according to golint check
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-13 15:17:18 +08:00
XuanYang-cn
88cb274644 Refactor master scheduler: drop_collection
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-11-13 12:46:24 +08:00
cai.zhang
b68322d469 Open golangci-lint check
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-12 17:58:05 +08:00
neza2017
14d4a400dd Load etcd port from etcd
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-12 17:15:01 +08:00
cai.zhang
1cd2fd9954 Add the standard of golangci-lint
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-12 12:04:12 +08:00
neza2017
980084ff15 Enable github action for go unittest
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-09 17:08:48 +08:00
zhenshan.cao
e9122921b9 Use type alias
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-04 17:58:43 +08:00
godchen
bade037fb2 Add meta_table implementation
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-11-04 16:01:28 +08:00
zhenshan.cao
e962b57fbb Add timestamp allocator
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-03 14:53:36 +08:00
neza2017
10217f9485 Add test for etcd kv
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-02 10:26:33 +08:00