admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-04 18:02:08 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,896 Commits 136 Branches 188 Tags
Commit Graph

956 Commits

Author SHA1 Message Date
dragondriver
457e964e41
Add channelsTimeTicker to proxy (#5339) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-05-24 01:49:28 +00:00
neza2017
e326eaad1e
master client,reconnect if grpc failed (#5360) 
master client,reconnect if grpc failed

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-05-24 01:44:49 +00:00
godchen
d189bf4d33
Change pulsar version (#5352) 
Change pulsar version.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-22 03:36:49 +00:00
zwd1208
7d49096e8c
Add prometheus metrics (#5342) 
Add prometheus metrics for components

Resolves: #5325

Signed-off-by: Weida Zhu <weida.zhu@zilliz.com>
 2021-05-21 13:48:04 +00:00
congqixia
4aa6a8fe18
Dataservice use lastest grpc design and saves ddl/dml pos pair (#5348) 
Dataservice use lastest grpc design and saves ddl/dml pos pair

* Update SaveBinlogPaths grpc definition
* Saves segment ddl & dml flush pos pair(start, end)

See also #5220 

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-05-21 11:38:58 +00:00
godchen
0eee7e5252
Change session (#5329) 
Change session.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-21 11:28:52 +00:00
sunby
02f2348500
Save segment allocation status to meta (#5184) 
To ensure segments being flushed correctly, we need to load 
the segment allocation status from meta after starting the service.
 2021-05-21 10:30:41 +00:00
congqixia
7a82462da8
DataService publishes flush-completed message into its channel (#5320) 
DataService publishes flush-completed message  into related msgstream

See also: #5220 

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-05-21 08:54:29 +00:00
yukun
156d5b9f73
Add RetrieveTask PostExecute function (#5336) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>

See also: #5257
 2021-05-21 08:39:28 +00:00
congqixia
46790d9c68
Fix issue 5333 uninitialized return value (#5338) 
Fix uninitialized named value `meta` caused crash

Resolves #5333 

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-05-21 08:21:08 +00:00
Cai Yudong
761350e44d
Refactor timetick msg handling between proxy node and master (#5321) 
Proxy node send channel timetick msg to master, master reduce timetick
msg for each channel and send the min timetick to msg stream.

Resolves: #5276 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-21 08:08:12 +00:00
sunby
c1efa53779
Remove redundant fields in SegmentInfo and related structure (#5280) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-05-21 06:51:25 +00:00
zhenshan.cao
70241a8bf9
Cancel local timestamp caching logic (#5327) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-05-21 05:11:21 +00:00
dragondriver
f6ab3c81df
Add channelsMgr to proxy (#5323) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-05-21 01:52:20 +00:00
Xiangyu Wang
23c4de0eb8
Flush statistics for all int64 fields (#5318) 
Resolves: #5262

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-05-20 10:38:45 +00:00
godchen
913712fbea
Improve session manager (#5310) 
Improve session manager.
Issue #5174

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-20 07:07:25 +00:00
yukun
f18dfb4ff6
Add RetrieveTask implementation (#5313) 
Resolves: #5257 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-05-20 07:02:31 +00:00
neza2017
c0daf8e41d
set timestamp on dd requests (#5298) 
- set dd request's time tick on master
- send time tick into dd channel

See also: #5291

Signed-off-by: yefu.chen yefu.chen@zilliz.com
 2021-05-20 06:14:14 +00:00
congqixia
38f5b0826f
Add SaveBinlogPaths grpc in dataservice (#5277) 
* Add SaveBinlogPaths grpc in dataservice

* fix merged compile error

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-05-20 11:34:45 +08:00
XuanYang-cn
1c779f3efd
Refactor ddl binlog flush (#5303) 
DDL won't auto-flush now

See also: #5289 , #5220 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-05-19 11:42:07 +00:00
yukun
a7bb701f73
Add Retrieve method grpc definition (#5294) 
See also: #5253 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-05-19 10:45:15 +00:00
godchen
e224d1e725
Add SessionManager (#5288) 
Add SessionManager.
Resolves: #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-19 10:36:05 +00:00
quicksilver
3db653f2d0
Update cpp thirdparty source URL (#5295) 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

Resolves: #5293
 2021-05-19 09:39:29 +00:00
congqixia
d5ab8ac345
Restore flush and stats stream pos (#5284) 
Restore segment flush stream & statistic stream to last success pos

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-05-19 06:13:53 +00:00
XuanYang-cn
fc630bc1c8
Correct auto-flush behaviour (#5282) 
Before this PR, DataNode considered auto-flush a valid flush
complete. It's wrong. So I open this PR to correct this behaviour
in DataNode.

Now binlog paths from auto-flush will be buffered in replica,
waiting until the manul flush to save into etcd all together.

See also: #5220, #5268
A follow up job of #5271 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-05-18 11:45:00 +00:00
godchen
ccc0130bb4
Add exclusive parameter (#5278) 
Add exclusive parameter.
Issue #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-18 11:07:27 +00:00
neza2017
d68ee3fbc2
query base on timeline (#5279) 
query base on timeline, add timestamp on these queries:
- ShowCollections
- HasCollection
- DescribeCollection

Resolves: #5219

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-05-18 09:12:17 +00:00
XuanYang-cn
c3ac1375d8
Add buffer function for auto flush (#5271) 
Auto-flush of a segment is not considered flush-completed.
So we need to buffer binlog paths generated by auto-flush.

See also: #5268 #5220 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-05-18 08:33:05 +00:00
neza2017
49f6542b1d
query master service's with timestamp (#5275) 
let meta support snapshot
so collection could query meta with timestamp

Resolves: #5219

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-05-18 06:18:02 +00:00
godchen
cca75583a6
Add session util (#5269) 
Add session util.
Issue #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-18 03:39:21 +00:00
Cai Yudong
e8ceb37f81
Add ShardsNum when create collection (#5245) 
Updata PhysicalChannelNames and VirtualChannelNames based
on ShardsNum when create collection, and update collVChan2Chan
map.

Resolves: #5240 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-17 11:15:01 +00:00
Cai Yudong
001795ee70
Optimize CreateIndex routine in master service (#5238) 
Remove CreateIndexTask struct and indexTaskQueue

Resolves: #5228 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-15 10:08:08 +00:00
Cai Yudong
08bb1b2ec3
Resend unsuccessful ddMsg when master start (#5214) 
Resend unsuccessful ddMsg when master start

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-14 21:26:06 +08:00
XuanYang-cn
9151e13480
Remove not using dependence and proto (#5226) 
- FlushMsg proto message is no longer in use

- We don't depend on "github.com/prometheus/common/log"

Signed-off-by: yangxuan xuan.yang@zilliz.com
 2021-05-14 08:50:55 +00:00
yukun
4e6e502df1
Add data sorter in storage (#5200) 
GetEntityByID needs the flushed segment to be sorted by RowID field, 
then do binary search to get the target id and entities.

See also: #5177 

Signed-off-by: fishpenguin kun.yu@zilliz.com
 2021-05-14 02:59:49 +00:00
godchen
b74afd7a0e
Add service registration (#5189) 
Add service registration.
Part of Issue #5174.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-14 02:05:18 +00:00
Cai Yudong
7b83b11e95
Save ddmsg info and type into ETCD (#5173) 
Save ddmsg info and type into ETCD to support re-send ddmsg 
when system restart or recover from failure (#5172)

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-12 07:33:53 +00:00
dragondriver
64ab4d740b
Add row_count to partition statistics (#5162) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-05-10 09:39:08 +00:00
sunby
a00422bac2
Remove unused interfaces in dataservice (#5157) 
Remove unused interfaces in dataservice

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-05-10 09:03:24 +00:00
Cai Yudong
6f3a460b4e
Add pulsar seek example in pulsar_client_test.go (#5154) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-10 02:32:10 +00:00
dragondriver
df71f4c77c
Migrate softTimeTickBarrier from proxyservice to timesync (#5146) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-05-10 01:28:59 +00:00
XuanYang-cn
eb557b289b
Add docs in package datanode (#5117) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-05-08 07:24:12 +00:00
Cai Yudong
5f0006d0f9
Improve code coverage for distributed/datanode (#5133) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-08 06:07:04 +00:00
yukun
ecaef24fea
Change should_not to must_not in dsl (#5131) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-05-07 11:52:24 +00:00
FluorineDog
b1974aede2
Merge knowhere unittests (#5139) 
Re-enable knowhere unittests and merge them all together. 
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-05-07 11:50:13 +00:00
dragondriver
b7977698a8
Change the codec style of expr plan from text to binary (#5129) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-05-07 11:27:17 +00:00
Cai Yudong
d66a5acca3
improve code coverage for distributed/dataservice (#5123) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
improve code coverage for distributed/dataservice
 2021-05-07 07:43:52 +00:00
dragondriver
e5d4963ba0
Distinguish the dsl and expr in proxy and query node (#5118) 
Pass a parameter `IsExpr` from proxy to query node, so query 
node can create search plan according to the variable `IsExpr`.
It makes `segcore` unnecessary to distinguish `dsl` or `expr`
using dynamic exception. When `IsExpr` is set to true, query
node will call `CreatePlanByExpr` according to the passed
information about expression. Otherwise query node will keep
still to use `CreatePlan` according to the `dsl` information. At
the same time, this pr adds some unittests to `CreatePlanByExpr`,
these unittests translate already exist case with `dsl` to case with
`expr`.

Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-05-07 07:20:47 +00:00
xige-16
1e7205fa03
fix querynode's port not set correctly when conflict with other node (#5066) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-05-07 12:03:53 +08:00
FluorineDog
396b3f33e9
Support TermExpr, NotExpr, LogicalExpr (#5096) 
1. Support Term, like `A in [1, 2, 3]`
2. Support Not, like `! A < 3`
3. Support logical combination, like `A < 3 && B > 5 or C == 0`

Type: Feature

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-04-30 07:19:52 +00:00