128 Commits

Author SHA1 Message Date
cai.zhang
2b02b869ec
Return real-time indexed rows and total rows (#20460)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-11-14 14:37:07 +08:00
cai.zhang
889eb02172
Add goroutine pool for creating index (#20391)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-11-08 19:41:02 +08:00
Enwei Jiao
7e56e7a976
Fix datarace at Setlogger (#20376)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-08 14:33:03 +08:00
groot
b847c425e1
Fix listImportTasks bug (#20301)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-07 17:11:02 +08:00
cai.zhang
8ea8f91334
Make index log better (#20327)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-11-07 14:23:02 +08:00
Enwei Jiao
956c5e1b9d
Make Params singleton (#20088)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-04 14:25:38 +08:00
Xiaofan
2bfecf5b4e
Refine bloomfilter and memory usage (#20168)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-10-31 17:41:34 +08:00
cai.zhang
bcffb3c389
Check indexes in indexcoord instead of proxy (#20193)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-10-31 11:39:33 +08:00
jaime
d8ca56e084
Fix empty segment handoff fails (#20073)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-26 16:49:31 +08:00
cai.zhang
a60db370e6
Compatibility with handling index params (#19997)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-10-25 16:59:30 +08:00
cai.zhang
85ef7331ee
Set timeout for rpc (#19535)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-10-25 11:55:30 +08:00
cai.zhang
c551de8f72
Catch errors on gloabl meta cache (#20023)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-10-25 11:29:30 +08:00
xige-16
c8ecb81c04
Return IndexInfo if index build done (#19976)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-24 17:17:30 +08:00
yah01
470e8ef419
Store index ID to filter out invalid handoff event (#19703)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-10-20 11:29:27 +08:00
jaime
b79687687d
Replace segment index file path with index file id within segment index meta (#19866)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-19 16:55:27 +08:00
jaime
59bcbf0cf6
Support check health for the service level (#19779)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-18 13:39:26 +08:00
SimFG
a55f739608
Separate public proto files (#19782)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
xige-16
a1db9038fb
Move disk index params to config file (#19714)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-14 17:51:24 +08:00
SimFG
5091a7f362
Add the TotalRows value for the DescribeIndex (#19759)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-13 17:43:24 +08:00
wayblink
85e04d845c
Move GetComponentStates to milvus.proto, make it an external interface (#19594)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-10-10 15:55:22 +08:00
zhenshan.cao
0ac4bc32a5
Add logic of autoindex (#19570)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-10-08 15:38:58 +08:00
SimFG
45f5007410
Improve the DescribeIndex RPC about the state and row num (#19528)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-30 10:56:54 +08:00
wayblink
54e2c079c4
MEP-30:Support basic coordinators active-standby mechanism #16901 (#19166)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-09-29 18:35:02 +08:00
cai.zhang
c58a8d8fd6
Support bind IndexNode mode (#19247)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-27 22:38:55 +08:00
SimFG
13ca8936d1
Deprecate GetIndexState and GetIndexBuildProgress (#19451)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-27 19:12:54 +08:00
zhenshan.cao
c15b880f0e
Remove redundant logs (#19412)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-09-26 15:40:53 +08:00
Xiaofan
928a213e31
Fix Chunk Manager Path mis use (#19141)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-09-23 14:40:51 +08:00
cai.zhang
17f5e3c7fe
Drop segment indexes when drop partition (#19022)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-23 09:36:51 +08:00
xige-16
428840178c
Support diskann index for vector field (#19093)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-21 20:16:51 +08:00
cai.zhang
fc3309b8b8
All indexes segments both write handoff (#19230)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-21 16:34:51 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
cai.zhang
525201decc
Mark index meta as deleted when segment is compacted (#19163)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-15 15:44:31 +08:00
cai.zhang
e65ba6331e
Only store segmentID for flushed segments event (#19075)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-09 15:52:35 +08:00
cai.zhang
d6b0fdc944
Fill index size and fail reason when query index job (#19026)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-06 17:19:11 +08:00
cai.zhang
34c9eae16f
Add IndexCoord ut and fix data race (#18897)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-08-31 10:23:04 +08:00
cai.zhang
c924f73105
Refactor for IndexCoord to support cloud (#18643)
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-25 15:48:54 +08:00
aoiasd
8a70f2badb
Add ShowConfigurations for all remaining session (#18594)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-08-12 13:20:39 +08:00
cai.zhang
0ad15a22c8
IndexCoord handle events correctly (#17878)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-07 14:44:21 +08:00
cai.zhang
01fc411566
Handle etcd compacted error (#17886)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-30 18:54:19 +08:00
cai.zhang
899a9bf86d
IndexNode still writes finished state to ETCD when task was deleted (#17761)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-28 12:08:17 +08:00
cai.zhang
ef0fc37ad7
Fixed the bug that IndexCoord lost some meta information (#17818)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-27 21:52:17 +08:00
cai.zhang
e8f53af749
When the task is canceled, reduce the load of node (#17735)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-23 19:22:15 +08:00
cai.zhang
1fbdafc943
IndexCoord does not assign task to IndexNode prematurely (#17717)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-23 10:40:13 +08:00
cai.zhang
141a3bbe00
Mark deleted before drop index (#17490)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-17 18:08:12 +08:00
cai.zhang
ea5041aec2
Acquiring the segment reference lock on task level (#17544)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-15 21:38:10 +08:00
Xiaofan
af994b5e1c
Fix etcd watch error (#17364)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-06 16:26:06 +08:00
cai.zhang
bcf3b7426a
Add distributed lock for segment refgerence (#16782)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-31 16:36:03 +08:00
cai.zhang
b37b87eb97
The GetIndexFilePaths interface judges whether the index is complete (#17052)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-17 20:41:56 +08:00
cai.zhang
b0c9f25352
Sort out the log of IndexCoord (#16810)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-09 12:05:52 +08:00
cai.zhang
2be46a01c0
Reduce the number of retries and add error log (#16754)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-05-05 09:31:51 +08:00