23 Commits

Author SHA1 Message Date
Bingyi Sun
e1258b8cad
feat: integrate storagev2 into loading segment (#29336)
issue: #29335

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-01-12 18:10:51 +08:00
congqixia
13aa174b8a
enhance: add log when release segment created for load failure (#29464)
Add log for releasing segment created during load process when load
error happens

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-26 00:46:46 +08:00
yah01
dc89730a50
Support collection-level mmap control (#26901)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-02 23:52:16 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
yah01
0a750408d0
Skip delta logs have been applied (#26971)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-19 16:21:23 +08:00
congqixia
2a5d574a0d
Fix querynodev2 concurrent load logic (#26959)
Fix logic error from #26926
function `waitSegmentLoadDone` shall return error when context is done

Make delegator control concurrency for each same segment
Related to #26908

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-10 07:41:18 +08:00
congqixia
c8f9f22c4a
Fix segment loader return false success (#26926)
`waitSegmentLoadDone` did not check waitCh result is success or failure
after load return without error, delegator will assume all segments are loaded

This PR changes waitCh to loadResult with `sync.Cond` with `atomic.Int32` to represent status

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-08 16:41:16 +08:00
wei liu
b47a72bfcf
fix set dirty segment distribution to leader view (#26180)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-11 11:21:32 +08:00
yah01
889424b3f9
Fix load index with empty file list (#26236)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-09 18:39:16 +08:00
aoiasd
7fec0d61cc
Refine test chunkmanager of querynode v2 (#25888)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-08-01 10:33:04 +08:00
yah01
3735a097e6
Refine the memeory usage prediction (#25889)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-25 17:41:01 +08:00
Cai Yudong
9a4761dcc7
Remove binary metrics TANIMOTO/SUPERSTRUCTURE/SUBSTRUCTURE (#25708)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-07-19 16:16:58 +08:00
yah01
d216f9abda
Clear collection meta after all channels/segments released (#25486)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-14 10:28:30 +08:00
xige-16
04082b3de2
Migrate the ability to upload and download binlog to cpp (#22984)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-25 14:38:44 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
yah01
400364483d
Improve predict the resource usage (#23968)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-11 15:33:24 +08:00
congqixia
884f7904a1
Re-calculate segment entry num when binlog field is zero (#23883)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-06 14:24:39 +08:00
foxspy
6f4ed517de
add growing segment index (#23615)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-04-26 10:14:41 +08:00
Enwei Jiao
967a97b9bd
Support json & array types (#23408)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
2023-04-20 11:32:31 +08:00
yah01
dc0eb433ae
Add IoPoolSize config (#23360)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-13 14:50:28 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
yah01
826c1fb850
Support mmap on QueryNodeV2 (#23073)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-28 21:30:05 +08:00
yah01
081572d31c
Refactor QueryNode (#21625)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00