yah01
c96d07682e
enhance: enhance the accuracy of memory usage ( #28554 )
...
before this, Milvus use container/system's memory info to get the memory
usage, which could be inaccurate.
we allocates the memory by private anon mmap,
then `rss - shared` would be the accurate memory usage
resolve #28553
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-23 15:12:23 +08:00
Enwei Jiao
b80a3e19d3
Add code for PanicInfo ( #27364 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-27 12:01:28 +08:00
cai.zhang
a362bb1457
Support array datatype ( #26369 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-09-19 14:23:23 +08:00
yah01
0459a662e4
use MADV_WILLNEED for scalar column data ( #27170 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-18 18:05:22 +08:00
yihao.dai
bb6711f28c
Add ChunkCache: support get vector from storage ( #26142 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-15 10:21:20 +08:00
Enwei Jiao
c3f15c6b95
Refactor duplicate error class into one place ( #26985 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-11 20:43:17 +08:00
yah01
9605c03c3c
Fix the number of rows of column not correct ( #26347 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-16 13:35:33 +08:00
yah01
127c23d999
Check data consistency after loading ( #26312 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-14 09:01:32 +08:00
yah01
a173486d2e
Fix calculation of memory usage prediction for mmap mode ( #26264 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-12 17:19:31 +08:00
yah01
300fef446b
Enable mmap for vector index ( #25877 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-10 13:59:15 +08:00
yah01
dd5f896dc8
Load batch by batch ( #25212 )
...
This will significantly reduce the memory usage while loading
- 1x memory usage and MBs overhead for buffer (memory mode)
- only MBs overhead for buffer (mmap mode)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-06 13:58:27 +08:00
yah01
cb4b88d5cf
Refactor the column type ( #25147 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-27 19:50:45 +08:00
xige-16
04082b3de2
Migrate the ability to upload and download binlog to cpp ( #22984 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-25 14:38:44 +08:00
