43 Commits

Author SHA1 Message Date
groot
5be395354c
fix: minio ssl compatible issue (#31607)
issue: https://github.com/milvus-io/milvus/issues/30709

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2024-03-27 14:41:20 +08:00
groot
c81909bfab
enhance: Support MinIO TLS connection (#31311)
issue: https://github.com/milvus-io/milvus/issues/30709
pr: #31292

Signed-off-by: yhmo <yihua.mo@zilliz.com>
Co-authored-by: Chen Rao <chenrao317328@163.com>
2024-03-21 11:15:20 +08:00
cai.zhang
6cf2f09b60
feat: Support tencent cloud object storage for milvus (#30163)
issue: #30162

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-01-23 11:28:56 +08:00
yah01
aef483806d
enhance: improve the segcore logs (#29372)
- remove the streaming logging
- refine existing logs

fix #29366

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-23 21:52:43 +08:00
zhagnlu
a602171d06
enhance: Refactor runtime and expr framework (#28166)
#28165

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-12-18 12:04:42 +08:00
zhagnlu
0d9d098186
enhance: Add precheck when chunk manager init (#28330)
#28329

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-11-23 19:56:32 +08:00
PowderLi
a1c505dbd5
add internal storage metrics (#28278)
/kind improvement
issue: #28277

Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-19 17:22:25 +08:00
zhagnlu
3920bbc55f
Force set aliyun use_virtual_host to true for all (#28158)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-11-10 10:32:20 +08:00
PowderLi
7bb0fa9c70
reduce useless ObjectExists (#28156)
replace ListBlobs() with GetProperties()
unified style std::string& / char*
config azure requestTimeoutMs

Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-07 16:32:20 +08:00
PowderLi
0c0f012e03
add a custom http header: Accept-Type-Allow-Int64 (#27901)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-01 11:42:16 +08:00
zhagnlu
6060dd7ea8
Add chunk manager request timeout (#27692)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-10-23 20:08:08 +08:00
Xiaofan
d83869aaeb
Refine minio chunks manager (#27510)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-10-13 14:15:35 +08:00
PowderLi
8d3069b1db
update openssl to 3.1.2 (#27399)
deal with root path's normalization

Signed-off-by: PowderLi <min.li@zilliz.com>
2023-10-08 19:17:31 +08:00
PowderLi
4feb3fa7c6
support azure (#26398)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-09-19 10:01:23 +08:00
Enwei Jiao
0afdfdb9af
Remove other Exceptions, keeps SegcoreError only (#27017)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-14 14:05:20 +08:00
Enwei Jiao
c3f15c6b95
Refactor duplicate error class into one place (#26985)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-11 20:43:17 +08:00
zhagnlu
08547fe1a6
Add errcode to minio chunkmanager exception(#25153) (#25154)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-08-17 20:22:27 +08:00
zhagnlu
411f9ac823
Upgrade minio-go and add region and virtual host config for segcore chunk manager (#26194)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-08-11 10:37:36 +08:00
zhagnlu
833674c1cb
add glog configurable function and redirect aws log to segcore log (#25664)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-07-27 19:49:02 +08:00
shaoyue
7bf39aed08
chunk manager supports using gcs & oss with accessKey (#25233)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2023-07-20 15:38:58 +08:00
xige-16
04082b3de2
Migrate the ability to upload and download binlog to cpp (#22984)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-25 14:38:44 +08:00
zhagnlu
a79fd1c5a2
Fix GetObject return null value bug in MacOs (#24959) (#25002)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-06-21 13:40:42 +08:00
Bingyi Sun
cc56efb134
Fix build index performance downgrade (#24651) (#24708)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-06-19 10:50:47 +08:00
congqixia
cbfbc4e63a
Memset sigaction before using it (#24933)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-16 14:08:40 +08:00
congqixia
561021ec5a
Refine sigpipe handling (#24872)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-14 10:10:38 +08:00
congqixia
e729b504ab
Use SA_ONSTACK flag for SIGPIPE handler (#24688)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-08 15:50:35 +08:00
zhagnlu
0fe8342648
Support google cloud for C++ chunk manager (#21100) (#24418)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-05-29 01:05:28 +08:00
Enwei Jiao
66a5efeb3d
Fix load failed with DiskANN (#23760)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-04-27 16:56:35 +08:00
Enwei Jiao
ef57ba3bac
Remove easylogging and replace by GLog (#22979)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-03-27 18:22:02 +08:00
zhagnlu
7c633e9b9d
Support aliyun OSS for chunk manager (#22663) (#22842)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-03-20 14:45:57 +08:00
xige-16
9aa99aedbb
[Cherry-Pick] Remove arrow uasge in FieldData (#22726)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-03-20 10:41:56 +08:00
yah01
a4031da634
Refine string parameters, avoid coping or deref (#22708)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-13 17:53:53 +08:00
yah01
7bc3309918
Replace NULL with nullptr (#22701)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-13 10:35:52 +08:00
yah01
bdd6bc7695
Re-format cpp code (#22513)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-02 15:55:49 +08:00
xige-16
a20770c172
Delete logs that print sensitive information (#20889)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-12-01 10:35:16 +08:00
zhagnlu
cb2591d1fe
Parallel operations of disk-ann file manager (#20120) (#20122)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-11-03 14:39:40 +08:00
bigsheeper
37dce8a660
Make s3 client creation thread safe, prevent crashing (#20200)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-31 17:09:33 +08:00
xige-16
89a9b6fda0
Init storage config's SSL and IAM config (#20082)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-26 16:07:30 +08:00
xige-16
32ba4fafb8
Delete Creating bucket when create core/minioChunkManager (#20049)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-25 14:15:30 +08:00
zhagnlu
02a8a07322
Fix BucketExists bug in minio chunk manager (#19548) (#19854)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-10-20 10:23:27 +08:00
xige-16
8c9c1672ae
Assign different storage config for indexes (#19517)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-14 14:45:23 +08:00
xige-16
428840178c
Support diskann index for vector field (#19093)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-21 20:16:51 +08:00
xige-16
4de1bfe5bc
Add cpp data codec (#18538)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: zhagnlu lu.zhang@zilliz.com

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-09 22:12:34 +08:00