admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
18,616 Commits 137 Branches 188 Tags
Commit Graph

1083 Commits

Author SHA1 Message Date
cai.zhang
49b8657f95
enhance: Support implicit type conversion for parquet (#29046) 
issue: #29019

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-12 16:14:44 +08:00
Enwei Jiao
0e65e90338
enhance: Support otlp with insecure (#29115) 
issue: https://github.com/milvus-io/milvus/issues/28914

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-12-12 11:14:37 +08:00
MrPresent-Han
464bc9e8f4
fix: fix reduce precision for search(#27325) (#29031) 
related: #27325

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-12-08 10:04:37 +08:00
congqixia
dcb662d9ed
enhance: Refine C.NewSegment response and handle exception (#28952) 
See also #28795

Orignal `C.NewSegment` may panic if some condition is not met, this pr
changes response struct to `CNewSegmentResult`, which contains
`C.CStatus` and may return catched exception

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-07 13:34:35 +08:00
cai.zhang
fb089cda8b
enhance: Load raw data while scalar index doesn't have raw data (#28888) 
issue: #28886

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-06 20:36:36 +08:00
Bingyi Sun
36f69ea031
feat: integrate storagev2 in building index of segcore (#28768) 
issue: https://github.com/milvus-io/milvus/issues/28655

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-05 16:48:54 +08:00
PowderLi
20fc90c591
enhance: find collection schema from cache (#28782) 
issue: #28781 #28329

1. There is no need to call `DescribeCollection`, if the collection's
schema is found in the globalMetaCache
2. did `GetProperties` to check the access to Azure Blob Service while
construct the ChunkManager

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-03 19:22:33 +08:00
yah01
342635ed61
enhance: enable assert method to format arguments (#28812) 
for now the assert method in segcore could accept a string information,
too many codes don't print the value they assert.

make it happy
related #28811

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-01 18:04:33 +08:00
yihao.dai
f5856812a2
fix: Fix get binary vector from chunk cache (#28866) 
The way of getting binary vector size is wrong. This PR will fix it.

issue: https://github.com/milvus-io/milvus/issues/28865

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-12-01 14:40:32 +08:00
Bingyi Sun
8036ee13fa
feat: avoid dereferencing nullptr (#28862) 
issue: #28793

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-01 10:20:32 +08:00
PowderLi
cac802ef7f
enhance: use already installed vcpkg (#28703) 
issue #28686
1. Update Builder gpu image changes, see changes #28505
2. update azure-identity-cpp from beta to release

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-30 15:58:32 +08:00
yah01
d69440524b
fix: bypass growing index if no index meta (#28791) 
we shouldn't panic if no index meta, just skip building it
fix #28022

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-11-30 14:10:27 +08:00
congqixia
1dc086496f
fix: schema->size() check logic with system field (#28802) 
Now segcore load system field info as well, the growing segment
assertion shall not pass with "+ 2" value
This will cause all growing segments load failure
Fix #28801
Related to #28478
See also #28524

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-29 22:40:28 +08:00
cqy123456
3b1b14dd78
fix: update binlog index memory uasge before loading segments (#28528) 
issue: #27678 
when interimIndex = true, memory predict should be update with the
memory usage of binlog index build process.

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-11-29 16:42:27 +08:00
cai.zhang
6f7a9264d5
enhance: Handle knowhere error for creare diskann index (#28690) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-24 11:58:23 +08:00
zhagnlu
0d9d098186
enhance: Add precheck when chunk manager init (#28330) 
#28329

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-11-23 19:56:32 +08:00
zhagnlu
e9e9b30e8e
enhance: Disable stdout buffer (#28391) 
#28390

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-11-23 19:26:25 +08:00
Bingyi Sun
e5ce385ffd
enhance: remove -inl.h files (#28674) 
issue: https://github.com/milvus-io/milvus/issues/28673
Move template implementations from -inl.h to .cpp file and make explicit
instantiation

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-11-23 17:20:25 +08:00
yah01
c96d07682e
enhance: enhance the accuracy of memory usage (#28554) 
before this, Milvus use container/system's memory info to get the memory
usage, which could be inaccurate.

we allocates the memory by private anon mmap,
then `rss - shared` would be the accurate memory usage

resolve #28553

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-23 15:12:23 +08:00
Bingyi Sun
e8c5e75786
enhance: ignore proto generated files (#28537) 
ignore proto generated files
issue: https://github.com/milvus-io/milvus/issues/28566

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-11-22 10:20:22 +08:00
yah01
bfccfcd0ca
enhance: refine error messages (#28424) 
- Split the simple reason and full detail
- Refine existing error messages
related: #28422

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-21 17:02:24 +08:00
yah01
02c5a649cf
enhance: store system fields in segcore (#28524) 
we need the system fields info for some usacase
fix: #28523

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-21 09:28:22 +08:00
PowderLi
a1c505dbd5
add internal storage metrics (#28278) 
/kind improvement
issue: #28277

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-19 17:22:25 +08:00
yah01
f7d2ab6677
enhance: reduce 1x copy for variable length field while retrieving (#28345) 
- Reduce 1x copy for varchar/string/JSON/array types while retrieving
- Reduce 1x copy for int8/int16 while retrieving

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-15 18:08:20 +08:00
MrPresent-Han
836f300536
support skip-index based on chunk-metrics to accelerate expr filter(#27925) (#28297) 
related: #27925

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-11-15 11:20:19 +08:00
Xu Tong
8ec85f5f4c
Add template for VectorMemIndex (#28324) 
Signed-off-by: Writer-X <1256866856@qq.com>
 2023-11-11 13:20:22 +08:00
yah01
267c67dfee
enhance: reduce 1x copy while retrieving data from growing segment (#28323) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-10 15:44:22 +08:00
zhagnlu
3920bbc55f
Force set aliyun use_virtual_host to true for all (#28158) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-11-10 10:32:20 +08:00
cai.zhang
01c5484817
No hit when the index exceeds the array length (#28301) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-10 10:04:19 +08:00
yah01
30847cad3e
Handle exception while loading (#28304) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-09 17:59:12 +08:00
PowderLi
9f9726f8b7
print azure sdk log (#28240) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-08 17:50:18 +08:00
cai.zhang
2b5f632fa4
Fix bug for constructing ArrayView with fixed-length type (#28185) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-07 23:38:21 +08:00
PowderLi
7bb0fa9c70
reduce useless ObjectExists (#28156) 
replace ListBlobs() with GetProperties()
unified style std::string& / char*
config azure requestTimeoutMs

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-07 16:32:20 +08:00
yah01
863e26969a
Reduce 1x memory copy for retrieving data (#28106) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-06 17:48:19 +08:00
cai.zhang
8011054a2a
Check length before comparing strings (#28110) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-04 10:04:29 +08:00
congqixia
501cba90ce
Use IVFFLAT_CC instead of IVFFLAT for sealed segment tmp index (#28164) 
See also #28152

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-03 23:50:15 +08:00
cai.zhang
fc2df9514f
Refine code for fixed-length types array (#28108) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-03 00:40:14 +08:00
yah01
dc89730a50
Support collection-level mmap control (#26901) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-02 23:52:16 +08:00
Enwei Jiao
f8dd589755
Refactor collection's cgo call (#28055) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-11-02 13:02:13 +08:00
yihao.dai
873b29e226
Fix unstable cpp ut (#28083) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-11-02 00:58:16 +08:00
MrPresent-Han
854c0e8a16
fix precision for segcore reduce(#27325) (#28062) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-11-01 21:52:13 +08:00
PowderLi
0c0f012e03
add a custom http header: Accept-Type-Allow-Int64 (#27901) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-01 11:42:16 +08:00
Enwei Jiao
8ae9c947ae
Use OpenDAL to access object store (#25642) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-11-01 09:00:14 +08:00
cqy123456
4fbe3c9142
replace loaded binlog with binlog index for search performance (#27673) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-11-01 02:20:15 +08:00
yah01
1b9b8a31a8
Fix QueryNode panic while upgrading (#28034) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-31 15:50:13 +08:00
yah01
ab6dbf7659
Limit max thread num for pool (#28018) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-31 13:00:13 +08:00
yah01
2af46d7333
Increase the ChunkManager request timeout (#28015) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-31 09:06:13 +08:00
yihao.dai
ab6b0103a3
Get vector concurrently (#27838) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-30 15:44:12 +08:00
yah01
f212158d61
Fix delete records timestamp may be reordered (#27941) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-27 10:24:10 +08:00
Enwei Jiao
4a33391b8f
rename createindex (#27903) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-10-27 10:12:14 +08:00