admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00
Code Issues Packages Projects Releases Wiki Activity
5,522 Commits 146 Branches 196 Tags
Commit Graph

1179 Commits

Author SHA1 Message Date
groot
762f5f296f
#4066 Rewrite SqliteMetaImpl to discard sqlite_orm (#4114) 
* #4066 Rewrite SqliteMetaImpl to discard sqlite_orm

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix unittest

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-10-29 15:29:02 +08:00
groot
84083294c3
fix typo of #4075 (#4094) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-10-26 14:06:52 +08:00
groot
773deab62e
#4075 Improve performance for create large amount of partitions (#4083) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-10-24 10:35:58 +08:00
cqy123456
52dbf7ed3b
Let uid be shared pointer to not copy really in 0.10.4 (#4064) 
* Let uid be shared pointer to not copy really in 0.10.4

Signed-off-by: cqy <yaya645@126.com>
 2020-10-23 19:30:13 +08:00
jackhouchina
8d5c1a87d5
#3773 milvus support IVF_PQ run on fpga device (#3996) 
* This is my commit message Signed-off-by jack hou Developer <364427912@qq.com>

Signed-off-by: root <364427912@qq.com>

* fix build with -u -a

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix run on GPU

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix ThirdPartyPackages.cmake

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
 2020-10-23 09:41:28 +08:00
cqy123456
0b020b8255
sq8h loading problem (#4060) 
Signed-off-by: cqy <yaya645@126.com>
 2020-10-22 17:11:46 +08:00
shengjun.li
8e0f0a3437
Revert PR3525 and PR3553 (#4014) 
* Revert "add NGT index (#3525)"

This reverts commit c2ba9be24986d3155c4b87ae01601233246a9540.

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* Revert "[skip ci]add license (#3553)"

This reverts commit 5787a2af4d44a7258c37b9f481c286dbd7b6b44f.

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-10-17 18:51:00 +08:00
shengjun.li
235c51c385
Fix server crashed during search with index pq (#3954) 
* fix pq distances cal bug

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix pq distance calculation outofrange according to faiss suggests

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix change log

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
 2020-10-17 09:42:19 +08:00
groot
23b94306fa
#3998 fix a dead lock issue (#4011) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-10-16 19:29:48 +08:00
op-hunter
e2f63c9bd1
withdraw pr 2467, which is a failure optimization of k-selection (#3999) 
Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
 2020-10-15 20:43:25 +08:00
shengjun.li
3596d07e74
fix version (#3971) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-10-13 15:42:30 +08:00
cqy123456
19bda5cf97
The search of SQ8H index is slower than the previous version (#3957) 
Signed-off-by: cqy <yaya645@126.com>
 2020-10-12 19:55:26 +08:00
Wang Xiangyu
6836c81f44
Support topk up to 16384 on GPU (#3936) 
* large topk on gpu

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>

* change GPU_QUERY_MAX_TOPK to 16384

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
 2020-10-12 09:48:04 +08:00
chen qingxiang
f46e2a621d
fix the bug of DeleteTask state not change (#3935) 
* fix the bug of DeleteTask state not change

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2020-10-10 14:16:31 +08:00
Wang Xiangyu
e397dd20b9
Disable query slicing (#3931) 
Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
 2020-10-10 09:43:25 +08:00
lighteningzhang
237ce2aba3
Modify faiss files to reduce the usage of GPU in 0.10.4 (#3893) 
* Modify faiss files to reduce the usage of GPU in 0.10.4

Signed-off-by: lighteningzhang <lighteningzhang@163.com>

* Change the format of the code and release memories of arrays resident on host

Signed-off-by: lighteningzhang <lighteningzhang@163.com>

* Change the format of the code and release memories of arrays resident on host, fix some typos

Signed-off-by: lighteningzhang <lighteningzhang@163.com>

* Delete some blank lines

Signed-off-by: lighteningzhang <lighteningzhang@163.com>

* Temporarily comment the delete function in IVFFlat.cu

Signed-off-by: lighteningzhang <lighteningzhang@163.com>

* Uncomment some delete functions

Signed-off-by: lighteningzhang <lighteningzhang@163.com>

* Uncomment some delete functions

Signed-off-by: lighteningzhang <lighteningzhang@163.com>

* Use tensor to replace arrays in IVFFlat.cu

Signed-off-by: lighteningzhang <lighteningzhang@163.com>

* Fix some typos

Signed-off-by: lighteningzhang <lighteningzhang@163.com>
 2020-09-28 17:58:32 +08:00
shengjun.li
7692a6d437
set blacklist a NULL pointer if no item deleted (#3827) 
* set blacklist a nullptr is no item deleted

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix multi thread access bitset

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-09-22 10:01:04 +08:00
op-hunter
7a2d847743
raise the limitation of hnsw ef parameter to 32768 (#3738) 
* raise hnsw ef parameter to 32768

Signed-off-by: cmli <chengming.li@zilliz.com>
 2020-09-16 09:20:56 +08:00
shengjun.li
18695db507
If the GPU cache is too small, IVF_SQ8H using multiple GPUs will cause to crash (#3744) 
* fix ivf_sq8h

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix cache release

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-09-15 20:21:10 +08:00
shengjun.li
0af3804974
fix large param (#3684) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-09-10 19:32:51 +08:00
cqy123456
dad5806b61
Making the IVFSQ8H support auto-tune CPU search when nprobe > 2048 (#3677) 
* ivfsq8h

Signed-off-by: cqy <yaya645@126.com>

* fix gtest

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
 2020-09-10 14:14:34 +08:00
shengjun.li
384b09e993
fix stucked tasiks (#3666) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-09-10 09:14:03 +08:00
cqy123456
49a5cdf682
When the topk of GPU search larger than 2048,turning into the CPU search (#3647) 
* topk of GPU searching

Signed-off-by: cqy <yaya645@126.com>

* change the changelog

Signed-off-by: cqy <yaya645@126.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
 2020-09-09 13:32:10 +08:00
shengjun.li
74639a3ba7
fix to check search param of BIN_IVF_FLAT (#3660) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-09-09 09:14:40 +08:00
cqy123456
8a52fc70d9
selector code refactoring (#3617) 
* selector code refactoring

Signed-off-by: cqy <yaya645@126.com>

* modify test

Signed-off-by: cqy <yaya645@126.com>
 2020-09-08 13:58:50 +08:00
shengjun.li
18c630a4be
release search task in time (#3609) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-09-05 11:36:55 +08:00
cqy123456
d5ecdf2dda
GPU search turn into CPU search when nprobe >2048 (#3594) 
* gpu nprobe

Signed-off-by: cqy <yaya645@126.com>

* [skip ci] changelog

Signed-off-by: cqy <yaya645@126.com>
 2020-09-04 18:49:52 +08:00
shengjun.li
273863f54d
fix users' metric type (#3568) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-09-03 10:21:26 +08:00
shengjun.li
5787a2af4d
[skip ci]add license (#3553) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-09-02 16:02:09 +08:00
flynn
c2ba9be249
add NGT index (#3525) 
Signed-off-by: fenglv <fenglv15@mails.ucas.ac.cn>

add LISENCE

Signed-off-by: fenglv <fenglv15@mails.ucas.ac.cn>

format code style

Signed-off-by: fenglv <fenglv15@mails.ucas.ac.cn>
 2020-09-02 10:21:59 +08:00
cqy123456
a75c0a572c
Allow more choices for the parameter ‘m’ of CPU IVF_PQ (#3469) 
* fix the IVFPQ fixed M problem

Signed-off-by: cqy <yaya645@126.com>

* fix compile

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
 2020-08-27 11:37:25 +08:00
shengjun.li
83cbe0f490
fix hamming (#3338) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-08-20 09:42:04 +08:00
MXDA
1962b6e78b
Specifying distance type at runtime for Flat index (#3273) 
* modfied IndexIDMAP.cpp && SearchTask.cpp

Signed-off-by: Zhengda Wu <wuzhengda55@gmail.com>

* Specifying distance type at runtime for Flat index(CPU)

Signed-off-by: Zhengda Wu <wuzhengda55@gmail.com>

* Specifying distance type at runtime for Flat index(GPU)

Signed-off-by: Zhengda Wu <wuzhengda55@gmail.com>

Co-authored-by: Zhengda Wu <wuzhengda55@gmail.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
 2020-08-17 17:05:59 +08:00
shengjun.li
5f830293f9
rm find faiss (#3269) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-08-15 19:18:03 +08:00
jinhai
f59c240457 Update to 0.10.3 
Signed-off-by: jinhai <hai.jin@zilliz.com>
 2020-08-15 16:21:43 +08:00
shengjun.li
43c272fd13
fix annoy l2 (#3232) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-08-12 19:11:13 +08:00
Xiaohai Xu
058cdf03bb
Add ivf index nprobe logging (#3070) 
* add nprobe logging

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* add all cases

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
 2020-08-01 09:46:53 +08:00
shengjun.li
09cf1b323e
fix crash when GPU search (#3074) 
* fix crash when GPU search

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-07-31 09:46:55 +08:00
shengjun.li
b1b1165c56
Opt faiss builder (#3067) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-07-31 09:35:31 +08:00
shengjun.li
1c36e6d83d
optimizer parallel policy (#3014) 
* optimizer parallel policy

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* modify parallel policy

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-07-26 01:37:23 +08:00
BossZou
c394608d4f
[skip ci] Up to v0.10.2 (#3006) 
Signed-off-by: yhz <413554850@qq.com>
 2020-07-25 10:02:09 +08:00
cqy123456
91d3fe5cbc
Improve IVF search performance when NQ and nProbe are both large (#2984) 
* fix indexflat  search

Signed-off-by: cqy <yaya645@126.com>

* add parallel_policy_threshold

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
 2020-07-24 15:56:47 +08:00
Tinkerrr
dd938878ea
fix (#2981) 
Signed-off-by: Tinkerrr <xiaojun.lin@zilliz.com>
 2020-07-23 17:02:28 +08:00
shengjun.li
3908d10498
Fix the result merging of IVF_PQ IP (#2953) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-07-22 10:08:05 +08:00
shengjun.li
a88f5b6cbb
Fix index size (#2941) 
* add virtual IF UpdateIndexSize

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* update ivf index size

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix PQ logic

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix index size of index hnsw, annoy and nsg

Signed-off-by: cmli <chengming.li@zilliz.com>

* add GetSize() interface 4 SPTAG

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix binary ivf

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: sahuang <xiaohai.xu@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
 2020-07-21 14:26:08 +08:00
groot
f31a81ab16
add error log for empty result (#2923) 
* add error log for empty result

Signed-off-by: groot <yihua.mo@zilliz.com>

* add error log for empty result

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-07-18 19:30:29 +08:00
godchen0212
c083556189
add web_impl util unittest (#2886) 
* add testcase of empty vector insert

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format python code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* add web_impl util unittest

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2020-07-18 10:38:46 +08:00
Tinkerrr
8046f2946e
Hotfix 2487 (#2892) 
* update.

Signed-off-by: Tinkerrr <linxiaojun.cn@outlook.com>

* update.

Signed-off-by: Tinkerrr <xiaojun.lin@zilliz.com>

* add fd check

Signed-off-by: Tinkerrr <xiaojun.lin@zilliz.com>

* fix

Signed-off-by: Tinkerrr <xiaojun.lin@zilliz.com>
 2020-07-17 21:00:42 +08:00
Wang XiangYu
283e087a78
add config value gpu.cache.enable (#2899) 
* add config value gpu.cache.enable

Signed-off-by: wxyu <xy.wang@zilliz.com>

* clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix ut

Signed-off-by: wxyu <xy.wang@zilliz.com>
 2020-07-17 20:59:31 +08:00
XuanYang-cn
8135cee414
Add some cpp unittests about scheduler algorithm (#2888) 
* add some testcases

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* reformat code

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
 2020-07-16 19:46:21 +08:00