90 Commits

Author SHA1 Message Date
Cai Yudong
722f95df02
Abstract range search APIs into distances_range.cpp (#14485)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-29 11:43:24 +08:00
Xieql
9c1c7bc54b
[skip e2e] Improve annotation (#14386)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2021-12-28 13:27:17 +08:00
Cai Yudong
88b125643f
[skip e2e] Remove ConcurrentBitset.h reference (#14387)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-28 12:04:09 +08:00
Cai Yudong
5b8b68c9f2
[skip e2e] Remove redundant files (#14346)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-28 00:08:07 +08:00
Cai Yudong
fdfa93d385
Merge ConcurrentBit.h into BitsetView.h (#14353)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-28 00:02:06 +08:00
Xieql
b022615b2c
[skip e2e] improve annotation (#14127)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2021-12-27 10:24:17 +08:00
Xieql
89f3bbf454
[skip e2e] improve annotation (#14125)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2021-12-27 10:22:18 +08:00
Cai Yudong
132b3c2248
Remove redundant ConcurrentBitset dependency (#14230)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-26 01:20:18 +08:00
Cai Yudong
4be6a442e8
Remove index build suspend support (#14219)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-25 21:38:20 +08:00
Cai Yudong
86de70a4ba
[skip e2e] Disable FAISS GPU build (#14084)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 21:13:21 +08:00
Cai Yudong
583fc21e15
[skip e2e] Fix core/src/index build issue (#14083)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 21:11:16 +08:00
Xieql
18c0ff116b
[skip e2e] improve annotation (#14074)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2021-12-23 15:25:13 +08:00
Xieql
c769c77d98
[skip e2e] improve annotation (#14056)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2021-12-23 15:14:11 +08:00
Xieql
e6e58b69d5
[skip e2e] improve annotation (#14055)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2021-12-23 14:40:14 +08:00
Xieql
ac8ae914b4
[skip e2e] improve annotation (#13965)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2021-12-22 17:05:42 +08:00
Xieql
c36439b13e
[skip e2e] improve annotation (#13949)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2021-12-22 14:25:12 +08:00
Xieql
6af7741c5f
[skip e2e] improve annotation (#13948)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2021-12-22 14:23:23 +08:00
Xieql
f332e14034
[skip e2e] improve annotation (#13838)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2021-12-21 11:59:33 +08:00
cxytz01
88e399b729
[skip e2e] Grammar correction (#13772)
Signed-off-by: cxytz01 <xueyou.chen@zilliz.com>
2021-12-20 19:24:08 +08:00
Cai Yudong
85efcd8582
Move fiu nlohmann and easylogging to core/thirdparty (#12981)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-08 18:45:07 +08:00
Cai Yudong
45bac3e4ec
Move profiler under core/thirdparty (#12949)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-08 13:01:04 +08:00
Cai Yudong
df875c5985
Clean useless thirdparty version (#12944)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-08 12:15:03 +08:00
Cai Yudong
7b97b155e8
Remove duplicated thirdparty (#12925)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-08 10:37:37 +08:00
Cai Yudong
41ba52bb28
Support C unittest profiling (#12899)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-07 20:17:03 +08:00
Ji Bin
26b9d0e9f7
Support cpu/qemu only support SSE42 (#12337)
Compiler options for compiling faiss updated:
- for common objs using -mss4.2 for tunning, remove -mf16c, which may
  need newer CPU.
- for avx2 objs using -mavx2 and -mf16c
- for avx512 objs using -mavx512* and -mf16c

Issue: #12335

Signed-off-by: Ji Bin <matrixji@live.com>
2021-12-01 17:05:32 +08:00
congqixia
8a9fb9b287
Fix CMakeLists.txt comment error (#12141)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-19 19:33:13 +08:00
jaime
11110906a4
[skip ci] Add a TODO for a faiss cpp file (#12117)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-11-19 17:57:46 +08:00
jaime
c6fea64aa7
[skip ci] Add a comment to faiss CMakeLists (#12116)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-11-19 17:55:40 +08:00
dragondriver
daaeb27ee1
Make AdapterMgr.GetAdapter thread-safe (#11674)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-11 20:42:44 +08:00
Cai Yudong
14c37058e3
Rename simd configuration valid values for better readability (#11410)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-11-08 22:07:05 +08:00
Cai Yudong
caf3a2405d
Update KnowhereException (#11425)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-11-08 22:05:05 +08:00
Ji Bin
68d6839ab0
Improvements: Some adaptation for gcc-10 (#7853)
Support windows compile with MSYS (partial). This part including:

- CMakeLists.txt using CMAKE_SHARED_LIBRARY_SUFFIX instead .so
- Replace finite() with std::isfinite()
  ref: https://github.com/facebookresearch/faiss/pull/1341
- Replace bzero (deprecated in POSIX 2001) with memset.
  ref: https://github.com/facebookresearch/faiss/pull/1340
- Some int types match.

Signed-off-by: Ji Bin <matrixji@live.com>
2021-11-03 16:00:16 +08:00
Cai Yudong
492dd1a66f
Remove redundant file (#10674)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-26 19:32:33 +08:00
Cai Yudong
1227b9ebae
Rename utils/tools.h to utils/Utils.h (#10665)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-26 17:28:23 +08:00
Cai Yudong
fa01a888a0
Remove redundant files under internal/core (#10658)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-26 17:26:30 +08:00
groot
73c9ab43e0
Simplify cpp (#10413)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-10-22 14:33:11 +08:00
groot
a91486e966
Fix a compile error (#10358)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-10-21 19:10:37 +08:00
yukun
e91eafd871
Fix Bitsetview bug in segcore (#10272)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-10-20 16:34:36 +08:00
ZhiShen
6a770e5c38
Make knowhere compile independently (#7606)
Make knowhere compile independently

1. Make knowhere compile independently
    * Add gtest, arrow, and some other libraries to index.
    * Add cache, log and some other files to knowhere.
    * Add CMakeLists files to index's thirdparty.

2. Modified the compilation content of knowhere
    * Delete some content of compile library.
    * Add IMPORTED_GLOBAL property to faiss.

3.  Change the compilation location of some libraries
    * Make OpenBLas compiled in thirdpartycore.cmake.
    * Make faiss compiled in thirdparty/CMakeLists.

Change the content of knowhere/CMakeLists

1. Change easyloggingpp and nlohmann into index/thirdparty.
2. Change MILVUS_THIRDPARTY_SRC into KNOWHERE_THIRDPARTY_SRC.

Delete FindOpenBLAS

1. Delete Openblas.cmake.

2. The search task for openBlas is assigned to ThirdpartyCore.

3. Some changes were made to build.sh in index.

Fix the openBLas compilation problem

Delete the if-else in compilation of faiss;

Now when complie faiss, it will find the Openblas as we wish.

Fix some problem:

1. delete arrow

2. set openblas_source to AUTO

3. change a include_dir

4. delete MKL

5. delete the CMakeLists in index/utils,cache,log

Change variable build_test to knowhere_build_test in index/build.sh

Change the include location of  GNUInstallDirs

set CMAKE_INSTALL_LIBDIR

Resolves: milvus-io#5183
See also: milvus-io#6604

Signed-off-by: Shen Zhi <m13120163046@163.com>
2021-10-13 17:06:33 +08:00
yukun
6c88774624
Add get final filtered bitmap in SegmentGrowing (#9780)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-10-13 16:54:34 +08:00
dragondriver
a10f421c14
Get SIMD type used in faiss (#8849)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-29 20:50:19 +08:00
ZhiShen
0e767f0e1b
Change cmake version (#8666)
Change cmake version from 3.16 to 3.18

1. Change the CMakeList's minimum require version.

2. Change the describtion document of cmake version.

Resolves: milvus-io#8407

Signed-off-by: Shen Zhi <m13120163046@163.com>
2021-09-28 14:00:03 +08:00
groot
985e63cbf2
Reduce cpp unittest time cost (#8424)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-23 21:50:11 +08:00
groot
b897ca5703
Fix unittest build problem (#8369)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-23 14:19:54 +08:00
Cai Yudong
c1e229cb7e
Add segcore logger (#8262)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-22 10:11:52 +08:00
Cai Yudong
e6c384b4d4
Move segcore chunk_size configuration to querynode.yaml (#7913)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-15 10:35:52 +08:00
yukun
5c997710ab
Change Assert() to AssertInfo() to return error messages (#7843)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-14 10:06:40 +08:00
xige-16
468bef4478
Fix dataset's memory leak in knowWhere when search (#7527)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-09-07 14:25:59 +08:00
xige-16
76066b5659
Fix IVF_FLAT index's wrong meta_data (#7167)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-08-19 14:26:11 +08:00
dragondriver
d2767f920c
Fix index parameters check of RHNSWPQ (#6950)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-08-03 10:25:25 +08:00