45 Commits

Author SHA1 Message Date
yudong.cai
6b81dd55df #1536 move Config.cpp to config directory
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-07 17:02:21 +08:00
BossZou
1963f3b9e9
Make cache_insert_data take effect in-service(# 1521) (#1540)
* make cache_insert_data take effect in-service (fix #1521)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* complete omitted config setter

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* cache_insert_data take effect for MemTable

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add config test

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* rename config handler

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-03-07 09:34:35 +08:00
BossZou
df47fad78c
Add setter API for config preload_table (#1526)
* update code

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* optimize http module code

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* move gpu res handler to config holder

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Add setter API for config preload_table (fix #1525)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add else branch at end of if condition

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-03-06 13:26:01 +08:00
BossZou
c63a50c635
Allow system conf modifiable and some take effect directly #1263 (#1287)
* finish configure modifiable and some take effect runtime

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* move test file to unittet folder

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add test case for cofig

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* fix format issue

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* optimize callback

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* fix error when get config by http in CPU mode

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update callback map

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* correct typo

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update callback register

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add identity in registers

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add identity in FaissPass class

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add comments at cmd test if fail

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* remove some comments and add test for validate storage path

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* format pass

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* remove endpoint info and update CORS

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add test case in web module to test drop a non-existent table

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* optimize config store

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add check code in config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add scheduler gpu handler

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update optimizer faiss pass

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add default value of gpu_enable_ in GpuCacheMgr

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* optimize code: remove comments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* format pass

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add gpu macro in optimizer handler module to fix compile bug on cpu version

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* change cache setting unit  bytes

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* rename variables

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-02-26 18:54:32 +08:00
Jin Hai
e1027e9e65
#1240 Update license declaration of each file (#1241)
* #1240 Update license declaration of each files

Signed-off-by: jinhai <hai.jin@zilliz.com>

* #1240 Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-02-17 23:40:58 +08:00
shengjh
4dee7dfac6
Improve codecov (#1095)
* Optimize config test. Dir src/config 99% lines covered

* add unittest coverage

* optimize cache&config unittest

* code format

* format

* format code

* fix merge conflict

* cover src/utils unittest

*  '#831 fix exe_path judge error'

* #831 fix exe_path judge error

* add some unittest coverage

* add some unittest coverage

* improve coverage of src/wrapper

* improve src/wrapper coverage

* *test optimize db/meta unittest

* fix bug

* *test optimize mysqlMetaImpl unittest

* *style: format code

* import server& scheduler unittest coverage

* handover next work

* *test: add some test_meta test case

* *format code

* *fix: fix typo

* feat(codecov): improve code coverage for src/db(#872)

* feat(codecov): improve code coverage for src/db/engine(#872)

* feat(codecov): improve code coverage(#872)

* fix config unittest bug

* feat(codecov): improve code coverage core/db/engine(#872)

* feat(codecov): improve code coverage core/knowhere

* feat(codecov): improve code coverage core/knowhere

* feat(codecov): improve code coverage

* feat(codecov): fix cpu test some error

* feat(codecov): improve code coverage

* feat(codecov): rename some fiu

* fix(db/meta): fix switch/case default action

* feat(codecov): improve code coverage(#872)
* fix error caused by merge code
* format code

* feat(codecov): improve code coverage & format code(#872)

* feat(codecov): fix test error(#872)

* feat(codecov): fix unittest test_mem(#872)

* feat(codecov): fix unittest(#872)

* feat(codecov): fix unittest for resource manager(#872)

* feat(codecov): code format (#872)

* feat(codecov): trigger ci(#872)

* fix(RequestScheduler): remove a wrong sleep statement

* test(test_rpc): fix rpc test

* Fix format issue

* Remove unused comments

* Fix unit test error

Co-authored-by: ABNER-1 <ABNER-1@users.noreply.github.com>
Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-02-11 23:03:29 +08:00
aaronjin2010
c787b84daa refactor code and add error log (#976)
* refactor code and add error log

* remove the space

* refactor

* refactor
2020-01-13 09:30:14 +08:00
fishpenguin
10d50d2fb7 IVF_PQ search on CPUs when using GPU-version 2019-11-30 14:57:35 +08:00
fishpenguin
a718c7c9bb Optimizer specify wrong gpu_id 2019-11-28 19:08:15 +08:00
Yukikaze-CZR
47abe4ff47 #504 The code coverage rate of core/src/scheduler/optimizer is too low 2019-11-27 14:21:53 +08:00
Jin Hai
04ab48f25d
Merge pull request #496 from yhmo/0.6.0
#485 Increase code coverage rate
2019-11-23 14:46:37 +08:00
fishpenguin
8a4cc9263e [skip ci]Add more log msg 2019-11-23 11:56:34 +08:00
groot
2b805489a1 #485 Increase code coverage rate 2019-11-23 11:34:45 +08:00
fishpenguin
288490f980 gpu no usage during index building 2019-11-23 11:07:42 +08:00
fishpenguin
b68f4f43c1 Add log in scheduler/optimizer 2019-11-23 10:29:11 +08:00
yudong.cai
c89bb6699e #306 Use int64 for all config integer 2019-11-20 14:26:39 +08:00
fishpenguin
a13677b65f Refactor optimizer 2019-11-19 21:10:43 +08:00
fishpenguin
30dcfa6719 fix conflict in SchedInst 2019-11-19 19:30:53 +08:00
fishpenguin
7f5c2fefd8 Remove OnlyGpuPass and add MILVUS_GPU_VERSION define 2019-11-19 19:18:54 +08:00
wxyu
9f063aa289 Add a Fallback pass in optimizer close #409
add Fallback pass condition
2019-11-19 17:11:14 +08:00
fishpenguin
caacf40ed2 fix for config change 2019-11-19 14:18:45 +08:00
wxyu
7ad5c51497 Add virtual method Init() in Pass abstract class close #404 2019-11-19 12:29:05 +08:00
fishpenguin
35f1796355 fix conflicts 2019-11-18 14:32:09 +08:00
fishpenguin
513ad3b842 Support build index with multiple gpu 2019-11-18 11:38:48 +08:00
wxyu
1b0ef89e07 fix flat search doesn't use gpu bug 2019-11-15 19:45:17 +08:00
youny626
4ca6b59b91 add CPU version 2019-11-05 10:23:59 +08:00
fishpenguin
63033ebc1f Code format
Former-commit-id: 8c9c010fd45c0a6737b8ad6c4493469440292707
2019-11-01 20:14:18 +08:00
fishpenguin
44d648c3af #164 - Add CPU version for building index
Former-commit-id: 02f9f2a04dc765f41c474af04659f70d121cdc18
2019-11-01 17:01:57 +08:00
yudong.cai
6ad1ed74e9 #159 Change the configuration name from 'use_gpu_threshold' to 'gpu_search_threshold'
Former-commit-id: b4e576b389a720d66ec5455ca05b20184c8c911d
2019-11-01 14:51:20 +08:00
fishpenguin
32dac025af Merge remote-tracking branch 'upstream/0.5.1' into branch-0.5.1-yk
Former-commit-id: 7b3f48e3acfa6c35b62dc5de8731a6b85aaa5dce
2019-10-31 16:41:59 +08:00
fishpenguin
1dc7b321a4 Read config once in optimizer
Former-commit-id: 8fee2a3cfa20cdc8b51e86867b7b3eb71dd2e759
2019-10-31 16:41:48 +08:00
fishpenguin
a68255f9d0 clang format
Former-commit-id: 3f99d7df6b042683696dd70c9dc0f14fee57b388
2019-10-31 14:44:59 +08:00
fishpenguin
49105e44f3 Add only GPU and only CPU version for IVF_SQ8 and IVF_FLAT
Former-commit-id: 14f5b095df2988d5f65ecd1cb2628fd3a3791a14
2019-10-31 14:39:02 +08:00
wxyu
892bd4daac Improve large query optimizer pass
Former-commit-id: 1f0c283ec4dc5560c14d4e17d76719d38dfc2280
2019-10-31 14:30:17 +08:00
wxyu
87963408bf fix lint
Former-commit-id: 2f5166b4c2a461c379c55eb5a49d431e878aa19a
2019-10-30 20:24:10 +08:00
wxyu
7dc552fdea New config opion use_gpu_threshold
Former-commit-id: 3d33145d8fa473f2b81cc98022c1bb93853a2fbc
2019-10-30 19:57:55 +08:00
xiaojun.lin
f23e8e2143 clang-format...
Former-commit-id: 3bd5e246b34d8ca7800dbc6ce0e784dfa26ce18a
2019-10-19 17:51:59 +08:00
xiaojun.lin
6572e6bf25 MS-671 Fix HybridIndex Crash
MS-670 Adapt to faiss-1.6


Former-commit-id: 12300b390b6d300dc460f0729a368a240c972a80
2019-10-19 17:49:28 +08:00
Yu Kun
1f7dad0497 fix clang format
Former-commit-id: b158a9b2a85a182882a666c068601f7dd4ef1f23
2019-10-17 19:38:03 +08:00
Yu Kun
7a10d01fa1 add wrapper unittest
Former-commit-id: b4f68eeaf55e4960f25239e983c3ae75e5e5ed40
2019-10-17 19:37:17 +08:00
Yu Kun
01de85b5c7 annotate some unused code to improve code coverage
Former-commit-id: 801df9b679a8a3622473ddefb81ce1baba524a04
2019-10-17 17:16:52 +08:00
Yu Kun
e819c96777 23 Add unittest to improve code coverage
Former-commit-id: 5d5629da7cad9503816f63917425bcde3c5b6a90
2019-10-17 09:56:56 +08:00
Yu Kun
6c7bb57cb2 add unitteest
Former-commit-id: 9ad0fddedf0ace50845a7d516452b986162204f8
2019-10-16 20:46:09 +08:00
starlord
994274a130 merge 0.5.0 branch
Former-commit-id: c687efe7864690004aecb296c1b75ce7084eb315
2019-10-14 11:44:56 +08:00
starlord
41d37f9648 re-organize project
Former-commit-id: 898c2c8b0b2cfdc1b8664062aa0ea910a93566dd
2019-10-14 09:51:48 +08:00