admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00
Code Issues Packages Projects Releases Wiki Activity
5,997 Commits 146 Branches 196 Tags
Commit Graph

95 Commits

Author SHA1 Message Date
groot
28eca5de38
wal implement (#3464) 
* refactor LogMgr (#3372)

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* add wal unittest (#3367)

* add wal unittest

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix test fail

Signed-off-by: groot <yihua.mo@zilliz.com>

* Find system headers with cmake and clang-tidy (#3364)

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip ci]add google/crc32c into NOTICE.md (#3373)

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* prepare change memmanager for wal

Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip ci]update issue template (#3379)

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* fix wal test case

Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip ci] fix java sdk test case (#3375)

* java main class

Signed-off-by: zongyufen <zongyufen@foxmail.com>

* [skip ci] fix java sdk test

Signed-off-by: zongyufen <zongyufen@foxmail.com>

* [skip ci] fix java sdk test

Signed-off-by: zongyufen <zongyufen@foxmail.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* optimize download & compile of sqlite (#3361)

* optimize sqlite

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* rm sqlite url

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix bug

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip ci] rm log (#3378)

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip ci]  fix generate default entities (#3382)

* java main class

Signed-off-by: zongyufen <zongyufen@foxmail.com>

* [skip ci] fix java sdk test

Signed-off-by: zongyufen <zongyufen@foxmail.com>

* [skip ci] fix java sdk test

Signed-off-by: zongyufen <zongyufen@foxmail.com>

* [skip ci] fix generate default entities

Signed-off-by: zongyufen <zongyufen@foxmail.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Skip flat search params (#3381)

* assert top ids

Signed-off-by: zw <zw@milvus.io>

* update milvus-helm to 0.11.0

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip ci] update clang-tidy rules (#3386)

* [skip ci] update clang-tidy rules

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* [skip ci] update clang-tidy rules

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Optimize download & compile of Aws (#3384)

* runable aws

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* optimize aws

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip ci] modify test flush and compact (#3390)

* java main class

Signed-off-by: zongyufen <zongyufen@foxmail.com>

* [skip ci] fix java sdk test

Signed-off-by: zongyufen <zongyufen@foxmail.com>

* [skip ci] fix java sdk test

Signed-off-by: zongyufen <zongyufen@foxmail.com>

* [skip ci] fix generate default entities

Signed-off-by: zongyufen <zongyufen@foxmail.com>

* [skip ci] modify test flush

Signed-off-by: zongyufen <zongyufen@foxmail.com>

* [skip ci] modify test compact

Signed-off-by: zongyufen <zongyufen@foxmail.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* rewrite insert memmanager

Signed-off-by: groot <yihua.mo@zilliz.com>

* Optimize thirdparty download workflow  (#3394)

* Optimize thirdparty download workflow

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Optimize thirdparty download workflow

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Optimize thirdparty download workflow

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Optimize thirdparty download workflow

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Enlarge event queue (#3393)

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Add GetPageEntity unittest (#3397)

* Add web server interface

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add unittest/server

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add web server ut

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix web server insert bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix web server ut crash bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix web server ut gpu compile error

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix codacy quality

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix row_num error

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change row insert to column insert

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix dsl issue

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix GetEntityByID bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add GetPageEntity interface

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix some webserver bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove server_config.yaml

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Fix thirdparty not find ccache but still use it (#3398)

* fix sqlite ccache

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* thirdparty EP using ccache configure

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* optimize oatpp  (#3377)

* optimize oatpp

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix some bug

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fit atomic not find bug

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add one config fo oatpp

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* change oatpp version

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* comment url_md5

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* change oatpp version

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* change aws target name

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Split Ftype into Ftype and FEtype (#3341)

* fix include directories not find bug (#3323)

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* reduce grpc download file size from 380M to 130M  (#3326)

* reduce grpc download file size from 380M to 130M

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix bug

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* update clang-tidy config (#3314)

* update clang-tidy config

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update .clang-tidy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update clang-tidy config

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update clang-tidy HeaderFilterRegex

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* enable fetype

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* rename field_element_method

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* add FETYPE_TYPE

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* Add WebServer unittest (#3321)

* Add web server interface

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add unittest/server

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add web server ut

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* modify db schema and remove conversion

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* weak fetype

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* enforce strong fetype

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* lint

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* format code

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* [skip ci] fix

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

Co-authored-by: XuanYang-cn <51370125+XuanYang-cn@users.noreply.github.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: yukun <kun.yu@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Master codacy check (#3366)

* codacy fix

Signed-off-by: cqy <yaya645@126.com>

* codacy

Signed-off-by: cqy <yaya645@126.com>

* codacy

Signed-off-by: cqy <yaya645@126.com>

* codacy check

Signed-off-by: cqy <yaya645@126.com>

* codacy

Signed-off-by: cqy <yaya645@126.com>

* codacy

Signed-off-by: cqy <yaya645@126.com>

* codacy

Signed-off-by: cqy <yaya645@126.com>

* clang-tiny

Signed-off-by: cqy <yaya645@126.com>

* clang-tidy check

Signed-off-by: cqy <yaya645@126.com>

* clang-tindy check

Signed-off-by: cqy <yaya645@126.com>

* clang-tidy check

Signed-off-by: cqy <yaya645@126.com>

* clang-tidy

Signed-off-by: cqy <yaya645@126.com>

* clang-tidy check

Signed-off-by: cqy <yaya645@126.com>

* clang-tidy check

Signed-off-by: cqy <yaya645@126.com>

* clang-tidy check

Signed-off-by: cqy <yaya645@126.com>

* clang-tidy check

Signed-off-by: cqy <yaya645@126.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip ci] add constants.java (#3404)

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Signed-off-by: groot <yihua.mo@zilliz.com>

* fix dsl term with multi fields (#3403)

Signed-off-by: zongyufen <zongyufen@foxmail.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip ci] comment unused insert case (#3405)

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* enable clang-tidy check (#3396)

* enable clang-tidy check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update run_clang_tidy.py

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable clang-tidy check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update run_clang_tidy.py

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update run_clang_tidy.py

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update run_clang_tidy.py

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove rule modernize-use-equals-default

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* fix the bugs of delete all and compact (#3395)

* fix the bugs of delete all and compact

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* fix the wrong usages in unittest

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* fix the bug of insert makes no effect

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* add character check in ExtraFileInfo and change the const size and type

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* fix wrong test case

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Fix TestSearchDSL multi fields bug (#3411)

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Optimize fiu and finish thirdparty optimization (#3412)

* fiu runable

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* runable riu

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* optimize fiu and rm ThirdPartyPackages.cmake

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix bug

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix test using fiu

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip ci] Java sdk test: add testIndex.java  (#3409)

* [skip ci] add constants.java

Signed-off-by: zw <zw@milvus.io>

* [skip ci] add testIndex.java

Signed-off-by: zw <zw@milvus.io>

* [skip ci] add TestDeleteEntities.java

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip ci] Test get entity by id (#3417)

* [skip ci] update for Contasts

Signed-off-by: zongyufen <zongyufen@foxmail.com>

* [skip ci] test get entity by id

Signed-off-by: zongyufen <zongyufen@foxmail.com>

* [skip ci] test get entity by id

Signed-off-by: zongyufen <zongyufen@foxmail.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* update clang-tidy rules (#3416)

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest failed

Signed-off-by: groot <yihua.mo@zilliz.com>

* remove the _id fileld when get collection info (#3414)

* fix the bug of issue #3336

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* fix wrong test case

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* rewrite insert machinery

Signed-off-by: groot <yihua.mo@zilliz.com>

* insert fields validation

Signed-off-by: groot <yihua.mo@zilliz.com>

* code format

Signed-off-by: groot <yihua.mo@zilliz.com>

* Milvus build stage parallel processing (#3423)

* Milvus build stage parallel processing

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Milvus build stage parallel processing

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Milvus build stage parallel processing

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Milvus build stage parallel processing

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Milvus build stage parallel processing

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Milvus build stage parallel processing

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* avoid build hang

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix faiss cannot build gpu bug (#3424)

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* wal path

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip ci] add TestSearchEntities.java  (#3433)

* [skip ci] add constants.java

Signed-off-by: zw <zw@milvus.io>

* [skip ci] add testIndex.java

Signed-off-by: zw <zw@milvus.io>

* [skip ci] add TestDeleteEntities.java

Signed-off-by: zw <zw@milvus.io>

* update TestSearchEntities.java

Signed-off-by: zw <zw@milvus.io>

* [skip ci] add TestSearchEntities.java

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Signed-off-by: groot <yihua.mo@zilliz.com>

* #3265 fix memory leak (#3413)

* #3265 fix memory leak

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>

* fix clang-format

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>

* update

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>

* comment some change

Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Fix webserver set_config bug (#3425)

* Fix TestSearchDSL multi fields bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix set_config bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Update mergify

Signed-off-by: jinhai <hai.jin@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Fix Server get stuck if create index with invalide metric types after entities inserted (#3428)

* add create index with invalid metric type case

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix bin index validation

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* change changelog

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Add partition_tag in GetPageEntities (#3434)

* Fix TestSearchDSL multi fields bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix set_config bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add partition_tag in GetPageEntities

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Fix Server crashed during random test (#3436)

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* Optimize clang-tidy workflow for code static analysis (#3432)

* Optimize clang-tidy for code static analysis

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Optimize clang-tidy for code static analysis

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip ci] add assert by collection stats (#3437)

Signed-off-by: zongyufen <zongyufen@foxmail.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* fix get entity by id bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix wal bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip ci] upgrade master version to v0.11.0 (fix #3449) (#3450)

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* fix a bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix wal bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix wal test bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix the data type of crc32c (#3455)

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
Signed-off-by: groot <yihua.mo@zilliz.com>

* fix wal path bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* Update version (#3454)

* [skip ci] add constants.java

Signed-off-by: zw <zw@milvus.io>

* [skip ci] update server version to 0.11.0 in test cases

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Signed-off-by: groot <yihua.mo@zilliz.com>

* fix: shards/requirements.txt to reduce vulnerabilities (#3457)

The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-SQLALCHEMY-590109

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix test failure

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com>
Co-authored-by: ThreadDao <zongyufen@foxmail.com>
Co-authored-by: XuanYang-cn <51370125+XuanYang-cn@users.noreply.github.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com>
Co-authored-by: zw <zw@milvus.io>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: BossZou <40255591+BossZou@users.noreply.github.com>
Co-authored-by: yukun <kun.yu@zilliz.com>
Co-authored-by: FluorineDog <fluorinedog@gmail.com>
Co-authored-by: cqy123456 <39671710+cqy123456@users.noreply.github.com>
Co-authored-by: chen qingxiang <67679556+godchen0212@users.noreply.github.com>
Co-authored-by: jinhai <hai.jin@zilliz.com>
Co-authored-by: Snyk bot <snyk-bot@snyk.io>
 2020-08-27 09:43:01 +08:00
BossZou
d1c4ab9136 Mishards 010 upgrade & Add config unit in demo config (#2550) 
* Update mishards config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Update mishards config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Upgrade mishards for milvus v0.10.0 (fix #2548)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Launch server fail using demo config (#2549)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Remove surplus config yaml

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-06-15 14:00:47 +08:00
BossZou
2decf951e2 read_only node detect delete vector operation using mishards #2368 (#2473) 
* Add new request ReLoadSegments

Signed-off-by: yhz <413554850@qq.com>

* Finish load segments functionality

Signed-off-by: yhz <413554850@qq.com>

* Add api in grpc

Signed-off-by: yhz <413554850@qq.com>

* update Reloadsegments

Signed-off-by: yhz <413554850@qq.com>

* .

Signed-off-by: yhz <413554850@qq.com>

* create new blacklist if not exists

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update api names

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Finish mishard for support api reloadsegments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Add more details when failed in search task

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Fix compile issue

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update mishards requirements

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Code format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* modify docker images in mysql all_in_one

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update shards code

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Move updatedeldocs function to dbimpl

Signed-off-by: yhz <413554850@qq.com>

* Move reload segment del docs function to dbimpl

Signed-off-by: yhz <413554850@qq.com>

* [skip ci] correct shards requirements

Signed-off-by: yhz <413554850@qq.com>
 2020-06-13 14:10:07 +08:00
Snyk bot
b39c5e7016
fix: shards/requirements.txt to reduce vulnerabilities (#2536) 
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-RSA-570831
 2020-06-12 11:14:24 +08:00
BossZou
38de8c92be Mishards 091 (#2462) 
* Upgrade mishards for milvus 0.9.1 (#2461)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci]add allowed server version 0.9.0

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-06-11 17:04:00 +08:00
BossZou
c8b3dbfc7c [skip ci] Upgrade mishards for v0.9.0 (#2358) (#2359) 
Signed-off-by: yhz <413554850@qq.com>
 2020-05-15 20:55:34 +08:00
BossZou
71145f73b3 [skip ci] Mishards 0.9.0 update (#2317) 
* Update docker images and sdk invoke for preparing version 0.9.0 (#2311)

Signed-off-by: yhz <413554850@qq.com>

* [skip ci]Update mishards methods

Signed-off-by: yhz <413554850@qq.com>
 2020-05-15 20:43:50 +08:00
BossZou
15a603fa46
[skip ci] Upgrade mishrads configuration to version 0.4 (#2301) 
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-05-11 21:03:17 +08:00
BossZou
1016dbae1a
[skip ci] update shards requirements (#2262) 
Signed-off-by: yhz <413554850@qq.com>
 2020-05-08 19:14:33 +08:00
BossZou
8a68d34919
Mishards upgrade (#2260) 
* shards ci

Signed-off-by: yhz <413554850@qq.com>

* update interface

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update mishards

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update changlog

Signed-off-by: yhz <413554850@qq.com>

* [skip ci] mishards dev test pass (fix #2252)

Signed-off-by: yhz <413554850@qq.com>
 2020-05-08 16:40:30 +08:00
Snyk bot
22dd03f8a1
fix: shards/requirements.txt to reduce vulnerabilities (#2222) 
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-PYLINT-568073
 2020-05-06 21:44:13 +08:00
peng.xu
7dfdcbb278
[skip ci](shards): print more server info for debug 
Signed-off-by: peng.xu <peng.xu@zilliz.com>
 2020-04-30 23:43:45 +08:00
peng.xu
29a41ef645
[skip ci](shards): handle connection empty cases 
Signed-off-by: peng.xu <peng.xu@zilliz.com>
 2020-04-30 23:43:42 +08:00
SkyYang
c754182a6f
[skip ci](shards): fix issue #2116 
Signed-off-by: SkyYang <mooncity.sky@163.com>
 2020-04-25 23:03:13 +08:00
SkyYang
29a4d58432
[skip ci](shards): fix issue to show exception messages in error log (#2115) 
Signed-off-by: SkyYang <mooncity.sky@163.com>
 2020-04-25 20:40:30 +08:00
xiaocai2333
e5dab2a43c
Remove unused ConnectionMgr (#2102) 
Signed-off-by: xiaocai2333 <46207236+xiaocai2333@users.noreply.github.com>
 2020-04-25 15:50:38 +08:00
FluorineDog
d3daa0d3ea
Remove import TopKQueryResult (#2100) 
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2020-04-25 15:47:31 +08:00
ashyshyshyman
3c53ee2700
Remove unused utilities (#2104) 
Signed-off-by: ashyshyshyman <1204034196@qq.com>
 2020-04-25 15:35:43 +08:00
zhenshan.cao
8a63d27612
Remove unused settings (#2095) 
Signed-off-by: czs007 <czs@alumni.hust.edu.cn>
 2020-04-25 15:31:44 +08:00
weishuo
c4290271ca
Remove unused defaultdict in shards/mishards/service_handler.py. (#2092) 
Signed-off-by: weishuo <2411602273@qq.com>
 2020-04-25 15:30:13 +08:00
Murpphy
87fc760416
Remove unsed datetime (#2094) 
Signed-off-by: Murpphy <31717785+Murpphy@users.noreply.github.com>
 2020-04-25 15:27:38 +08:00
SwaggySong
d8b1b54bd6
Remove unused enum (#2089) 
Signed-off-by: SwaggySong 36157116+SwaggySong@users.noreply.github.com
 2020-04-25 14:50:47 +08:00
Cupchen
889681f63a
Remove whole class ConnectionMgr (#2048) 
* Update __init__.py: Remove unused ConnectionMgr

Signed-off-by: Cupchen <class_cyl@163.com>

* Remove unused ConnectionMgr

Signed-off-by: Cupchen <class_cyl@163.com>

* Remove unused ConnectionMgr

Signed-off-by: Cupchen <class_cyl@163.com>

* Remove whole class ConnectionMgr 

Signed-off-by: Cupchen <class_cyl@163.com>
 2020-04-23 11:11:21 +08:00
Cupchen
26478a1bbb
Update __init__.py: Remove unused ConnectionMgr (#2045) 
Signed-off-by: Cupchen <class_cyl@163.com>
 2020-04-23 10:30:57 +08:00
XuPeng-SH
753c2ff906
Mishards k8s fix (#1958) 
* [skip ci](shards): update kubernetes related

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): add pending buff in kubernetes provider

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): some bug fix in k8s provider

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): update log

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): update k8s config

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): update Makefile

Signed-off-by: peng.xu <peng.xu@zilliz.com>
 2020-04-16 22:04:47 +08:00
Jin Hai
0323aa1aad
Merge 080 (#1940) 
* #1910 C++ SDK GetIDsInSegment could not work for large dataset (#1911)

Signed-off-by: groot <yihua.mo@zilliz.com>

* #1903 Fix invalid annoy result (#1912)

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* #1914: Partition max size should be 4096 (#1915)

Signed-off-by: jinhai <hai.jin@zilliz.com>

* add log (#1913)

* add log

Signed-off-by: groot <yihua.mo@zilliz.com>

* add log

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: groot <yihua.mo@zilliz.com>

* partition limit 4096

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix py test

Signed-off-by: groot <yihua.mo@zilliz.com>

* update server version (#1916)

Signed-off-by: zw <zw@zilliz.com>

* Update to 0.8.0 (#1918)

* Create new branch 0.8.0 and change preload_table to preload_collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix format

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* update helm version

Signed-off-by: zw <zw@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>

* fix issue 1901 (#1920)

* fix issue 1901

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>

* #1900 (#1923)

* add log

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix #1900

Signed-off-by: groot <yihua.mo@zilliz.com>

* Upgrade mishards to 0.8.0 (#1933)

* update grpc server of milvus & rename table name to collection

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Skip CI

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Caiyd 1883 fix rw (#1926)

* #1883 use DiskIO

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix logic error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>

* #1928 Too many data and uid copies when loading files (#1931)

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>

* Update mishards configure files (#1938)

* Update web readme

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] update configure files

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] rename table to collection

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Update test.groovy

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update test.groovy

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: jinhai <hai.jin@zilliz.com>

Co-authored-by: groot <yhmo@zeronedata.com>
Co-authored-by: shengjun.li <49774184+shengjun1985@users.noreply.github.com>
Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com>
Co-authored-by: zw <zw@zilliz.com>
Co-authored-by: op-hunter <ophunter52@gmail.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: BossZou <40255591+BossZou@users.noreply.github.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
 2020-04-15 21:32:20 +08:00
Jin Hai
3e03850169
Merge from 0.7.1 to master (#1840) 
* update mishards images to 0.7.1 (fix #1799) (#1802)

* update mishards images to 0.7.1 (fix #1799)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Skip CI

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* fix lsn meta bug (#1813)

* fix lsn meta bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix mysql lsn meta bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* Update CHANGELOG.md (#1814)

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compile (#1815)

* Update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compile (#1816)

* Update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compile (#1817)

* Update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Increase Resources

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Decrease the compiling thread

Signed-off-by: jinhai <hai.jin@zilliz.com>

* #1818 Duplicate data generated after restart milvus server (#1819)

* #1818 Duplicate data generated after restart milvus server

Signed-off-by: groot <yihua.mo@zilliz.com>

* set compile thread number

Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip-ci]Remove auto flush interval from config (#1837)

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compile error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: BossZou <40255591+BossZou@users.noreply.github.com>
Co-authored-by: groot <yhmo@zeronedata.com>
 2020-04-02 14:20:33 +08:00
XuPeng-SH
a459a1efdd
[skip ci](shards): support server 0.7.1 (#1811) 
Signed-off-by: peng.xu <peng.xu@zilliz.com>
 2020-03-31 18:52:44 +08:00
BossZou
5a83ba4e49
Optimize behavior to get file ids from metadata in mishards (#1754) (#1755) 
* update mishards get file ids

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update behaviour for getting file ids

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* check default partition

Signed-off-by: yhz <413554850@qq.com>

* remove exception when file id not found

Signed-off-by: yhz <413554850@qq.com>

* Optimize behavior to get file ids from metadata in mishards (fix #1754)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-03-26 12:28:20 +08:00
BossZou
3c3617fd81
Mishards 070 (#1699) 
* update mishards to 0.7.0

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update mishards to 0.7.0

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* fix search bug and example pass

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add new api

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add segment stat

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* fix table info issue

Signed-off-by: yhz <413554850@qq.com>

* fix mishards api issue

Signed-off-by: yhz <413554850@qq.com>

* update mishards

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update all_in_one docker images

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* fix delete_by_id param parser & remove some comments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update yaml config file

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update all_in_one config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* remove delete_by_range comments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update cmd api

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add warning when search

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update service_handler

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update shrads requiremtns

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] remove surplus log info

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-03-20 20:21:11 +08:00
XuPeng-SH
b520e60b8a
[skip ci](shards): fix #1679 (#1681) 
Signed-off-by: peng.xu <peng.xu@zilliz.com>
 2020-03-17 11:06:51 +08:00
Snyk bot
bb6ce0abe1
fix: shards/requirements.txt to reduce vulnerabilities (#1680) 
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-URLLIB3-559452
 2020-03-17 10:30:31 +08:00
XuPeng-SH
4173626088
[skip ci](shards): fix low concurrency issue (#1672) 
Signed-off-by: peng.xu <peng.xu@zilliz.com>
 2020-03-17 09:57:36 +08:00
XuPeng-SH
5f2f8bdc8b
[skip ci] (shards) Upgrade Mishards for #1569 (#1570) 
* [skip ci](shards): export MAX_WORKERS as configurable parameter

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): skip mishards .env git info

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): support more robust static discovery host configuration

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): update static provider that terminate server if connection to downstream server error during startup

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): add topology.py

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): add connection pool

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): add topology test

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): refactory using topo

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): refactory static discovery using topo

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): refactory kubernetes discovery using topo

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): add more test for connection pool

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): export 19541 and 19542 for all_in_one demo

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): check version on new connection

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): mock connections

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* [skip ci](shards): update tests

Signed-off-by: peng.xu <peng.xu@zilliz.com>
 2020-03-14 13:30:21 +08:00
snyk-bot
bdb87db380 fix: shards/requirements.txt to reduce vulnerabilities 
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-URLLIB3-559452
 2020-03-07 02:46:33 +00:00
quicksilver
5ec1baaede run hadolint with reviewdog (#1060) 
* run hadolint with reviewdog

* add LINCENSE in Dockerfile

* run hadolint with reviewdog

* Reporter of reviewdog command is "github-pr-check"

* format Dockerfile

* ignore DL3007 in hadolint
 2020-01-16 19:59:26 +08:00
milvus
54435fc03b [skip ci] fix broken links (#985) 
Co-authored-by: JackLCL <53512883+JackLCL@users.noreply.github.com>
 2020-01-10 19:23:30 +08:00
XuPeng-SH
0a134dab82 Mishards synced to 0.6.0 release (#725) 
* [skip ci](shards): update to support milvus 0.6.0

* [skip ci](shards): add show partitions

* [skip ci](shards): add drop partition

* [skip ci](shards): add create partition

* [skip ci](shards): update search for partition

* [skip ci](shards): fix search partion bugs

* [skip ci](shards): update all in one demo

* [skip ci](shards/doc): update image in readme
 2019-12-09 10:55:32 +08:00
peng.xu
d98e673fdc [skip ci](shards): remove color decorator in log file, fixes #573 2019-11-27 14:53:24 +08:00
peng.xu
dc96690a71 [skip ci](shards): format log, fixes #568 2019-11-27 12:57:25 +08:00
peng.xu
b734491698 [skip ci](shards): fix search result bug, fixes #564 2019-11-27 10:58:34 +08:00
yhz
2d5b908c05 fix changlog conflict 2019-11-26 18:33:50 +08:00
yhz
55b4acc27b [skip ci]afix shards bug when serch result is empty 2019-11-26 14:40:02 +08:00
peng.xu
10d1f21d59 [skip ci](shards): fix db create engine typo and exception errors 2019-11-26 14:29:20 +08:00
peng.xu
06394c39fd [skip ci](shards): update makefile 2019-11-26 13:43:28 +08:00
peng.xu
646802e53f [skip ci](shards): update git ignore and docker ignore 2019-11-26 13:36:27 +08:00
peng.xu
e2e6cb11b3 [skip ci](shards): update Makefile for all in one with mysql 2019-11-26 13:36:00 +08:00
peng.xu
8b00b11fe5 [skip ci](shards): fix bug in search 2019-11-26 13:35:02 +08:00
peng.xu
2cce773589 [skip ci](shards): add all_in_one_with_mysql 2019-11-26 13:34:32 +08:00
peng.xu
ba50927d6e [skip ci](shards): bug fix for mysql startup 2019-11-26 11:31:50 +08:00