* refactor LogMgr (#3372) Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * add wal unittest (#3367) * add wal unittest Signed-off-by: groot <yihua.mo@zilliz.com> * fix test fail Signed-off-by: groot <yihua.mo@zilliz.com> * Find system headers with cmake and clang-tidy (#3364) Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * [skip ci]add google/crc32c into NOTICE.md (#3373) Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * prepare change memmanager for wal Signed-off-by: groot <yihua.mo@zilliz.com> * [skip ci]update issue template (#3379) Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * fix wal test case Signed-off-by: groot <yihua.mo@zilliz.com> * [skip ci] fix java sdk test case (#3375) * java main class Signed-off-by: zongyufen <zongyufen@foxmail.com> * [skip ci] fix java sdk test Signed-off-by: zongyufen <zongyufen@foxmail.com> * [skip ci] fix java sdk test Signed-off-by: zongyufen <zongyufen@foxmail.com> Signed-off-by: groot <yihua.mo@zilliz.com> * optimize download & compile of sqlite (#3361) * optimize sqlite Signed-off-by: yangxuan <xuan.yang@zilliz.com> * rm sqlite url Signed-off-by: yangxuan <xuan.yang@zilliz.com> * fix bug Signed-off-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * [skip ci] rm log (#3378) Signed-off-by: shengjun.li <shengjun.li@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * [skip ci] fix generate default entities (#3382) * java main class Signed-off-by: zongyufen <zongyufen@foxmail.com> * [skip ci] fix java sdk test Signed-off-by: zongyufen <zongyufen@foxmail.com> * [skip ci] fix java sdk test Signed-off-by: zongyufen <zongyufen@foxmail.com> * [skip ci] fix generate default entities Signed-off-by: zongyufen <zongyufen@foxmail.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Skip flat search params (#3381) * assert top ids Signed-off-by: zw <zw@milvus.io> * update milvus-helm to 0.11.0 Signed-off-by: zw <zw@milvus.io> Co-authored-by: zw <zw@milvus.io> Signed-off-by: groot <yihua.mo@zilliz.com> * [skip ci] update clang-tidy rules (#3386) * [skip ci] update clang-tidy rules Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * [skip ci] update clang-tidy rules Signed-off-by: yudong.cai <yudong.cai@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Optimize download & compile of Aws (#3384) * runable aws Signed-off-by: yangxuan <xuan.yang@zilliz.com> * optimize aws Signed-off-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * [skip ci] modify test flush and compact (#3390) * java main class Signed-off-by: zongyufen <zongyufen@foxmail.com> * [skip ci] fix java sdk test Signed-off-by: zongyufen <zongyufen@foxmail.com> * [skip ci] fix java sdk test Signed-off-by: zongyufen <zongyufen@foxmail.com> * [skip ci] fix generate default entities Signed-off-by: zongyufen <zongyufen@foxmail.com> * [skip ci] modify test flush Signed-off-by: zongyufen <zongyufen@foxmail.com> * [skip ci] modify test compact Signed-off-by: zongyufen <zongyufen@foxmail.com> Signed-off-by: groot <yihua.mo@zilliz.com> * rewrite insert memmanager Signed-off-by: groot <yihua.mo@zilliz.com> * Optimize thirdparty download workflow (#3394) * Optimize thirdparty download workflow Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * Optimize thirdparty download workflow Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * Optimize thirdparty download workflow Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * Optimize thirdparty download workflow Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Enlarge event queue (#3393) Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Add GetPageEntity unittest (#3397) * Add web server interface Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Add unittest/server Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Add web server ut Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Fix web server insert bug Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Fix web server ut crash bug Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Fix web server ut gpu compile error Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Fix codacy quality Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Fix row_num error Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Change row insert to column insert Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Fix dsl issue Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Fix GetEntityByID bug Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Add GetPageEntity interface Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Fix some webserver bugs Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Remove server_config.yaml Signed-off-by: fishpenguin <kun.yu@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Fix thirdparty not find ccache but still use it (#3398) * fix sqlite ccache Signed-off-by: yangxuan <xuan.yang@zilliz.com> * thirdparty EP using ccache configure Signed-off-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * optimize oatpp (#3377) * optimize oatpp Signed-off-by: yangxuan <xuan.yang@zilliz.com> * fix some bug Signed-off-by: yangxuan <xuan.yang@zilliz.com> * fit atomic not find bug Signed-off-by: yangxuan <xuan.yang@zilliz.com> * add one config fo oatpp Signed-off-by: yangxuan <xuan.yang@zilliz.com> * change oatpp version Signed-off-by: yangxuan <xuan.yang@zilliz.com> * comment url_md5 Signed-off-by: yangxuan <xuan.yang@zilliz.com> * change oatpp version Signed-off-by: yangxuan <xuan.yang@zilliz.com> * change aws target name Signed-off-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Split Ftype into Ftype and FEtype (#3341) * fix include directories not find bug (#3323) Signed-off-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> Signed-off-by: fluorinedog <fluorinedog@gmail.com> * reduce grpc download file size from 380M to 130M (#3326) * reduce grpc download file size from 380M to 130M Signed-off-by: yangxuan <xuan.yang@zilliz.com> * fix bug Signed-off-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> Signed-off-by: fluorinedog <fluorinedog@gmail.com> * update clang-tidy config (#3314) * update clang-tidy config Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update .clang-tidy Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update clang-tidy config Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update clang-tidy HeaderFilterRegex Signed-off-by: yudong.cai <yudong.cai@zilliz.com> Signed-off-by: fluorinedog <fluorinedog@gmail.com> * enable fetype Signed-off-by: fluorinedog <fluorinedog@gmail.com> * rename field_element_method Signed-off-by: fluorinedog <fluorinedog@gmail.com> * add FETYPE_TYPE Signed-off-by: fluorinedog <fluorinedog@gmail.com> * Add WebServer unittest (#3321) * Add web server interface Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Add unittest/server Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Add web server ut Signed-off-by: fishpenguin <kun.yu@zilliz.com> Signed-off-by: fluorinedog <fluorinedog@gmail.com> * modify db schema and remove conversion Signed-off-by: fluorinedog <fluorinedog@gmail.com> * weak fetype Signed-off-by: fluorinedog <fluorinedog@gmail.com> * enforce strong fetype Signed-off-by: fluorinedog <fluorinedog@gmail.com> * lint Signed-off-by: fluorinedog <fluorinedog@gmail.com> * format code Signed-off-by: fluorinedog <fluorinedog@gmail.com> * [skip ci] fix Signed-off-by: fluorinedog <fluorinedog@gmail.com> Co-authored-by: XuanYang-cn <51370125+XuanYang-cn@users.noreply.github.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: Cai Yudong <yudong.cai@zilliz.com> Co-authored-by: yukun <kun.yu@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Master codacy check (#3366) * codacy fix Signed-off-by: cqy <yaya645@126.com> * codacy Signed-off-by: cqy <yaya645@126.com> * codacy Signed-off-by: cqy <yaya645@126.com> * codacy check Signed-off-by: cqy <yaya645@126.com> * codacy Signed-off-by: cqy <yaya645@126.com> * codacy Signed-off-by: cqy <yaya645@126.com> * codacy Signed-off-by: cqy <yaya645@126.com> * clang-tiny Signed-off-by: cqy <yaya645@126.com> * clang-tidy check Signed-off-by: cqy <yaya645@126.com> * clang-tindy check Signed-off-by: cqy <yaya645@126.com> * clang-tidy check Signed-off-by: cqy <yaya645@126.com> * clang-tidy Signed-off-by: cqy <yaya645@126.com> * clang-tidy check Signed-off-by: cqy <yaya645@126.com> * clang-tidy check Signed-off-by: cqy <yaya645@126.com> * clang-tidy check Signed-off-by: cqy <yaya645@126.com> * clang-tidy check Signed-off-by: cqy <yaya645@126.com> Co-authored-by: Cai Yudong <yudong.cai@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * [skip ci] add constants.java (#3404) Signed-off-by: zw <zw@milvus.io> Co-authored-by: zw <zw@milvus.io> Signed-off-by: groot <yihua.mo@zilliz.com> * fix dsl term with multi fields (#3403) Signed-off-by: zongyufen <zongyufen@foxmail.com> Signed-off-by: groot <yihua.mo@zilliz.com> * [skip ci] comment unused insert case (#3405) Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * enable clang-tidy check (#3396) * enable clang-tidy check Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update run_clang_tidy.py Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * enable clang-tidy check Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update run_clang_tidy.py Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update run_clang_tidy.py Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update run_clang_tidy.py Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * remove rule modernize-use-equals-default Signed-off-by: yudong.cai <yudong.cai@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * fix the bugs of delete all and compact (#3395) * fix the bugs of delete all and compact Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com> * fix the wrong usages in unittest Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com> * fix the bug of insert makes no effect Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com> * add character check in ExtraFileInfo and change the const size and type Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com> * format code Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com> * fix wrong test case Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Fix TestSearchDSL multi fields bug (#3411) Signed-off-by: fishpenguin <kun.yu@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Optimize fiu and finish thirdparty optimization (#3412) * fiu runable Signed-off-by: yangxuan <xuan.yang@zilliz.com> * runable riu Signed-off-by: yangxuan <xuan.yang@zilliz.com> * optimize fiu and rm ThirdPartyPackages.cmake Signed-off-by: yangxuan <xuan.yang@zilliz.com> * fix bug Signed-off-by: yangxuan <xuan.yang@zilliz.com> * fix test using fiu Signed-off-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * [skip ci] Java sdk test: add testIndex.java (#3409) * [skip ci] add constants.java Signed-off-by: zw <zw@milvus.io> * [skip ci] add testIndex.java Signed-off-by: zw <zw@milvus.io> * [skip ci] add TestDeleteEntities.java Signed-off-by: zw <zw@milvus.io> Co-authored-by: zw <zw@milvus.io> Signed-off-by: groot <yihua.mo@zilliz.com> * [skip ci] Test get entity by id (#3417) * [skip ci] update for Contasts Signed-off-by: zongyufen <zongyufen@foxmail.com> * [skip ci] test get entity by id Signed-off-by: zongyufen <zongyufen@foxmail.com> * [skip ci] test get entity by id Signed-off-by: zongyufen <zongyufen@foxmail.com> Signed-off-by: groot <yihua.mo@zilliz.com> * update clang-tidy rules (#3416) Signed-off-by: yudong.cai <yudong.cai@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * fix unittest failed Signed-off-by: groot <yihua.mo@zilliz.com> * remove the _id fileld when get collection info (#3414) * fix the bug of issue #3336 Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com> * fix wrong test case Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * rewrite insert machinery Signed-off-by: groot <yihua.mo@zilliz.com> * insert fields validation Signed-off-by: groot <yihua.mo@zilliz.com> * code format Signed-off-by: groot <yihua.mo@zilliz.com> * Milvus build stage parallel processing (#3423) * Milvus build stage parallel processing Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * Milvus build stage parallel processing Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * Milvus build stage parallel processing Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * Milvus build stage parallel processing Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * Milvus build stage parallel processing Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * Milvus build stage parallel processing Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * avoid build hang Signed-off-by: groot <yihua.mo@zilliz.com> * fix faiss cannot build gpu bug (#3424) Signed-off-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * wal path Signed-off-by: groot <yihua.mo@zilliz.com> * typo Signed-off-by: groot <yihua.mo@zilliz.com> * [skip ci] add TestSearchEntities.java (#3433) * [skip ci] add constants.java Signed-off-by: zw <zw@milvus.io> * [skip ci] add testIndex.java Signed-off-by: zw <zw@milvus.io> * [skip ci] add TestDeleteEntities.java Signed-off-by: zw <zw@milvus.io> * update TestSearchEntities.java Signed-off-by: zw <zw@milvus.io> * [skip ci] add TestSearchEntities.java Signed-off-by: zw <zw@milvus.io> Co-authored-by: zw <zw@milvus.io> Signed-off-by: groot <yihua.mo@zilliz.com> * #3265 fix memory leak (#3413) * #3265 fix memory leak Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com> * fix clang-format Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com> * update Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com> * comment some change Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Fix webserver set_config bug (#3425) * Fix TestSearchDSL multi fields bug Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Fix set_config bug Signed-off-by: fishpenguin <kun.yu@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Update mergify Signed-off-by: jinhai <hai.jin@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Fix Server get stuck if create index with invalide metric types after entities inserted (#3428) * add create index with invalid metric type case Signed-off-by: yangxuan <xuan.yang@zilliz.com> * fix bin index validation Signed-off-by: yangxuan <xuan.yang@zilliz.com> * change changelog Signed-off-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Add partition_tag in GetPageEntities (#3434) * Fix TestSearchDSL multi fields bug Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Fix set_config bug Signed-off-by: fishpenguin <kun.yu@zilliz.com> * Add partition_tag in GetPageEntities Signed-off-by: fishpenguin <kun.yu@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Fix Server crashed during random test (#3436) Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * Optimize clang-tidy workflow for code static analysis (#3432) * Optimize clang-tidy for code static analysis Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> * Optimize clang-tidy for code static analysis Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * [skip ci] add assert by collection stats (#3437) Signed-off-by: zongyufen <zongyufen@foxmail.com> Signed-off-by: groot <yihua.mo@zilliz.com> * fix get entity by id bug Signed-off-by: groot <yihua.mo@zilliz.com> * typo Signed-off-by: groot <yihua.mo@zilliz.com> * fix wal bug Signed-off-by: groot <yihua.mo@zilliz.com> * [skip ci] upgrade master version to v0.11.0 (fix #3449) (#3450) Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * fix a bug Signed-off-by: groot <yihua.mo@zilliz.com> * fix wal bug Signed-off-by: groot <yihua.mo@zilliz.com> * fix wal test bug Signed-off-by: groot <yihua.mo@zilliz.com> * fix the data type of crc32c (#3455) Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com> Signed-off-by: groot <yihua.mo@zilliz.com> * fix wal path bug Signed-off-by: groot <yihua.mo@zilliz.com> * Update version (#3454) * [skip ci] add constants.java Signed-off-by: zw <zw@milvus.io> * [skip ci] update server version to 0.11.0 in test cases Signed-off-by: zw <zw@milvus.io> Co-authored-by: zw <zw@milvus.io> Signed-off-by: groot <yihua.mo@zilliz.com> * fix: shards/requirements.txt to reduce vulnerabilities (#3457) The following vulnerabilities are fixed by pinning transitive dependencies: - https://snyk.io/vuln/SNYK-PYTHON-SQLALCHEMY-590109 Signed-off-by: groot <yihua.mo@zilliz.com> * fix test failure Signed-off-by: groot <yihua.mo@zilliz.com> * typo Signed-off-by: groot <yihua.mo@zilliz.com> Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com> Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com> Co-authored-by: ThreadDao <zongyufen@foxmail.com> Co-authored-by: XuanYang-cn <51370125+XuanYang-cn@users.noreply.github.com> Co-authored-by: yangxuan <xuan.yang@zilliz.com> Co-authored-by: shengjun.li <shengjun.li@zilliz.com> Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com> Co-authored-by: zw <zw@milvus.io> Co-authored-by: Cai Yudong <yudong.cai@zilliz.com> Co-authored-by: BossZou <40255591+BossZou@users.noreply.github.com> Co-authored-by: yukun <kun.yu@zilliz.com> Co-authored-by: FluorineDog <fluorinedog@gmail.com> Co-authored-by: cqy123456 <39671710+cqy123456@users.noreply.github.com> Co-authored-by: chen qingxiang <67679556+godchen0212@users.noreply.github.com> Co-authored-by: jinhai <hai.jin@zilliz.com> Co-authored-by: Snyk bot <snyk-bot@snyk.io>
What is Milvus
As an open source vector similarity search engine, Milvus is easy-to-use, highly reliable, scalable, robust, and blazing fast. Adopted by over 100 organizations and institutions worldwide, Milvus empowers applications in a variety of fields, including image processing, computer vision, natural language processing, voice recognition, recommender systems, drug discovery, and more.
The following is Milvus architecture:
For more detailed introduction of Milvus and its architecture, see Milvus overview. See Milvus release notes to keep up-to-date with its releases and updates.
Milvus is an LF AI Foundation incubation project.
Get started
Install Milvus
See Milvus install guide to install Milvus using Docker. To install Milvus from source code, see build from source.
Try example programs
Try an example program with Milvus using Python, Java, Go, or C++ example code.
Supported clients
Application scenarios
You can use Milvus to build intelligent systems in a variety of AI application scenarios. See Milvus Scenarios for live demos. You can also see Milvus Bootcamp for detailed solutions and application scenarios.
Benchmark
See our test reports for more information about performance benchmarking of different indexes in Milvus.
Roadmap
To learn what's coming up soon in Milvus, read our Roadmap.
It is a Work in Progress, and is subject to reasonable adjustments when necessary. And we greatly appreciate any comments/requirements/suggestions regarding Milvus' roadmap.👏
Contribution guidelines
Contributions are welcomed and greatly appreciated. Please read our contribution guidelines for detailed contribution workflow. This project adheres to the code of conduct of Milvus. You must abide by this code in order to participate.
We use GitHub issues to track issues and bugs. For general questions and public discussions, please join our community.
Join our community
❤️To connect with other users and contributors, welcome to join our Slack channel.
See our community repository to learn more about our governance and access more community resources.
