505 Commits

Author SHA1 Message Date
Zhen Ye
944f018375
enhance: [skip e2e] only use mixcoord for scripts (#40273)
issue: #38399

Signed-off-by: chyezh <chyezh@outlook.com>
2025-03-02 18:05:58 +08:00
Zhen Ye
bc8e02df44
enhance: always enable streaming service (#40253)
issue: #38399

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-28 15:38:01 +08:00
Linkwei
0a4e7b5116
enhance: [skip e2e] Add configuration file detection functionality to the standalone_embed script. (#40246)
enhance: Add configuration file detection functionality to the
standalone_embed script.
issue:https://github.com/milvus-io/milvus/issues/40066

Signed-off-by: Linkwei <link.xie@zilliz.com>
2025-02-28 11:13:57 +08:00
sthuang
f47320e0e7
enhance: clean up legacy storage v2 (#39987)
related: https://github.com/milvus-io/milvus/issues/39173

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-02-19 15:42:52 +08:00
SimFG
047254665d
feat: support to replicate import msg (#39171)
- issue: #39849

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: chyezh <chyezh@outlook.com>
Co-authored-by: chyezh <chyezh@outlook.com>
2025-02-16 00:08:13 +08:00
sthuang
c4ae9f4ece
feat: introduce third-party milvus-storage (#39418)
related: https://github.com/milvus-io/milvus/issues/39173

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-01-24 17:21:13 +08:00
sre-ci-robot
cd068f2c3a
[automated] Bump milvus version to v2.5.4 (#39548)
Bump milvus version to v2.5.4
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-01-24 10:29:15 +08:00
Bingyi Sun
cb959cd1f9
enhance: upgrade rust version to 1.83 (#39295)
#39254

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-01-20 11:15:03 +08:00
Ted Xu
9209a70bb6
fix: clang format broken under osx (#38427)
See: #38434

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2025-01-17 10:43:03 +08:00
sre-ci-robot
cc8e6ebd5e
[automated] Bump milvus version to v2.5.3 (#39215)
Bump milvus version to v2.5.3
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-01-13 18:37:03 +08:00
Zhen Ye
bb8d1ab3bf
enhance: make new go package to manage proto (#39114)
issue: #39095

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-10 10:49:01 +08:00
sre-ci-robot
e6bf141e0b
[automated] Bump milvus version to v2.5.2 (#38959)
Bump milvus version to v2.5.2
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-01-03 10:56:55 +08:00
sre-ci-robot
18a3bc7794
[automated] Bump milvus version to v2.5.1 (#38802)
Bump milvus version to v2.5.1
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-27 13:58:58 +08:00
sre-ci-robot
9e1bb3bd8f
[automated] Bump milvus version to v2.5.0 (#38640)
Bump milvus version to v2.5.0
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-23 11:48:54 +08:00
Linkwei
80c62f8c11
enhance: add windows startup script for milvus docker deployment (#38089)
[enhance:add window bat milvus standalone ](enhance: add windows startup
script for milvus docker deployment)
issue:https://github.com/milvus-io/milvus/issues/38098

Signed-off-by: zilliz <zilliz@zillizdeMacBook-Pro-6.local>
Co-authored-by: zilliz <zilliz@zillizdeMacBook-Pro-6.local>
2024-11-29 15:06:37 +08:00
sre-ci-robot
930112e55e
[automated] Bump milvus version to v2.5.0-beta (#37961)
Bump milvus version to v2.5.0-beta
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-11-23 21:48:32 +08:00
sre-ci-robot
7ba85504ba
[automated] Bump milvus version to v2.4.16 (#37795)
Bump milvus version to v2.4.16
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-11-20 11:46:37 +08:00
congqixia
f813fb4563
enhance: [GoSDK] Remove example in main pkg to make ut script work (#37472)
Related to #31293

Example with main func/package causes `go list` command failed with vcs
error. This PR removes example files with main package. The example
snippet shall be added back in runnable test format.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-07 10:50:25 +08:00
yellow-shine
b3de4b0edd
enhance: refine the pipeline (#37412)
1. enhance: allow to pass conan repo url
2. enhance: add throttle for cpu-e2e and cpp-unit-test pipeline

---------

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-11-06 10:24:30 +08:00
sre-ci-robot
b2c36efb5f
[automated] Bump milvus version to v2.4.15 (#37458)
Bump milvus version to v2.4.15
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-11-05 21:18:32 +08:00
yellow-shine
22745a4980
Bump milvus version to v2.4.14 (#37253)
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-10-29 20:58:23 +08:00
yellow-shine
8902e2220e
enhance: enable asan for cpp unittest (#37041)
https://github.com/milvus-io/milvus/issues/35854

Signed-off-by: chyezh <chyezh@outlook.com>
Co-authored-by: chyezh <chyezh@outlook.com>
2024-10-23 17:21:27 +08:00
sre-ci-robot
aee61973fb
[automated] Bump milvus version to v2.4.13-hotfix (#36947)
Bump milvus version to v2.4.13-hotfix
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-17 14:17:28 +08:00
Linkwei
c271c21f17
enhance: improve scripts standalone_embed,add upgrade (#36924)
[[skip ci] improve scripts standalone_embed,add upgrade](enhance:
improvement scripts standalone_embed,add upgrade #36921)

Signed-off-by: zilliz <zilliz@zillizdeMacBook-Pro.local>
Co-authored-by: zilliz <zilliz@zillizdeMacBook-Pro.local>
2024-10-16 17:51:23 +08:00
sre-ci-robot
d45e2677b9
[automated] Bump milvus version to v2.4.13 (#36801)
Bump milvus version to v2.4.13
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-12 15:07:24 +08:00
cai.zhang
d1060c0e05
enhance: Update antlr version and refine parsing not in (#36745)
issue: #36672

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-10-11 14:03:21 +08:00
Rijin-N
a05a37a583
enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214)
Native support for Google cloud storage using the Google Cloud Storage
libraries. Authentication is performed using GCS service account
credentials JSON.

Currently, Milvus supports Google Cloud Storage using S3-compatible APIs
via the AWS SDK. This approach has the following limitations:

1. Overhead: Translating requests between S3-compatible APIs and GCS can
introduce additional overhead.
2. Compatibility Limitations: Some features of the original S3 API may
not fully translate or work as expected with GCS.

To address these limitations, This enhancement is needed.

Related Issue: #36212
2024-09-30 13:23:32 +08:00
cai.zhang
2c9bb4dfa3
feat: Support stats task to sort segment by PK (#35054)
issue: #33744 

This PR includes the following changes:
1. Added a new task type to the task scheduler in datacoord: stats task,
which sorts segments by primary key.
2. Implemented segment sorting in indexnode.
3. Added a new field `FieldStatsLog` to SegmentInfo to store token index
information.

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-02 14:19:03 +08:00
zhenshan.cao
d10aa4626f
enhance: [skip e2e] add make run-test-cpp with support for filter gtest (#35829)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-08-29 20:03:02 +08:00
Zhen Ye
70bc0e4538
enhance: add skip 3rdparty env var (#35736)
issue: #35611

- call `conan install` is too slow for developer, add option to skip it.

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-29 14:21:01 +08:00
Zhen Ye
75da36d1aa
enhance: enable asan for milvus (#35627)
issue: #35626

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-23 21:06:58 +08:00
Zhen Ye
a773836b89
enhance: optimize milvus core building (#35610)
issue: #35549,#35611,#35633

- remove milvus_segcore milvus_indexbuilder..., add libmilvus_core
- core building only link once
- move opendal compilation into cmake
- fix odr

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-23 12:35:02 +08:00
yellow-shine
7420115b5e
enhance: pinpoint revision of 3thirdparty package (#35325)
issue: https://github.com/milvus-io/milvus/issues/35340

---------

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-08 09:46:22 +08:00
chyezh
c725416288
enhance: move streaming proto into pkg (#35284)
issue: #33285

- move streaming related proto into pkg.
- add v2 message type and change flush message into v2 message.

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-07 10:34:16 +08:00
chyezh
14051fed7d
enhance: streaming service client (#34656)
issue: #33285

- implement streaming service client.
- implement producing and consuming service client by streaming coord
client and streaming node client.

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-05 21:38:15 +08:00
yellow-shine
241c71fdde
enhance: use docker compose instead of docker-compose (#35208)
https://github.com/milvus-io/milvus/issues/35209

---------

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-02 19:32:32 +08:00
yellow-shine
7bf68d8657
enhance: upgrade conan version (#35215)
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-02 19:22:15 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394)
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-29 11:31:51 +08:00
Ted Xu
63f0154dfb
fix: enable milvus.yaml check (#34567)
See #32168

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-07-25 18:05:46 +08:00
chyezh
39c7e06bc5
enhance: add message and msgstream msgpack adaptor (#34874)
issue: #33285

- make message builder and message conversion type safe
- add adaptor type and function to adapt old msgstream msgpack and
message interface

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-22 20:59:42 +08:00
chyezh
cc8f7aa110
fix: streaming service related fix patch (#34696)
issue: #33285

- add idAlloc interface
- fix binary unsafe bug for message
- fix service discovery lost when repeated address with different server
id

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-16 15:49:38 +08:00
chyezh
fda720b880
enhance: streaming service grpc utilities (#34436)
issue: #33285

- add two grpc resolver (by session and by streaming coord assignment
service)
- add one grpc balancer (by serverID and roundrobin)
- add lazy conn to avoid block by first service discovery
- add some utility function for streaming service

Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-15 20:49:38 +08:00
pingliu
8c42f1341d
doc: [skip e2e] add extraConfig to embed milvus (#34395)
Signed-off-by: ping.liu <ping.liu@zilliz.com>
2024-07-03 18:20:17 +08:00
sammy.huang
b14482baa6
enhance: [skip e2e] use disk ann when os is ubuntu (#34236)
issue: https://github.com/milvus-io/milvus/issues/34222

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-06-27 20:50:13 +08:00
pingliu
b12c34a8ba
doc: [skip e2e] change milvus docker image version to v2.4.5 (#34135)
Signed-off-by: ping.liu <ping.liu@zilliz.com>
2024-06-25 14:30:04 +08:00
chyezh
b9237280c2
enhance: wal interface definition (#33745)
issue: #33285

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-06-24 10:34:12 +08:00
Yinzuo Jiang
7d74edd6dd
fix: update clang-tidy and clang-format from 10 to 12 (#33141)
Default llvm toolchain version in Ubuntu 20.04 is 10, while Ubuntu 22.04
does not have `clang-tidy-10` or `clang-format-10` by default.

issue: #33142

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-06-13 15:27:58 +08:00
chyezh
2b7ee1968f
enhance: new messsage interface for log service (#33286)
issue: #33285

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-06-11 10:38:01 +08:00
wayblink
a1232fafda
feat: Major compaction (#33620)
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
2024-06-10 21:34:08 +08:00
chyezh
f53ab54c5d
enhance: async cgo utility (#33133)
issue: #30926, #33132

- implement future-based cgo utility.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-06-09 22:55:53 +08:00