554 Commits

Author SHA1 Message Date
congqixia
78b266a44f
enhance: Bump builder image go version to v1.24.6 (#44739)
Bump go version fixing CVE issues

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-10-11 09:53:57 +08:00
sangheee
bed94fc061
feat: support grpc tokenizer (#41994)
relate: https://github.com/milvus-io/milvus/issues/41035

This PR adds support for a gRPC-based tokenizer.

- The protobuf definition was added in
[milvus-proto#445](https://github.com/milvus-io/milvus-proto/pull/445).
- Based on this, the corresponding Rust client code was generated and
added under `tantivi-binding`.
  - The generated file is `milvus.proto.tokenizer.rs`.

I'm not very experienced with Rust, so there might be parts of the code
that could be improved.
I’d appreciate any suggestions or improvements.

---------

Signed-off-by: park.sanghee <park.sanghee@navercorp.com>
2025-09-19 17:40:01 +08:00
zhikunyao
008caae713
enhance: base images use https to install apt package (#44382)
Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>
2025-09-16 11:51:59 +08:00
sthuang
0ec47a50ba
enhance: clean up vcpkg build (#44386)
related: #44385

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-09-16 10:21:59 +08:00
aoiasd
2b2a11afba
enhance: use fixed vcpkg version for build image (#44378)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-09-15 19:33:59 +08:00
aoiasd
fb58701cbb
enhance: update rust version (#44322)
relate: https://github.com/milvus-io/milvus/issues/44321

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-09-12 10:53:57 +08:00
WAR10CK
2ad41872da
enhance: support enable asan for docker image builder (#44047)
#44045

Signed-off-by: YangYanbin <warlock.yyb@alibaba-inc.com>
2025-08-26 11:39:14 +08:00
liliu-z
003c348d6d
enhance: Upgrade go version to 1.24.4 to fix CVEs (#43019)
Signed-off-by: liliu-z <liliu-z@users.noreply.github.com>
Co-authored-by: liliu-z <liliu-z@users.noreply.github.com>
2025-07-16 11:28:50 +08:00
zhikunyao
9886d7d4b6
enhance: Master updatecmake zhikun (#43023)
Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>
2025-07-02 11:36:43 +08:00
Zhen Ye
ae43230703
enhance: set jemalloc prof disable by default (#41850)
issue: #40730

- add assertion for insert message
- add more buffer for seal notifier

Signed-off-by: chyezh <chyezh@outlook.com>
2025-05-15 20:10:23 +08:00
shaoyue
f0451d2f3a
enhance: update go version to fix cve (#41696)
/cc @congqixia @czs007

Signed-off-by: haorenfsa <haorenfsa@gmail.com>
2025-05-08 16:04:54 +08:00
zhikunyao
ac1e04372f
enhance: Update go env to 1.24.1 (#41415)
Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>
2025-04-21 16:29:50 +08:00
Zhen Ye
8db708f67d
enhance: enable memory prof based on jemalloc (#40731)
issue: #40730

also see: https://github.com/milvus-io/cgosymbolizer/pull/2

After these PR, at linux:

- the milvus will always enable jemalloc by default.
- jemalloc will always compiled with --enable-prof options.
- all image will always enable the jemalloc prof by default.
- a pprof http service for jemalloc at `/debug/jemalloc/` will be
registered into restful.
- `jeprof` can remote profile the memory of milvus.

Signed-off-by: chyezh <chyezh@outlook.com>
2025-03-19 14:46:18 +08:00
shaoyue
2cd03a20d4
enhance: [skip e2e] upgrade go toolchain to 1.22.7, remove unnecessary binary (#40398)
part of: #40370

Signed-off-by: haorenfsa <haorenfsa@gmail.com>
2025-03-06 21:34:04 +08:00
Bingyi Sun
cb959cd1f9
enhance: upgrade rust version to 1.83 (#39295)
#39254

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-01-20 11:15:03 +08:00
Zhen Ye
f07aa72589
enhance: make milvus image with asan available (#37050)
issue: #35854

Signed-off-by: chyezh <chyezh@outlook.com>
2024-10-24 10:05:29 +08:00
yellow-shine
ac736ad472
enhance: friendly for docker cache (#35789)
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-30 13:05:34 +08:00
Zhen Ye
4b7357ee80
fix: fix typo in milvus spec (#35684)
issue: #35611

introduced by #35610

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-25 16:28:57 +08:00
Zhen Ye
a773836b89
enhance: optimize milvus core building (#35610)
issue: #35549,#35611,#35633

- remove milvus_segcore milvus_indexbuilder..., add libmilvus_core
- core building only link once
- move opendal compilation into cmake
- fix odr

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-23 12:35:02 +08:00
sthuang
d8668fe405
enhance: upgrade go version to 1.21.11 (#35257)
issue: #35142

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-08-05 16:12:15 +08:00
yellow-shine
241c71fdde
enhance: use docker compose instead of docker-compose (#35208)
https://github.com/milvus-io/milvus/issues/35209

---------

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-02 19:32:32 +08:00
yellow-shine
7bf68d8657
enhance: upgrade conan version (#35215)
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-02 19:22:15 +08:00
pingliu
21aa23a23e
doc: [skip e2e] fix the build deb (#34751)
Signed-off-by: ping.liu <ping.liu@zilliz.com>
2024-07-17 19:39:42 +08:00
sammy.huang
2cc9df5c50
[automated] Update cpu Builder image changes (#34078)
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-06-25 10:56:10 +08:00
GenkenWei
a1248a19f8
fix: update ubuntu base image version (#33944)
related to #33945
FIX CVEs of milvus base image: MEDIUM: 8, **Total FIX: 8** for master
branch

![image](https://github.com/milvus-io/milvus/assets/27683687/71708028-265e-4f20-8732-56b7e76864fc)

Signed-off-by: weizhenkun <zhenkun.wei@qq.com>
Co-authored-by: weizhenkun <zhenkun.wei@qq.com>
2024-06-24 14:12:02 +08:00
sammy.huang
b77ab76d74
enhance: get environment variable from .env (#34081)
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-06-23 09:34:07 +08:00
sammy.huang
5be9929d64
enhance: upgrade build-env to ubuntu 22.04 and gcc12 (#33959)
issue: #33960

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-06-20 15:22:02 +08:00
shaoting-huang
eb3197eb1e
enhance: update amazonlinux2023 image builder (#33948)
#32982

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-06-19 16:44:01 +08:00
Yinzuo Jiang
7d74edd6dd
fix: update clang-tidy and clang-format from 10 to 12 (#33141)
Default llvm toolchain version in Ubuntu 20.04 is 10, while Ubuntu 22.04
does not have `clang-tidy-10` or `clang-format-10` by default.

issue: #33142

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-06-13 15:27:58 +08:00
GenkenWei
516f39b616
fix: remove date in the end of ubuntu base image version (#33073)
CVE issues https://github.com/milvus-io/milvus/issues/31997
This PR will fix **32** LOW and **42** MEDIUM CVEs

**ubuntu:focal-20220426 (ubuntu 20.04)**
Total: 99 (UNKNOWN: 0, LOW: 55, MEDIUM: 44, HIGH: 0, CRITICAL: 0)
**ubuntu:focal (ubuntu 20.04)**
Total: 25 (UNKNOWN: 0, LOW: 23, MEDIUM: 2, HIGH: 0, CRITICAL: 0)

![image](https://github.com/milvus-io/milvus/assets/27683687/49719035-8e28-498a-b00d-9c862d531c19)

![image](https://github.com/milvus-io/milvus/assets/27683687/7efc76c0-d333-4569-9455-4aa1a1aa3619)

---------

Signed-off-by: weizhenkun <zhenkun.wei@qq.com>
Co-authored-by: weizhenkun <zhenkun.wei@qq.com>
2024-06-05 10:25:50 +08:00
shaoting-huang
d27db99697
enhance: upgrade amazonlinux2023 builder image go version to 1.21 (#33176)
Signed-off-by: shaoting-huang [shaoting-huang@zilliz.com]

issue: https://github.com/milvus-io/milvus/issues/32982

Go 1.21 introduces several improvements and changes over Go 1.20, which
is quite stable now. This PR is mainly for upgrading images Golang
version from 1.20 to 1.21.

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-05-20 21:11:39 +08:00
shaoting-huang
c35eaaa358
enhance: upgrade images golang version from 1.20 to 1.21 (#33150)
Signed-off-by: shaoting-huang [shaoting-huang@zilliz.com]

issue: https://github.com/milvus-io/milvus/issues/32982

Go 1.21 introduces several improvements and changes over Go 1.20, which
is quite stable now. This PR is mainly for upgrading images Golang
version from 1.20 to 1.21.

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-05-20 15:01:43 +08:00
Yinzuo Jiang
2cc50d80a3
fix: add openblas in install_deps.sh (#33065)
Install openblas using apt or yum in scripts/install_deps.sh, update
documentations and fix some typos related to build and installation.

issue: #33056, #33066

Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-05-17 14:53:37 +08:00
PowderLi
6289f3a9eb
fix: build milvus in rockylinux8 (#32619)
issue: #32299

1. xz utils recovers
2. forget to install ninja

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-04-29 14:53:26 +08:00
sammy.huang
d343d2f3fc
enhance: add vcpkg for rockylinux8 (#32300)
issues:#32299

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-18 11:05:20 +08:00
sammy.huang
184d8000a9
enhance: hotfix for pipeline (#32339)
pr: #32338

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-16 22:58:42 +08:00
sammy.huang
15b9319523
enhance: hot fix publish pipeline (#32330)
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-16 19:48:15 +08:00
sammy.huang
df2ca7f312
enhance: use openblas-devel instead (#32284)
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-15 22:14:43 +08:00
sammy.huang
053347637b
enhance: add openblas-devel (#32049)
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-15 19:47:20 +08:00
sammy.huang
703fcac819
enhance: add openblas in runtime env (#31974)
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-08 14:57:16 +08:00
sammy.huang
db698756dc
enhance: make gpu builder both support amd64 and gpu64 (#31888)
the purpose of this pr is to build arm-based and x86-based
milvusdb/milvus-env for milvus gpu. milvusdb/milvus-env would be based
on ubuntu22.04

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-05 10:09:22 +08:00
sammy.huang
b11c2da930
enhance: for supporting ARM-based GPU images (#31757)
This PR is for supporting ARM-based GPU images.
issue: #31862

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-03 11:49:14 +08:00
sammy.huang
f7a59766df
enhance: support rockylinux8 (#31751)
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-01 10:23:12 +08:00
sammy.huang
b0a941c2fb
enhance: to reduce container image size (#31576)
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-01 09:39:12 +08:00
sammy.huang
1b31824f83
enhance: add openblas dependency (#31437)
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-03-21 15:59:12 +08:00
Guillaume Moutier
d15db7596d
fix: fix permissions on /milvus for OpenShift compatibility (#30775)
OpenShift runs Pods with a random uid and gid 0.
As Milvus needs to write into the /milvus directory, this fix modifies
the group permissions to allow the root group (gid 0) to write into it.
This is a simple solution that allows to run Milvus on OpenShift without
adding security constraints, use special ServiceAccounts or whatever.
This should also not impact any other kubernetes platform as it's a
simple group permission change, nothing regarding the userid.

Note: Other actions are necessary to properly deploy the full stack
(Minio, Pulsar,...) on OpenShift. I will document them in the helm chart
project.

Fixes #25565

Signed-off-by: Guillaume Moutier <guillaume.moutier@gmail.com>
2024-02-28 16:29:05 +08:00
sammy.huang
cbfe344ecf
enhance: [skip 2e2]add the option of host network mode into builder_gpu.sh file (#30573)
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
2024-02-19 14:14:55 +08:00
sammy.huang
46d9ac5931
enhance: move the option of network mode into builder.sh file (#30419)
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
2024-02-01 11:49:03 +08:00
sammy.huang
fb9d64ab1e
enhance: hotfix container network (#30389)
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
2024-01-31 10:23:05 +08:00
sammy.huang
fda6f8d04e
enhance: hotfix docker-compose.yml (#30378)
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
2024-01-30 19:23:04 +08:00