admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2025-12-08 01:58:34 +08:00
Code Issues Packages Projects Releases Wiki Activity
22,467 Commits 133 Branches 183 Tags
Commit Graph

54 Commits

Author SHA1 Message Date
yihao.dai
ee9d08375d
enhance: Accelerate dispatcher building (#42500) 
Reduce check interval to accelerate dispatcher building.

issue: https://github.com/milvus-io/milvus/issues/42067

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-06-05 16:36:32 +08:00
tinswzy
f55f900c85
fix insert hang caused by WAL writer writing to a closing logfile (#42078) 
related issue #42049 
wp commit
[94de4](94de4cbc60)

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
 2025-06-03 09:58:36 +08:00
tinswzy
4edb1bc6f1
fix: resolve wp WALImpls concurrent read/write bug (#41763) 
#41563 #41579 #41842 #41846 #41758
Upgraded the wp dependency to incorporate recent fixes addressing
multiple concurrency bugs in WALImpls.

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
 2025-05-16 12:02:27 +08:00
XuanYang-cn
6b08eff4fe
enhance: Expose configs Manager (#41586) 
See also: #41585

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2025-04-29 17:12:57 +08:00
Zhen Ye
9cb5271027
enhance: remove support of embeded nats mq (#41565) 
issue: #41564

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-04-28 23:06:49 +08:00
tinswzy
6fa68c1f16
enhance: Support Woodpecker as a WAL storage option for Milvus (#41095) 
#40916 Support Woodpecker as a WAL storage option for Milvus

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
 2025-04-20 22:22:42 +08:00
yihao.dai
004a1875dc
enhance: Introduce batch subscription in msgdispatcher (#39863) 
Introduce a batch subscription mechanism in msgdispatcher: the
msgdispatcher now includes a vchannel watch task queue, where all
vchannels in the queue will subscribe to the MQ only once and pull
messages from the oldest vchannel checkpoint to the latest.

issue: https://github.com/milvus-io/milvus/issues/39862

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-03-05 14:38:02 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990) 
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-22 23:15:58 +08:00
wei liu
41dc1a705b
enhance: dump pprof info if component stop progress timeout (#39726) 
issue: #39735

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2025-02-13 15:32:49 +08:00
yihao.dai
38f813bed3
enhance: Read metadata concurrently to accelerate recovery (#38403) 
Read metadata such as segments, binlogs, and partitions concurrently at
the collection level.

issue: https://github.com/milvus-io/milvus/issues/37630

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-01-23 14:27:27 +08:00
yihao.dai
a5a83a0904
fix: Fix consume blocked due to too many consumers (#38455) 
This PR limits the maximum number of consumers per pchannel to 10 for
each QueryNode and DataNode.

issue: https://github.com/milvus-io/milvus/issues/37630

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-01-15 21:37:01 +08:00
yihao.dai
a514f839b2
enhance: Speed up meta recovery (#38285) 
Increase the batchSize in WalkWithPrefix operations to 10000.

issue: https://github.com/milvus-io/milvus/issues/37630

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-12-12 10:58:43 +08:00
XuanYang-cn
a45a288a25
fix: Separate L0 and Mix trigger interval (#37190) 
See also: #37108

- Add MixCompactionTriggerInterval, default 60s
- Add L0CompactionTriggerInterval, default 10s
- Export Single related compaction configs
- Raise SingleCompactionDeltaLogMaxSize from 2MB to 16MB

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-11-12 10:56:37 +08:00
jaime
4746f47282
feat: management WebUI homepage (#36822) 
issue: #36784
1. Implement an embedded web server for WebUI access.  
2. Complete the homepage development.

Home page demo:
<img width="2177" alt="iShot_2024-10-10_17 57 34"
src="https://github.com/user-attachments/assets/38539917-ce09-4e54-a5b5-7f4f7eaac353">

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-10-23 11:29:28 +08:00
congqixia
1184319644
fix: Load original key if ts is MaxTimestamp (#36934) 
Related to #36933

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-10-17 14:11:29 +08:00
Rijin-N
a05a37a583
enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214) 
Native support for Google cloud storage using the Google Cloud Storage
libraries. Authentication is performed using GCS service account
credentials JSON.

Currently, Milvus supports Google Cloud Storage using S3-compatible APIs
via the AWS SDK. This approach has the following limitations:

1. Overhead: Translating requests between S3-compatible APIs and GCS can
introduce additional overhead.
2. Compatibility Limitations: Some features of the original S3 API may
not fully translate or work as expected with GCS.

To address these limitations, This enhancement is needed.

Related Issue: #36212
 2024-09-30 13:23:32 +08:00
jaime
52cce4de58
fix: iaccurate size estimation for encoded array data (#36373) 
issue: #36029

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-09-24 14:51:14 +08:00
SimFG
c50fe71163
fix: long buffering causes mq to be unable to receive messages. (#36420) 
- issue: #36397

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-09-23 16:33:18 +08:00
Xiaofan
885d891106
Change minio use virtual config to not panic (#35327) 
minio use virtual should not panic if empty. User may use other object
storage instead

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-08-07 09:58:15 +08:00
Ted Xu
b4d0f4df0a
enhance: update param documents (#35152) 
See: #35161

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-08-01 12:38:13 +08:00
XuanYang-cn
1629833060
enhance: Add consts of MsgDispatcher to configs (#33679) 
See also: #33676

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-06-07 14:21:59 +08:00
congqixia
c2ac692008
enhance: Add param item to ignore bad message id in checkpoint (#33123) 
See also #33122

This pr add param item `mq.ignoreBadPosition` to control behavior when
mq failed to parse message id from checkpoint

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-20 11:29:37 +08:00
chyezh
2586c2f1b3
enhance: use WalkWithPrefix api for oss, enable piplined file gc (#31740) 
issue: #19095,#29655,#31718

- Change `ListWithPrefix` to `WalkWithPrefix` of OOS into a pipeline
mode.

- File garbage collection is performed in other goroutine.

- Segment Index Recycle clean index file too.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-25 20:41:27 +08:00
Ted Xu
78d32bd8b2
enhance: update milvus.yaml (#31832) 
See #32168

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-04-16 16:17:19 +08:00
groot
5be395354c
fix: minio ssl compatible issue (#31607) 
issue: https://github.com/milvus-io/milvus/issues/30709

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2024-03-27 14:41:20 +08:00
groot
c81909bfab
enhance: Support MinIO TLS connection (#31311) 
issue: https://github.com/milvus-io/milvus/issues/30709
pr: #31292

Signed-off-by: yhmo <yihua.mo@zilliz.com>
Co-authored-by: Chen Rao <chenrao317328@163.com>
 2024-03-21 11:15:20 +08:00
groot
85de56e894
fix: Clean kafka default configuration (#30924) 
issue: #30917

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2024-03-01 18:17:03 +08:00
groot
ba6d33cd57
fix: Support TLS for kafka connection (#30468) 
#27977

Add extra configurations in milvus.yaml to pass certificates for kafka.

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2024-02-28 18:43:07 +08:00
chyezh
211143c5e6
enhance: add basic information of milvus into metrics (#29665) 
add basic build information and runtime component dependency into
metrics.

issue: #29664

Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2024-01-29 15:47:02 +08:00
PowderLi
08ca0a2ca5
feat: support etcd authentication (#30226) 
issue: #28895
add 3 configuration for ETCD config

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-01-24 11:35:00 +08:00
congqixia
a2fe9dad49
enhance: Make etcd kv request timeout configurable (#28661) 
See also #28660
This pr add request timeout config item for etcd kv request timeout
 Sync the default timeout value to same value for etcdKV & tikv config

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-23 19:34:23 +08:00
zhagnlu
bc43c5cf34
fix: Panic if minio port is empty (#28461) 
#28460

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-11-23 19:24:25 +08:00
zhagnlu
6060dd7ea8
Add chunk manager request timeout (#27692) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-10-23 20:08:08 +08:00
PowderLi
8d3069b1db
update openssl to 3.1.2 (#27399) 
deal with root path's normalization

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-10-08 19:17:31 +08:00
yihao.dai
fe01d54eca
Set kafka read timeout to 10s and make it configurable (#27238) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-20 22:01:27 +08:00
chyezh
791e6ef6c6
[Improvement] add pulsar metrics and fix timeout (#26907) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-09-13 12:03:19 +08:00
yiwangdr
337edc321b
tikv integration (#26246) 
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2023-09-07 07:25:14 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam (#26725) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-05 10:31:48 +08:00
congqixia
d343888f3c
Fix pkg codecov & static-check script (#26720) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-31 09:59:01 +08:00
jaime
c603f1c244
Remove mysql metastore (#26633) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-08-29 14:36:26 +08:00
congqixia
ee7aef9272
Make pulsar request timeout configurable (#26525) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-23 20:46:23 +08:00
chyezh
4c4b2903f9
[Fixup] nats log configuration, open nats log at info level by deafult (#26168) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-08-14 14:21:32 +08:00
zhagnlu
411f9ac823
Upgrade minio-go and add region and virtual host config for segcore chunk manager (#26194) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-08-11 10:37:36 +08:00
congqixia
767955ec6b
Reduce MQ buffer length and flowgraph wait queue length to 16 (#26179) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-09 10:05:14 +08:00
wayblink
775f03ed5d
Add default value for mq.type (#26064) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-08-04 18:35:07 +08:00
congqixia
fcde5b8d7f
Add pursuit mode for mqtt msgstream (#26098) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-04 12:33:06 +08:00
zhagnlu
833674c1cb
add glog configurable function and redirect aws log to segcore log (#25664) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-07-27 19:49:02 +08:00
chyezh
e24a8b3606
[Feature] Add benchmark and retention configuration for nmq (#25768) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-07-25 19:33:01 +08:00
aoiasd
3545b1a608
Refine rocksmq (#25031) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-07-18 20:18:57 +08:00
wayblink
50454bec4c
Allow empty minio.rootPath (#25534) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-07-13 14:46:29 +08:00