admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00
Code Issues Packages Projects Releases Wiki Activity
18,594 Commits 146 Branches 196 Tags
Commit Graph

104 Commits

Author SHA1 Message Date
yah01
38c61594c0
enhance: use GPU pool for gpu tasks (#29678) (#29706) 
- this much improve the performance for GPU index
- this also reduce 1x copy while parsing index meta
pr: #29678

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2024-01-09 14:04:48 +08:00
SimFG
a2365e4b2a
enhance: [2.3] Add concurrency for datacoord segment GC (#29557) 
issue: #29553
pr: https://github.com/milvus-io/milvus/pull/29561
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-01-03 13:16:57 +08:00
MrPresent-Han
757834602a
enhance: add param for bloomfilter(#29388) (#29614) 
related: https://github.com/milvus-io/milvus/issues/29388
pr: https://github.com/milvus-io/milvus/pull/29490

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-01-02 18:58:47 +08:00
wei liu
26b1853c54
fix: Auto balance param can't be updated by dynamic(#29501) (#29502) 
pr: #29501
This PR fixed that auto balance param can't be updated by dynamic

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-27 14:30:53 +08:00
aoiasd
f5bba79963
enhance: [Cherry-Pick] add default local path for access log (#29461) 
relate: https://github.com/milvus-io/milvus/issues/29459
pr: https://github.com/milvus-io/milvus/pull/29460

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-26 17:52:47 +08:00
MrPresent-Han
95d1056e32
refine the range of chunk size config value(#29388) (#29472) 
related: #https://github.com/milvus-io/milvus/issues/29388
pr: https://github.com/milvus-io/milvus/pull/29389

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-12-26 14:34:46 +08:00
Jiquan Long
8ef0c571b3
enhance: make consistency level used in delete configurable (#29280) (#29284) 
pr: #29280 
issue: #29279

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-20 14:14:34 +08:00
aoiasd
af54ce9e20
fix: [Cherry-pick] msgstream memory leak caused by config event don't deregister (#29268) 
pr: https://github.com/milvus-io/milvus/pull/29266
relate: https://github.com/milvus-io/milvus/issues/28620

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-19 12:34:39 +08:00
wei liu
2d33c7fe41
enhance: Add config for querycoord auto balance channel (#29231) (#29262) 
issue: #23726
pr: #29231
This PR add control config to querycoord's background auto balance
channel operation

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-18 14:32:41 +08:00
congqixia
ea688846d8
enhance: [cherry-pick] make conc.Pool resizable (#29225) 
See also #29223
cherry pick part of master commit
pr: #29224
Make `conc.Pool` resizable by adding Resize method for it.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-15 15:28:38 +08:00
congqixia
efef573c67
enhance: [cherry-pick] Add a config item for partition name as regexp feature (#29154) (#29183) 
Cherry pick from master
pr: #29154 
See also #29177
Add a config item for partition name as regexp feature and disable it by
default

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-14 14:46:37 +08:00
wei liu
286dce0d3a
enhance: Add config for auto upgrade segment index (#29112) (#29133) 
pr: #29112
This PR add config for auto upgrade segment index to index engine's
version.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-12 17:26:38 +08:00
Enwei Jiao
1e2a0dd127
enhance: Support otlp with insecure (#29131) 
pr: https://github.com/milvus-io/milvus/pull/29115

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-12-12 16:44:37 +08:00
yah01
4334e4e7ad
enhance: remove merger for load segments (#29062) (#29064) 
remove merger as now QueryNode could load segments concurrently
fix https://github.com/milvus-io/milvus/issues/29063
pr: #29062

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-12 16:22:50 +08:00
wayblink
e49860cb80
feat: Introduce channelCheckpointUpdater to reduce goroutine in ttNode (#29107) 
pr: #28570

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-12 13:48:42 +08:00
shaoyue
ac0f000c29
enhance: Add proxy.ginLogSkipPaths (#29008) 
pr: #28945

/cc @czs007

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2023-12-08 17:55:30 +08:00
shaoyue
ecbb6160aa
enhance: storeageType default value change to remote (#29009) 
pr: #28792
/cc @czs007

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2023-12-08 17:54:33 +08:00
zhenshan.cao
bba0e159ba
feat: [pick] Add bypassing logic for ttMsg in flowgraph of DataNode(#28756) (#29036) 
In order to minimize the CPU usage of the coroutine and avoid frequent
execution of time-consuming operations in the flowgraph when the message
stream consists solely of "ttMsg," it is recommended to implement a
mechanism for quickly bypassing the subsequent flowgraph node processing
logic.

If "ttMsg" is continuously received for a certain period of time
(coldTime), the flowgraph enters skipMode. Once in skipMode, every
skipNum "ttMsg" messages are merged into one for processing. If a
non-"ttMsg" message is received while in skipMode, the flowgraph exits
skipMode.

pr: #28756

Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
 2023-12-08 12:10:39 +08:00
PowderLi
10d3224638
enhance: add 3 builtin roles #28961 (#29010) 
issue: #28960
master pr: #28961

add new configuration: builtinRoles
user can define roles in config file: milvus.yaml

there is an example:

db_ro, only have read privileges, include load
db_rw, read and write privileges, include create/drop/rename collection
db_admin, not only read and write privileges, but also user
administration

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-08 10:52:36 +08:00
cai.zhang
7f59826089
enhance: [Pick]Make import-related error message clearer (#28979) 
issue: #28976 
master pr: #28978

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-08 10:16:35 +08:00
aoiasd
b639fd66b4
fix: [Cherry-pick] accesslog can not print search expression (#28931) 
relate: https://github.com/milvus-io/milvus/issues/28893
pr: https://github.com/milvus-io/milvus/pull/28899

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-08 02:00:36 +08:00
cqy123456
8fd38c8eea
enhance:[cherry-pick] Use binlog index for better search performance (#29012) 
this pr is cherry-pick from master:
pr: https://github.com/milvus-io/milvus/pull/28528
pr: https://github.com/milvus-io/milvus/pull/27673
related issue:
issue: https://github.com/milvus-io/milvus/issues/27678

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-12-07 09:52:34 +08:00
congqixia
3a33afd1fb
enhance: [cherry-pick] Change const magic number in querycoord to param (#28819) (#28947) 
Cherry-pick from master
pr: #28819 
See also #28817

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-04 19:06:40 +08:00
aoiasd
224f2755c5
enhance: [Cherry-pick]Refine accesslog to support format by yaml and support more metric (#28740) 
relate: https://github.com/milvus-io/milvus/issues/28086
pr: https://github.com/milvus-io/milvus/issues/28086

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-28 15:34:28 +08:00
wei liu
9e82a75c22
enhance: Increase heartbeat timeout (#28648) (#28721) 
pr: #28648
it's easy to trigger heartbeat timeout after 100ms when standalone cpu
usage reach 100%.
This PR increase the heartbeat timeout param to 2000ms

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-27 10:40:26 +08:00
aoiasd
2d4659fe4a
Add write cache for access log (#27792) (#28142) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-13 15:08:18 +08:00
wei liu
918333817e
Disable auto balance when old node exists (#28191) (#28224) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-08 07:10:17 +08:00
yah01
d10a82dba4
Fix getting incorrect CPU num (#28178) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-11-07 11:52:22 +08:00
Xiaofan
bbcaf7a703
Fix coordinator fast restart (#28205) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-11-07 10:04:16 +08:00
aoiasd
53246b1b38
Set accesslog default to close and use stdout (#27891) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-10-25 10:30:10 +08:00
wei liu
40723a292e
reduce compact parallel task num (#27899) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-25 09:40:12 +08:00
congqixia
93a877f55e
Make qcv2 target&leader observer execute in parallel (#27844) 
- Add `taskDispatcher` to submit and run task async safely
- Change `LeaderObeserver` and `TargetObserver` schedule and manual check action to submitting task into dispatcher
- Fix logic problem in collection observer when manual check return false

See also #27494

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-24 10:14:11 +08:00
SimFG
9b0ecbdca7
Support to replicate the mq message (#27240) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-10-20 14:26:09 +08:00
smellthemoon
4b0ec156b3
Set channel work pool size in datanode (#27728) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-10-19 08:28:08 +08:00
yah01
9467de79c3
Use pool to block exceeded tasks (#27767) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-19 03:30:06 +08:00
wei liu
43f2452f4b
speed up shard leader cache update frequency (#27640) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-16 10:52:10 +08:00
XuanYang-cn
56c94cdfa7
Add channel manager in DataNode (#27308) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-10-08 21:37:33 +08:00
yihao.dai
106c17f304
Make read ahead policy in ChunkCache configurable (#27291) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-28 15:47:27 +08:00
MrPresent-Han
505e8509b7
refine the default values for segcore threadpool size (#27406) 
/kind improvement

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-09-27 20:43:28 +08:00
wei liu
4071132f6a
reload loading collection when qc recover (#27300) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-09-27 11:43:28 +08:00
foxspy
5db4a0489e
dynamic index version control (#27335) 
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-25 21:39:27 +08:00
foxspy
370b6fde58
milvus support multi index engine (#27178) 
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-22 09:59:26 +08:00
SimFG
26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
Xiaofan
6635398a6d
Fix Bin log concurrency by adding a pool (#27189) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-09-19 18:05:22 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam (#26725) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-05 10:31:48 +08:00
wei liu
1097776477
stop heartbeat if reach heartbeat limit (#26728) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-09-04 17:51:48 +08:00
XuanYang-cn
b2e7cbdf4b
Remove TimeTravel in compactor (#26785) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-09-04 17:41:48 +08:00
yihao.dai
4340cbfba2
Merge syncCPLagTooBehind policy into syncPeriodically policy (#26713) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-04 14:37:47 +08:00
XuanYang-cn
86e9f1818f
Change default compaction timeout to 15mins (#26757) 
See also: #26566

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-08-31 18:07:00 +08:00
Gao
7e36f819b4
Make autoindex config clean (#26732) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-08-31 10:25:01 +08:00