admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00
Code Issues Packages Projects Releases Wiki Activity
18,560 Commits 146 Branches 196 Tags
Commit Graph

287 Commits

Author SHA1 Message Date
wei liu
322e9f39a3
fix: Remove Unnecessary lock in config manager (#29855) 
issue: #29709 #291712
pr: #29836
to avoid concurrent recursive RLock and Lock cause deadlock, This PR
remove the unnecessary lock in config manager

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-01-11 15:01:01 +08:00
yah01
38c61594c0
enhance: use GPU pool for gpu tasks (#29678) (#29706) 
- this much improve the performance for GPU index
- this also reduce 1x copy while parsing index meta
pr: #29678

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2024-01-09 14:04:48 +08:00
SimFG
a2365e4b2a
enhance: [2.3] Add concurrency for datacoord segment GC (#29557) 
issue: #29553
pr: https://github.com/milvus-io/milvus/pull/29561
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-01-03 13:16:57 +08:00
MrPresent-Han
757834602a
enhance: add param for bloomfilter(#29388) (#29614) 
related: https://github.com/milvus-io/milvus/issues/29388
pr: https://github.com/milvus-io/milvus/pull/29490

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-01-02 18:58:47 +08:00
MrPresent-Han
151a5c3ca8
fix: iterator lose data for duplicted result(#29406) (#29446) 
related: #29406
pr: #29451

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-12-27 23:22:46 +08:00
wei liu
26b1853c54
fix: Auto balance param can't be updated by dynamic(#29501) (#29502) 
pr: #29501
This PR fixed that auto balance param can't be updated by dynamic

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-27 14:30:53 +08:00
aoiasd
f5bba79963
enhance: [Cherry-Pick] add default local path for access log (#29461) 
relate: https://github.com/milvus-io/milvus/issues/29459
pr: https://github.com/milvus-io/milvus/pull/29460

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-26 17:52:47 +08:00
MrPresent-Han
95d1056e32
refine the range of chunk size config value(#29388) (#29472) 
related: #https://github.com/milvus-io/milvus/issues/29388
pr: https://github.com/milvus-io/milvus/pull/29389

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-12-26 14:34:46 +08:00
PowderLi
0c3de8d886
fix: Partition related privileges should belong to Collection (#29429) 
issue: #29427
master pr: #29428

it will try to find privilege_name from the object_type, while grant a
privilege to a role

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-25 14:18:43 +08:00
wei liu
5fa63da4e3
fix: Rename invalid parameterutil package path (#29334) (#29391) 
pr: #29334
This PR renames the invalid parameterutil package path

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-22 10:06:41 +08:00
SimFG
74e72ce27e
enhance: [2.3] Support to get the param value in the runtime (#29298) 
pr: #29297
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-12-21 20:36:43 +08:00
yihao.dai
d8800f5ca5
fix: Return time tick delay error and refine quota error messages (#29289) (#29363) 
This pr:
Handles the time tick delay error when converting old error codes to
milvus errors.
Enhances quota error messages by eliminating "force deny" and
substituting it with "quota exceeded."

pr: https://github.com/milvus-io/milvus/pull/29289
issue: https://github.com/milvus-io/milvus/issues/29288

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-12-21 14:58:49 +08:00
Jiquan Long
8ef0c571b3
enhance: make consistency level used in delete configurable (#29280) (#29284) 
pr: #29280 
issue: #29279

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-20 14:14:34 +08:00
aoiasd
af54ce9e20
fix: [Cherry-pick] msgstream memory leak caused by config event don't deregister (#29268) 
pr: https://github.com/milvus-io/milvus/pull/29266
relate: https://github.com/milvus-io/milvus/issues/28620

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-19 12:34:39 +08:00
wei liu
2d33c7fe41
enhance: Add config for querycoord auto balance channel (#29231) (#29262) 
issue: #23726
pr: #29231
This PR add control config to querycoord's background auto balance
channel operation

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-18 14:32:41 +08:00
PowderLi
9af24da65a
fix: rootCoord decide the builtin role cannot be deleted (#29248) 
issue: #29243

only rootCoord read the configuration item `builtinRoles`, so proxy
never know whether the role to be deleted is builtin.

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-18 14:22:42 +08:00
congqixia
ea688846d8
enhance: [cherry-pick] make conc.Pool resizable (#29225) 
See also #29223
cherry pick part of master commit
pr: #29224
Make `conc.Pool` resizable by adding Resize method for it.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-15 15:28:38 +08:00
congqixia
efef573c67
enhance: [cherry-pick] Add a config item for partition name as regexp feature (#29154) (#29183) 
Cherry pick from master
pr: #29154 
See also #29177
Add a config item for partition name as regexp feature and disable it by
default

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-14 14:46:37 +08:00
Jiquan Long
00e6160848
fix: autoindex panic with flat index type (#29071) (#29072) 
issue: https://github.com/milvus-io/milvus/issues/29048
pr: https://github.com/milvus-io/milvus/pull/29071

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-12 22:08:45 +08:00
wei liu
286dce0d3a
enhance: Add config for auto upgrade segment index (#29112) (#29133) 
pr: #29112
This PR add config for auto upgrade segment index to index engine's
version.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-12 17:26:38 +08:00
Enwei Jiao
1e2a0dd127
enhance: Support otlp with insecure (#29131) 
pr: https://github.com/milvus-io/milvus/pull/29115

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-12-12 16:44:37 +08:00
Jiquan Long
c77fe38c39
fix: txn control not applied to snapshot.Remove (#29124) (#29137) 
issue: #29123
pr: #29124 

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-12 16:37:44 +08:00
yah01
4334e4e7ad
enhance: remove merger for load segments (#29062) (#29064) 
remove merger as now QueryNode could load segments concurrently
fix https://github.com/milvus-io/milvus/issues/29063
pr: #29062

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-12 16:22:50 +08:00
wayblink
e49860cb80
feat: Introduce channelCheckpointUpdater to reduce goroutine in ttNode (#29107) 
pr: #28570

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-12 13:48:42 +08:00
PowderLi
92db41518d
enhance: upgrade proto to use partition related privileges (#29100) 
issue: [milvus-proto
#212](https://github.com/milvus-io/milvus-proto/issues/212)
master pr: #28961

milvus can't use partition related privileges until upgrade
milvus-proto, even if them were added to milvus-proto

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-12 13:22:38 +08:00
shaoyue
ac0f000c29
enhance: Add proxy.ginLogSkipPaths (#29008) 
pr: #28945

/cc @czs007

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2023-12-08 17:55:30 +08:00
shaoyue
ecbb6160aa
enhance: storeageType default value change to remote (#29009) 
pr: #28792
/cc @czs007

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2023-12-08 17:54:33 +08:00
zhenshan.cao
bba0e159ba
feat: [pick] Add bypassing logic for ttMsg in flowgraph of DataNode(#28756) (#29036) 
In order to minimize the CPU usage of the coroutine and avoid frequent
execution of time-consuming operations in the flowgraph when the message
stream consists solely of "ttMsg," it is recommended to implement a
mechanism for quickly bypassing the subsequent flowgraph node processing
logic.

If "ttMsg" is continuously received for a certain period of time
(coldTime), the flowgraph enters skipMode. Once in skipMode, every
skipNum "ttMsg" messages are merged into one for processing. If a
non-"ttMsg" message is received while in skipMode, the flowgraph exits
skipMode.

pr: #28756

Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
 2023-12-08 12:10:39 +08:00
PowderLi
10d3224638
enhance: add 3 builtin roles #28961 (#29010) 
issue: #28960
master pr: #28961

add new configuration: builtinRoles
user can define roles in config file: milvus.yaml

there is an example:

db_ro, only have read privileges, include load
db_rw, read and write privileges, include create/drop/rename collection
db_admin, not only read and write privileges, but also user
administration

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-08 10:52:36 +08:00
cai.zhang
7f59826089
enhance: [Pick]Make import-related error message clearer (#28979) 
issue: #28976 
master pr: #28978

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-08 10:16:35 +08:00
aoiasd
b639fd66b4
fix: [Cherry-pick] accesslog can not print search expression (#28931) 
relate: https://github.com/milvus-io/milvus/issues/28893
pr: https://github.com/milvus-io/milvus/pull/28899

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-08 02:00:36 +08:00
cqy123456
8fd38c8eea
enhance:[cherry-pick] Use binlog index for better search performance (#29012) 
this pr is cherry-pick from master:
pr: https://github.com/milvus-io/milvus/pull/28528
pr: https://github.com/milvus-io/milvus/pull/27673
related issue:
issue: https://github.com/milvus-io/milvus/issues/27678

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-12-07 09:52:34 +08:00
groot
1f8c1afe8a
fix: Change ranges of HNSW parameters according to Knowhere (#28877) 
fix: https://github.com/milvus-io/milvus/issues/28860
pr: https://github.com/milvus-io/milvus/pull/28875

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-12-05 16:20:42 +08:00
congqixia
3a33afd1fb
enhance: [cherry-pick] Change const magic number in querycoord to param (#28819) (#28947) 
Cherry-pick from master
pr: #28819 
See also #28817

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-04 19:06:40 +08:00
aoiasd
224f2755c5
enhance: [Cherry-pick]Refine accesslog to support format by yaml and support more metric (#28740) 
relate: https://github.com/milvus-io/milvus/issues/28086
pr: https://github.com/milvus-io/milvus/issues/28086

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-28 15:34:28 +08:00
cai.zhang
ef115dd7df
enhance: Support Array DataType for bulk_insert (#28578) 
issue: https://github.com/milvus-io/milvus/issues/28272
master pr: #28341 
Support array DataType for bulk_insert with json, binlog files.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-27 16:56:27 +08:00
wei liu
9e82a75c22
enhance: Increase heartbeat timeout (#28648) (#28721) 
pr: #28648
it's easy to trigger heartbeat timeout after 100ms when standalone cpu
usage reach 100%.
This PR increase the heartbeat timeout param to 2000ms

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-27 10:40:26 +08:00
congqixia
6512b12fba
enhance: [cherry-pick] Make etcd kv request timeout configurable (#28661) (#28701) 
Cherry-pick from master
pr: #28661
See also #28660
This pr add request timeout config item for etcd kv request timeout
 Sync the default timeout value to same value for etcdKV & tikv config

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-24 21:16:26 +08:00
yah01
c6ba4fa102
enhance: enhance the accuracy of memory usage (#28554) (#28559) 
before this, Milvus use container/system's memory info to get the memory
usage, which could be inaccurate.

we allocates the memory by private anon mmap,
then rss - shared would be the accurate memory usage

resolve https://github.com/milvus-io/milvus/issues/28553
pr: #28554

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-24 14:12:25 +08:00
yah01
5ca7851f4a
enhance: refine error messages (#28424) (#28614) 
- Split the simple reason and full detail
- Refine existing error messages related: #28422
related: https://github.com/milvus-io/milvus/issues/28422
pr: #28424

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-24 10:04:24 +08:00
zhagnlu
e13b3aa9b1
fix: panic if minio port is empty (#28595) 
#28460 
pr: https://github.com/milvus-io/milvus/pull/28461

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-11-23 20:18:33 +08:00
PowderLi
4a46ae7a8a
Improve HTTP server include metrics and RESTful API (#28210) 
/kind improvement
issue: #27653

pr: #28045

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-19 17:16:21 +08:00
yah01
6820124366
enhance: remove legacy error related code (#28385) (#28415) 
/kind improvement
pr: #28385

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-15 17:24:21 +08:00
aoiasd
2d4659fe4a
Add write cache for access log (#27792) (#28142) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-13 15:08:18 +08:00
wei liu
918333817e
Disable auto balance when old node exists (#28191) (#28224) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-08 07:10:17 +08:00
yah01
d10a82dba4
Fix getting incorrect CPU num (#28178) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-11-07 11:52:22 +08:00
Xiaofan
bbcaf7a703
Fix coordinator fast restart (#28205) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-11-07 10:04:16 +08:00
PowderLi
43db873770
add a custom http header: Accept-Type-Allow-Int64 (#28125) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-06 16:02:18 +08:00
yah01
218a988941
Increase the ChunkManager request timeout (#28015) (#28154) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-04 10:26:17 +08:00
wei liu
e2e2487f40
avoid sleep to ctx done during retry (#27992) (#28092) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-02 09:02:15 +08:00