admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-06 19:02:18 +08:00
Code Issues Packages Projects Releases Wiki Activity
18,660 Commits 137 Branches 188 Tags
Commit Graph

18660 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
aoiasd
010d8362ad
fix: datanode L0 segment num wrong (#29050) 
relate:  https://github.com/milvus-io/milvus/issues/27675

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-20 11:20:44 +08:00
congqixia
cbf0f9c527
enhance: change cp metric to absolute unix ts (#29328) 
See also #29327

Change channel checkpoint metrics to unix seconds instead of checkpoint
timestamp lag value

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-20 11:04:45 +08:00
yah01
61fc822207
fix: creating growing segments may introduce many threads (#29306) 
many growing segments may be created in a short time and there is no
restriction to the process, the CGO call will leave many threads

related: #29282

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-20 10:06:43 +08:00
yah01
8f89e9cf75
enhance: remove all unnecessary string formatting (#29323) 
done by two regex expressions:
- `PanicInfo\((.+),[. \n]+fmt::format\(([.\s\S]+?)\)\)`
- `AssertInfo\((.+),[. \n]+fmt::format\(([.\s\S]+?)\)\)`

related: #28811

---------

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-20 10:04:43 +08:00
congqixia
bcf8f27aa7
enhance: refine proxy meta cache partition logic (#29315) 
See also #29113

- Unify partition info refresh logic
- Prevent parse partition names for each partition key search request

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-20 10:02:43 +08:00
Bingyi Sun
89b208d27a
enhance: Fix format message (#29159) 
Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-20 09:30:44 +08:00
zhuwenxing
9e846d8db2
test: add bulk insert benchmark for different file size (#29320) 
add bulk insert benchmark for different file size

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2023-12-20 09:26:49 +08:00
congqixia
1ee016709d
fix: Unstable TestDataSyncService/TestStartStop unit test (#29291) 
fix #29290

Change EXPECT `NotifyCheckpointUpdated` call to `Maybe` expectation

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-19 23:16:42 +08:00
MrPresent-Han
bfca0a7926
fix: refine skipIndex to resolve cyclic dependcy(#29132) (#29189) 
related: #29132

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-12-19 10:26:40 +08:00
zhagnlu
56d7225673
fix: Add LD_LIBRARY_PATH when local start milvus (#29287) 
#28165

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-12-19 10:08:41 +08:00
nico
4eddcc2113
enhance: update pymilvus version (#29293) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2023-12-19 09:00:42 +08:00
nico
95ada1b679
test: update test cases (#29267) 
issue: #29227

Signed-off-by: nico <cheng.yuan@zilliz.com>
 2023-12-18 19:28:42 +08:00
aoiasd
011a9cee79
test: remove outdated e2e test case beacause disknn support search with search list num large than 200 or 65535 (#28949) 
relate: https://github.com/milvus-io/milvus/issues/28745

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-18 19:26:52 +08:00
PowderLi
bcd6865b29
enhance: add 3 builtin roles (#28961) 
issue: #28960 [milvus-proto
#212](https://github.com/milvus-io/milvus-proto/issues/212)

add new configuration: builtinRoles
user can define roles in config file: `milvus.yaml`

there is an example:
1. db_ro, only have read privileges, include load
2. db_rw, read and write privileges, include create/drop/rename
collection
3. db_admin, not only read and write privileges, but also user
administration

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-18 14:28:41 +08:00
zhagnlu
a602171d06
enhance: Refactor runtime and expr framework (#28166) 
#28165

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-12-18 12:04:42 +08:00
zhuwenxing
438f39e268
test: fix gen array field (#29233) 
fix gen array field

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2023-12-18 11:30:46 +08:00
wei liu
820ee692fc
enhance: Add config for querycoord auto balance channel (#29231) 
issue: #23726
This PR add control config to querycoord's background auto balance
channel operation

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-18 10:00:40 +08:00
PowderLi
3f7d636ee8
enhance: RESTFUL search api support range search (#29055) 
issue: #29004

add a new parameter: `params`, which is a map[string]float64;
but now only 2 valid item: radius + range_filter;

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-17 23:14:44 +08:00
congqixia
88b4b8b77c
enhance: make segments SQPool & LoadPool resizable (#29239) 
See also #29223

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-15 20:22:39 +08:00
nico
1f1a8b7770
enhance: modify test cases for error msg update (#29136) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2023-12-15 17:28:42 +08:00
nico
5da0c8d8e3
enhance: [skip e2e]update chart version (#29041) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2023-12-15 17:26:48 +08:00
XuanYang-cn
5164377e68
fix: Skip updating checkpoint after dropcollection (#29220) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-15 16:04:45 +08:00
Cai Yudong
26409d801e
enhance: Remove omp from segcore (#29207) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-12-15 14:00:39 +08:00
cai.zhang
2f7252b44e
enhance: Set default index name as field name (#29218) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-15 12:06:39 +08:00
congqixia
4731c1b0d5
enhance: make SyncManager pool size refreshable (#29224) 
See also #29223

This PR make `conc.Pool` resizable by adding `Resize` method for it. 
Also make newly added datanode `MaxParallelSyncMgrTasks` config
refreshable

---------

Signed-off-by: Congqi.Xia <congqi.xia@zilliz.com>
 2023-12-15 09:58:43 +08:00
congqixia
25a4525297
enhance: Change sync manager parallel config item (#29216) 
Since the sync manager is global in datanode now, the old
`maxParallelSyncTaskNum` does not fit into current implementation
anymore.

This PR add a new param item for sync mgr parallel control and enlarge
default value

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-14 20:46:41 +08:00
zhuwenxing
6efb7afd3f
test: add more request type checker for test (#29210) 
add more request type checker for test
* partition 
* database
* upsert

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2023-12-14 19:38:45 +08:00
congqixia
8a63e53421
enhance: Add http method to control datacoord garbage collection (#29052) 
See also #29051

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-14 19:26:39 +08:00
XuanYang-cn
cc727ace61
fix: Set compacted segments' level to level one (#29190) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-14 18:46:40 +08:00
yah01
13beb5ccc0
fix: load gets stuck probably (#29191) 
we found the load got stuck probably, and reviewed the logs.

the target observer seems not working, the reason is the taskDispatcher
removes the task in a goroutine, and modifies the task status after
committing the task into the goroutine pool, but this may happen after
the task removed, which leads to the task will never be removed

related #29086

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-14 18:28:38 +08:00
yah01
b8674811cf
fix: data race in ProxyClientManager (#29206) 
this PR changed the ProxyClientManager to thread-safe
fix #29205

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-14 18:22:39 +08:00
sre-ci-robot
bd640754ac
enhance: [skip e2e] skip maximize build space plugin if it is self-hosted runner (#29214) 
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
Co-authored-by: Sammy Huang <sammy.huang@zilliz.com>
 2023-12-14 17:44:39 +08:00
sre-ci-robot
3e66e78508
[automated] Update Knowhere Commit (#29178) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-14 17:16:39 +08:00
wei liu
008bae675d
enhance: Skip balance segment when channel need be balanced (#29116) 
issue: #28622
After we support balance segment with growing segment count #28623, if
we balance segment and channel at same time, some segments need to be
rebalanced after balance channel finish.

This PR skip balance segment when channel need be balanced.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-14 16:44:43 +08:00
SimFG
5a7fb97e07
fix: missing the grant info when using the SelectGrant api with the default db (#29173) 
issue: #29172

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-12-14 16:20:43 +08:00
congqixia
264505b706
enhance: remove reset seed for every shuffle (#29195) 
See also #29113
rand.Seed is deprecated and cost noticable CPU time during heavy payload

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-14 15:46:38 +08:00
wayblink
2274aa3b50
fix: bulkinsert binlog didn't consider ts order when processing delta data (#29163) 
#29162

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-14 14:36:40 +08:00
yah01
58dbb7872a
fix: forbid balancing level zero segments (#29168) 
related #29128
Signed-off-by: yah01 <yah2er0ne@outlook.com>

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-14 14:30:39 +08:00
Jiquan Long
f79c6146bd
enhance: [skip-e2e] auto cancel stale workflows (#29145) 
/kind improvement

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-14 11:30:38 +08:00
congqixia
9407be6eb2
enhance: Add a config item for partition name as regexp feature and disable it by default (#29154) 
See also #29177
Add a config item for partition name as regexp feature and disable it by
default

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-14 10:04:38 +08:00
zhuwenxing
b348827102
test: add array data type and parquet file type for bulk insert case (#29030) 
add array data type and parquet file type for the bulk insert case

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2023-12-13 19:56:38 +08:00
aoiasd
b5ee563914
fix: accesslog can not print search expression (#28899) 
relate: https://github.com/milvus-io/milvus/issues/28893

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-13 18:50:42 +08:00
XuanYang-cn
2c62bcbbf6
fix: Fix some left comments (#29027) 
See also: #28814

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-13 18:48:39 +08:00
Bingyi Sun
ad866d2889
feat: integrate storagev2 into index build process (#28995) 
issue: https://github.com/milvus-io/milvus/issues/28994

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-13 17:24:38 +08:00
Jiquan Long
ed79505d31
enhance: use specific rust version to compile opendal (#29106) 
/kind improvement

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-13 16:34:38 +08:00
Jiquan Long
3acb672497
[automated] Update Builder gpu image changes (#29157) 
/kind improvement
Update Builder gpu image changes
See changes:
4651f89944

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-12-13 14:32:37 +08:00
wei liu
fe1eeae2aa
enhance: Use mockery to replace manual mock code (#29074) 
issue: #29043
This PR remove mannul mock code for proxy and data coord

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-13 10:46:44 +08:00
sre-ci-robot
990d723496
[automated] Update Builder image changes (#29151) 
Update Builder image changes
See changes:
4651f89944
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-13 09:34:39 +08:00
nico
a1d6f02121
test: modify some cases for error msg update (#29022) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2023-12-13 08:54:40 +08:00
congqixia
cb75e73c77
enhance: Use zap.Stringer for large log field (#29143) 
See also #29113
Using zap.Stringer log field will evaluate log field value only when log
level meets the configuration, which could save some CPU time in search
route

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-12 22:52:39 +08:00