admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-01 00:15:30 +08:00
Code Issues Packages Projects Releases Wiki Activity
19,527 Commits 138 Branches 188 Tags
Commit Graph

19527 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
congqixia
a938951eea
enhance: [skip e2e] fix upload code cov if syntax (#32128) 
The re-upload step is always true since the syntax is wrong

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-11 10:05:19 +08:00
SimFG
789e014c74
enhance: add the db id for the describe collection response (#32114) 
/kind improvement
issue: #32110

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-10 21:23:20 +08:00
jaime
d4fd6c7283
enhance: add db label on binlog size metrics (#32003) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-04-10 21:01:20 +08:00
wei liu
1a98ce39f5
enhance: Remove useless logic about FromShardLeader (#32029) 
issue: #32047

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-10 20:11:19 +08:00
Buqian Zheng
2fdf1a6e76
feat: [Sparse Float Vector] added some integration tests (#31062) 
add some integration tests for sparse float vector support

https://github.com/milvus-io/milvus/issues/29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-04-10 19:57:18 +08:00
congqixia
25a1c9ecf0
fix: Make coordinator Register not blocked on ProcessActiveStandby (#32069) 
See also #32066

This PR make coordinator register successful and let
`ProcessActiveStandBy` run async. And roles may receive stop signal and
notify servers.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-10 18:49:18 +08:00
zhuwenxing
a697e80a3f
test: use strong as default consistency level (#32095) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-04-10 17:55:18 +08:00
zhuwenxing
5ddeec877a
test: fix vulnerability of python lib (#32109) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-04-10 17:53:18 +08:00
chyezh
a3d6110957
fix: ut failure (#32120) 
issue: #30647

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-10 17:30:48 +08:00
yihao.dai
1b5554c8cb
enhance: Support $meta key for json import (#32013) 
During JSON import:
1. Allow the specification of the $meta key
2. Prohibit duplicated keys within the $meta field, for instance,
`{"id": 1, "vector": [], "x": 6, "$meta": {"x": 8}}`

issue: https://github.com/milvus-io/milvus/issues/31835

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-04-10 17:27:17 +08:00
chyezh
0be67e7f99
fix: ut failure (#32119) 
issue: #30647

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-10 17:23:27 +08:00
Xiaofan
dbab9c5096
fix: reduce didn't handle offset without limit and reduceStopForBest correctly (#32089) 
fix https://github.com/milvus-io/milvus/issues/32059

this pr fix two issues:

offset is not handled correctly without specify a limit
reduceStopForBest doesn't guarantee to return limit result even if there
are more result when there is small segment

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-04-10 16:01:18 +08:00
wei liu
df208d538c
fix: Check exclude segment before add new growing segment (#31803) 
issue: #31479 #31797

milvus will add released segment to excluded info, and filter out it's
stream data in filter_node. but for data buffered in insert_node's
channel, if it belongs to growing segment which already be released,
then it will all the growing segment back again.

This PR maintain `excluded segments` in delegator, and check excluded
segment before new growing segment.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-10 15:29:17 +08:00
Chun Han
f3f2a5a7e9
fix: evicted segments in the serverlss mode(#31959) (#31961) 
related: #31959
1. reset segment index status after evicting to lazyload=true
2. reset num_rows to null_opt

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-04-10 15:15:19 +08:00
wei liu
c4806b69c4
enhance: Refactor leader view manager interface (#31133) 
issue: #31091
This PR add GetByFilter interface in leader view manager, instead of all
kind of get func

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-10 15:13:36 +08:00
XuanYang-cn
aad3ed3835
fix: [cherry-pick]Skip changing meta if nodeID not match with channel (#31672) 
See also: #31648
pr: #31665, #31694

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-04-10 15:09:18 +08:00
SimFG
90bed1caf9
enhance: add the related data size for the read apis (#31816) 
issue: #30436
origin pr: #30438
related pr: #31772

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-10 15:07:17 +08:00
chyezh
c9faa6d936
enhance: add more metrics for cache and search (#31777) 
issue: #30931

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-10 10:55:17 +08:00
congqixia
9d16aa0bd3
enhance: Bump jose2go for security alerts (#32040) 
See also #31986

- jose2go vulnerable to denial of service via large p2c value

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-10 10:05:19 +08:00
SimFG
a63cf8625c
enhance: support to invalid the database meta cache (#32078) 
issue: #32077
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-10 06:45:18 +08:00
wei liu
c5a9cae44e
enhance: [skip e2e]remove useless suspend/resume gc operation in integration test (#31954) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-09 19:55:17 +08:00
zhuwenxing
36514d431a
test: fix some negative case for restful v2 (#32056) 
fix some negative case for restful v2
* fix some error code and message
* fix some payload

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-04-09 18:03:23 +08:00
zhuwenxing
67160e77e0
test: increase the timeout of import (#32064) 
increase the timeout of import， since indexing time is also included

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-04-09 17:55:16 +08:00
SimFG
420baacb6d
enhance: use the rate log in the privilege interceptor (#32037) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-09 16:19:17 +08:00
Cai Yudong
a0a4ec8b67
enhance: make range search param check message more meaningful (#32006) 
Issue: #31970

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-04-09 16:17:26 +08:00
wei liu
177ddda47f
fix: Check stale should check leader task's leader id (#31962) 
issue: #30816

check stale rules for leader task:
1. for reduce leader task, it should keep executing until leader's node
become offline.
2. for grow leader task,it should keep executing until leader's node
become stopping.

This PR check leader node's stopping state for grow leader task

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-09 15:33:25 +08:00
aoiasd
5b693c466d
fix: delegator filter out all partition's delete msg when loading segment (#31585) 
May cause deleted data queryable a period of time.
relate: https://github.com/milvus-io/milvus/issues/31484
https://github.com/milvus-io/milvus/issues/31548

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-04-09 15:21:24 +08:00
zhenshan.cao
089c805e0a
enhance:Refactor hybrid search (#32020) 
issue: https://github.com/milvus-io/milvus/issues/25639
https://github.com/milvus-io/milvus/issues/31368

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-04-09 14:21:18 +08:00
cqy123456
8fda3cbeda
enhance: check fp16/bf16 nan or inf value (#31840) 
issue:https://github.com/milvus-io/milvus/issues/22837

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-04-09 14:19:27 +08:00
cai.zhang
1b767669a4
enhance: Throw error instead of crash when index cannot be built (#31844) 
issue: #27589

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-04-09 11:51:18 +08:00
yiwangdr
1cd15d9322
test: support segment release in integration test (#31190) 
issue: #29507

Notice that api_testonly.go files should be guarded by compiler tag
`test`, so that production build rules don't compile them and these APIs
don't get misused.

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-04-09 11:39:17 +08:00
congqixia
1f7f3993a1
fix: Validate PlaceholderGroups before combine them (#32016) 
See also #32015

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-09 11:33:17 +08:00
sammy.huang
08bfb431b7
enhance: [skip e2e]add prefix to differentiate (#32025) 
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-09 10:57:24 +08:00
congqixia
a860bf66f9
enhance: Bump google.golang.org/grpc to 1.57.1 (#31985) 
See also #31986
See dependency alert for "gRPC-Go HTTP/2 Rapid Reset vulnerability"

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-09 10:53:17 +08:00
congqixia
0fa8c2efd6
test: Use Strong consistency level verifying count(*) (#32031) 
See also #32030

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-09 10:05:17 +08:00
congqixia
7e0d03420e
enhance: Remove duplicated err check code (#32001) 
The `err != nil` check is duplicated here and shall be removed

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-08 21:59:17 +08:00
yihao.dai
49d109de18
enhance: Use an individual buffer size parameter for imports (#31833) 
Use an individual buffer size parameter for imports and set buffer size
to 64MB.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-04-08 21:07:18 +08:00
Ted Xu
9901958288
fix: deadlock in cache concurrent visit (#31938) 
See #31944

There is deadlock in concurrent invocation on `cache.DoWait()`. Suppose
2 callers are calling `DoWait` concurrently, the notification for cache
wait queue may be limited to just 1 caller's occupation instead of them
combined.

To fix this issue, this patch is trying to notify all waiters in queue.

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-04-08 18:55:16 +08:00
sammy.huang
a22dffd68d
enhance: [skip e2e] fetch value of GPU_DATE_VERSION from .env (#32004) 
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-08 17:49:23 +08:00
chyezh
73adf2a5cc
fix: use stateful lock to avoid load and release on LocalSegment concurrently (#31606) 
issue: #31605

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-08 17:09:16 +08:00
sammy.huang
2aa6f3d3ec
enhance: bump codecod-action to v4 (#31984) 
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-08 16:57:17 +08:00
chyezh
7b400252ff
fix: add configuration disk capacity config for lru and fix some bug (#31977) 
issue: #30361

- Add configurable disk capacity limit

- fix bitset reset logic

- make insert record reinsert after clear

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-08 15:55:16 +08:00
sammy.huang
703fcac819
enhance: add openblas in runtime env (#31974) 
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-08 14:57:16 +08:00
congqixia
1c2ae59ece
fix: [skip e2e] Dedup available ports and retry for integration setup (#31902) 
See also #31901

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-08 10:35:17 +08:00
yanliang567
019d76c79d
test: Enable group by tests in e2e (#31903) 
enable groupby tests in e2e

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-04-08 10:25:16 +08:00
nico
14a9785380
test: update pymilvus version and test cases (#31827) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-04-07 18:49:16 +08:00
nico
ddf759abb2
enhance: [skip e2e] update chart version in nightly test (#31960) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-04-07 17:31:23 +08:00
Cai Yudong
00438f408f
enhance: Unify data type check APIs for go (#31887) 
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-04-07 14:27:22 +08:00
cqy123456
aba4993c6c
fix: fix some fp16/bf16 code miss in segcore. (#31771) 
issue：https://github.com/milvus-io/milvus/issues/22837

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-04-07 14:13:16 +08:00
sre-ci-robot
5429c353c5
[automated] Update gpu Builder image changes (#31929) 
Update gpu Builder image changes
See changes:
db698756dc
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-04-07 09:51:21 +08:00