admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-03 17:31:58 +08:00
Code Issues Packages Projects Releases Wiki Activity
19,705 Commits 138 Branches 188 Tags
Commit Graph

8655 Commits

Author SHA1 Message Date
Xiaofan
1e47d7afc4
improve: change some frequent log to debug (#32779) 
remove the frequent log "filter insert messages"

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-05-06 20:51:29 +08:00
Bingyi Sun
fecd9c21ba
feat: LRU cache implementation (#32567) 
issue: https://github.com/milvus-io/milvus/issues/32783
This pr is the implementation of lru cache on branch lru-dev.

Signed-off-by: sunby <sunbingyi1992@gmail.com>
Co-authored-by: chyezh <chyezh@outlook.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
Co-authored-by: Ted Xu <ted.xu@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
 2024-05-06 20:29:30 +08:00
congqixia
6af5275066
fix: Refine index param check for Create/Alter Index (#32712) 
See also #32711

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-06 15:43:29 +08:00
chyezh
b904c8d377
enhance: resource group unittest refactory (#32739) 
issue: #30647

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-05-06 10:17:34 +08:00
chyezh
77f4f0c5a8
fix: gc only do once (#32722) 
issue: #29655

produced by pr: #31740

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-30 14:19:27 +08:00
wayblink
42d0412e93
enhance: Add channelCPs in FlushResponse (#32044) 
#32609

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-04-30 09:45:27 +08:00
Chun Han
ac82cef04d
enhance: disable reload partstats by config (#32702) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-04-29 19:11:26 +08:00
SimFG
0359139299
enhance: return the auth collection list when using the show collection api (#32576) 
issue: #32550
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-29 19:09:34 +08:00
liliu-z
a43baa0323
enhance: Upgrade knowhere to 2.3.1 (#32699) 
/kind improvement

Signed-off-by: Li Liu <li.liu@zilliz.com>
 2024-04-29 17:21:26 +08:00
PowderLi
6289f3a9eb
fix: build milvus in rockylinux8 (#32619) 
issue: #32299

1. xz utils recovers
2. forget to install ninja

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-04-29 14:53:26 +08:00
Buqian Zheng
858599d831
enhance: sparse float vector to support brute force iterator and range search (#32635) 
issue: #29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-04-29 14:35:26 +08:00
wei liu
d900e68440
fix: fix GetShardLeaders return empty node list (#32685) 
issue: #32449

to avoid GetShardLeaders return empty node list, this PR add node list
check in both client side and server side.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-29 14:19:26 +08:00
chyezh
ef4c875d4c
fix: resource group ut may failure (#32688) 
issue: https://github.com/milvus-io/milvus/issues/30647

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-29 14:17:26 +08:00
congqixia
2c1e8f4774
enhance: Use struct{} for sync task future result (#32673) 
Related to #27675

Use `struct{}` instead `error` for sync task future result type to
reduce result size and preventing logci error.

Also change some unused parameter to `_` to suppress lint warning

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-29 10:59:26 +08:00
SimFG
09cd56d44f
enhance: add the skip auto id and partition key check config (#32592) 
/kind improvement
issue: #32591

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-29 10:29:26 +08:00
wei liu
c0555d4b45
fix: Remove read only node from replica immedaitely after node down (#32666) 
issue: #32665

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-28 20:25:25 +08:00
presburger
cb1dbf20c7
enhance:remove openblas from conan (#32002) 
issue: #30683

Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
 2024-04-28 20:13:25 +08:00
yihao.dai
1594122c0a
enhance: Make the dynamic field file optional during numpy import (#32596) 
1. Make the dynamic field file optional during numpy import
2. Add integration importing test with dynamic
3. Disallow file of pk when autoID=true during numpy import

issue: https://github.com/milvus-io/milvus/issues/32542

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-04-28 19:39:25 +08:00
wei liu
07720f1a95
enhance: expose alter database api in proxy (#32639) 
issue: #30040

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-28 18:15:31 +08:00
congqixia
4cdf6c3c41
fix: Check partition nil before observe load progress (#32659) 
See also #32441 #32615

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-28 16:29:25 +08:00
congqixia
a239e9110e
enhance: Apply node-indexing and cache optimization for channel dist (#32595) 
See also #32165

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-28 16:19:24 +08:00
Aldrin
51297b51e1
fix: Removed chance for port conflict in service_test (#32626) 
issue: https://github.com/milvus-io/milvus/issues/32286

- Avoided chance for server initialisation at port 10000

Signed-off-by: Aldrin <imagesai32@gmail.com>
 2024-04-28 14:05:25 +08:00
Aldrin
cb8dbc3c83
fix: Removed minio bucket after use in test (#32624) 
issue: https://github.com/milvus-io/milvus/issues/32616

- Forcefully deleted the non empty minio bucket with dummy data.

Signed-off-by: Aldrin <imagesai32@gmail.com>
 2024-04-28 13:51:26 +08:00
SimFG
9a719ec89e
fix: skip the empty partition name in the rate limit interceptor (#32647) 
issue: https://github.com/milvus-io/milvus/issues/30577

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-28 11:01:28 +08:00
SimFG
bed6363feb
enhance: update the go-api version for the list api (#32605) 
issue: https://github.com/milvus-io/milvus/issues/32550
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-26 17:49:33 +08:00
dengxiaohai
00d0f7c199
enhance: indexnode building index record collection id (#32574) 
Adding a collection id to the index node log allows you to associate an
index building task with a specific collection.
If the host CPU usage is too high due to index build, you can use the
collection id to quickly locate a specific collection, improving fault
locating efficiency.

Signed-off-by: dengxiaohai <rolkdengxiaohai@didiglobal.com>
Co-authored-by: dengxiaohai <rolkdengxiaohai@didiglobal.com>
 2024-04-26 17:05:29 +08:00
smellthemoon
46d7298407
fix: file not find test_parquet.cpp (#32644) 
#32646

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-04-26 16:57:27 +08:00
jaime
3b5cbed86a
enhance: add db label for deletion and bulkinsert operation (#32611) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-04-26 15:53:29 +08:00
smellthemoon
4fb8044a27
enhance: delete some no lint code (#32182) 
#31728

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-04-26 14:15:26 +08:00
Jiquan Long
ccce1e928a
fix: regex query can't handle text with newline (#32569) 
issue: https://github.com/milvus-io/milvus/issues/32482

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-04-26 12:01:26 +08:00
Xiaofan
02ace25c68
enhance: reduce the cpu usage when collection number is high (#32245) 
related to #32165
1. for all the manager, support collection level index
2. remove collection level filter to avoid extra cpu usage when
collection number increases

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-04-26 11:49:25 +08:00
chyezh
2586c2f1b3
enhance: use WalkWithPrefix api for oss, enable piplined file gc (#31740) 
issue: #19095,#29655,#31718

- Change `ListWithPrefix` to `WalkWithPrefix` of OOS into a pipeline
mode.

- File garbage collection is performed in other goroutine.

- Segment Index Recycle clean index file too.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-25 20:41:27 +08:00
chyezh
f06509bf97
fix: get replica should not report error when no querynode serve (#32536) 
issue: #30647

- Remove error report if there's no query node serve. It's hard for
programer to use it to do resource management.

- Change resource group `transferNode` logic to keep compatible with old
version sdk.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-25 19:25:24 +08:00
Buqian Zheng
8a1017a152
enhance: add helpers to parse sparse float vector in JSON (#32543) 
issue: #29419

added helper functions to parse JSON representation of sparse float
vectors, will be used by both the restful server and the import utils.

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-04-25 14:47:24 +08:00
chyezh
b287fbaa2e
fix: return collection on recovering but not collection not loaded when target is not recovered (#32447) 
issue: #32398

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-25 11:21:26 +08:00
congqixia
f30c22626e
enhance: Pre-cache result for frequent filters (#32580) 
See also #32165

Add segment dist and leader view filter criterion struct to store
frequent filter conditions.
Add collection/channel filter results for these two meta

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-25 11:13:25 +08:00
wei liu
04f355a802
enhance: Enable alter database props in rootcoord (#32458) 
issue: #30040

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-25 10:53:25 +08:00
Jiquan Long
c002745902
enhance: retrieve output fields after local reduce (#32346) 
issue: #31822

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-04-25 09:49:26 +08:00
sre-ci-robot
b703d6cdbe
[automated] Update Knowhere Commit (#32586) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-04-25 02:27:24 +08:00
congqixia
faa559592d
enhance: Make applyDelete work in paralell in segment level (#32291) 
`applyDelete` used to be serial for delete entries on each segments.
This PR make it work in parallel with errgroup to improve performance

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-24 17:01:24 +08:00
congqixia
37ca32dbba
enhance: Make SegmentDistManager filter use node index (#32533) 
See also #32165

Change `SegmentDistFilter` to interface in order to provde node index
when filter segment dist.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-24 16:53:24 +08:00
yihao.dai
281a583eda
fix: Correct the negative queryable num entities metric (#32361) 
issue: https://github.com/milvus-io/milvus/issues/32281

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-04-24 15:55:24 +08:00
XuanYang-cn
15b989bb80
fix: Zero flushReq metric for all sealed segs (#32404) 
See also: #32399

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-04-24 14:27:23 +08:00
congqixia
dcc15e3ed8
enhance: Remove TestRateLimiter/tests_refresh_rate_by_config case (#32545) 
See also #32313

This case always fail since the watch config logic is removed

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-24 10:13:24 +08:00
PowderLi
29922829da
fix: [restful v2] parameters of create collection (#32485) 
issue: #31176

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-04-24 09:43:29 +08:00
smellthemoon
365e50b63e
fix: revert add range search params check in proxy (#32366) 
no need to check params in empty segment.
#30365

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-04-23 17:41:23 +08:00
congqixia
6ef677f79e
fix: Remove metrics after flowgraph closed (#32515) 
See also #32403

`fg_buffer_size` was decreased after metrics removed in flowgraph
ddnode, which make metrics value negative.

This PR move remove metrics logic into `dataSyncService.Close`

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-23 17:33:23 +08:00
smellthemoon
96d95e7743
enhance: fix pass error msg as channel name (#32511) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-04-23 16:45:22 +08:00
yihao.dai
3119a7b23f
fix: Remove watching config event in rate limiter (#32313) 
Remove watching config event in rate limiter to prevent object leaks.
Instead, reset limiter rates periodically.

issue: https://github.com/milvus-io/milvus/issues/32312

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-04-23 16:19:24 +08:00
Cai Yudong
16b8b7b35d
enhance: Add get_vector unittest for float16 & bfloat16 (#32153) 
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-04-23 16:15:23 +08:00