admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00
Code Issues Packages Projects Releases Wiki Activity
20,629 Commits 146 Branches 196 Tags
Commit Graph

323 Commits

Author SHA1 Message Date
congqixia
af4e008cd1
enhance: [2.4] Add thread watcher to provide actual thread num (#37905) (#37921) 
Cherry pick from master
pr: #37905 

Related to #37904

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-11-27 11:24:36 +08:00
Ted Xu
e928e15bfc
fix: refuse schedule compaction tasks if there is no slot (#37809) 
See #37621


pr: #37589

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: wei liu <wei.liu@zilliz.com>
 2024-11-25 14:02:34 +08:00
sthuang
d8f1af68e9
enhance: [2.4] RBAC built in privilege groups and grant v2 (#37787) 
cherry-pick from master: https://github.com/milvus-io/milvus/pull/37720,
https://github.com/milvus-io/milvus/pull/37785
issue: https://github.com/milvus-io/milvus/issues/37031

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2024-11-25 11:24:54 +08:00
congqixia
4aca68a739
enhance: Bump milvus & proto verison to v2.4.17 (#37920) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-11-22 14:08:34 +08:00
congqixia
a10f95d71c
enhance: Bump milvus & proto version to v2.4.16 (#37762) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-11-18 20:36:31 +08:00
wei liu
6dc879b1e2
enhance: Enable node assign policy on resource group (#36968) (#37588) 
issue: #36977
pr: #36968
with node_label_filter on resource group, user can add label on
querynode with env `MILVUS_COMPONENT_LABEL`, then resource group will
prefer to accept node which match it's node_label_filter.

then querynode's can't be group by labels, and put querynodes with same
label to same resource groups.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-11-13 11:10:29 +08:00
sthuang
9e8b6ace6d
enhance: [2.4] RBAC custom privilege group (#37560) 
Cherry-pick from master
pr: https://github.com/milvus-io/milvus/pull/37087,
https://github.com/milvus-io/milvus/pull/37558
issue: #37031

---------

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2024-11-11 14:20:29 +08:00
cai.zhang
651a56e3dd
enhance: [2.4]Update the template expression proto to improve transmission efficiency (#37485) 
issue: #36672 

master pr: #37484

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-11-07 12:12:25 +08:00
congqixia
b7c80f9b83
enhance: Bump milvus & proto version to v2.4.15 (#37435) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-11-05 14:46:24 +08:00
SimFG
d0e78cef06
enhance: [2.4] update the expr version to fix the method call error (#37260) 
/kind improvement
- pr: #37259

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-10-31 15:00:23 +08:00
congqixia
0b284ccc23
enhance: Bump milvus & proto version to v2.4.14 (#37198) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-10-29 10:44:25 +08:00
cai.zhang
9c0f59488a
feat: [cherry-pick]The expression supports filling elements through templates (#37058) 
issue: #36672 

master pr: #37033 

milvus-proto pr: https://github.com/milvus-io/milvus-proto/pull/332

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-10-28 15:17:30 +08:00
Xiaofan
2dc89b1cad
enhance: upgrade minio dependency (#37089) 
fix #34910
upgrade minio dependency

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-10-25 15:05:30 +08:00
SimFG
6b9e28bc8f
enhance: [2.4] update the expr version to support automatic conversion of variable types (#36847) 
/kind improvement
- pr: #36832

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-10-15 10:55:23 +08:00
congqixia
bacbfae542
enhance: Bump milvus & proto version to v2.4.13 (#36758) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-10-11 16:01:22 +08:00
SimFG
6604bbda8f
enhance: [2.4] update the expr version and format the expr http response (#36467) 
/kind improvement
- pr: #36406

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-09-26 14:27:20 +08:00
congqixia
2b796b180b
enhance: Bump milvus & proto version to v2.4.12 (#36376) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-23 14:51:13 +08:00
congqixia
9d0378ae84
enhance: Bump milvus & proto version to v2.4.11 (#36069) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-06 23:39:06 +08:00
SimFG
8b706122a8
enhance: [2.4] support to drop the role which is related the privilege list (#35863) 
- issue: #35545
- pr: #35727

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-08-31 21:57:02 +08:00
wei liu
da026b1e28
enhance: Add depguard rules to ban deprecated proto lib (#35140) (#35818) 
See also #34394 #34252
pr: #35140

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: congqixia <congqi.xia@zilliz.com>
 2024-08-30 14:13:01 +08:00
yihao.dai
a1d39326b8
fix: Fix arrow go client (#35819) (#35820) 
issue: https://github.com/milvus-io/milvus/issues/35662

pr: https://github.com/milvus-io/milvus/pull/35819

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-08-29 17:13:00 +08:00
congqixia
fedf3a7fe1
enhance: Bump milvus & proto version to v2.4.10 (#35798) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-29 14:55:01 +08:00
congqixia
ab261d0f8b
feat: [2.4] Support field partial load collection (#35416) (#35696) 
Cherry-pick from master
pr: #35416
Related to #35415

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-27 14:07:00 +08:00
wei liu
e2542a1bf5
enhance: Update protobuf-go to protobuf-go v2 (#34394) (#35555) 
issue: #34252
pr: #34394 #35072 #35084

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-21 18:50:58 +08:00
wei liu
14ec3dc357
enhance: Enable ReadOnly/ReadWrite/Admin Privilege Group to simplify RBAC grant progress (#35472) (#35543) 
issue: #35471
pr: #35472 #35515

---------

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-19 16:24:54 +08:00
wei liu
248a6ea401
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup (#35444) (#35513) 
issue: #35443
pr: #35444

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-18 13:42:55 +08:00
congqixia
4e098f1b23
enhance: Bump milvus & proto to v2.4.9 (#35516) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-16 16:24:54 +08:00
congqixia
e44c63883b
enhance: Bump milvus & proto to v2.4.8 (#35460) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-14 12:52:52 +08:00
congqixia
eebd9804d6
enhance: Bump milvus & proto version to v2.4.7 (#35034) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-06 12:40:16 +08:00
congqixia
935a117396
enhance: [2.4] Support otlp http exporter (#35053) (#35073) 
Cherry-pick from master
pr: #35053
See also #35052

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 21:00:08 +08:00
congqixia
d16320705e
enhance: [2.4] Add Segment Level in milvus segment info APIs (#34763) (#35023) 
Cherry-pick from master
pr: #34763
See also #34746

This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-29 10:11:52 +08:00
chyezh
0f1bb9c697
enhance: recover stack info when non-cgo thread crash (#34866) 
issue: #34864
pr: #34865

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-07-23 14:21:46 +08:00
Xiaofan
66b5cbe27c
fix: CVE by upgrading some dependencies. (#34693) 
fix #34434 and #34456
upgrade otelgrpc to fix CVE

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-07-16 14:05:37 +08:00
jaime
60aab15ef4
enhance: support setting properties in create database request (#34511) 
issue: #34493
pr: #34510

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-09 18:16:18 +08:00
wei liu
d3e94f9861
enhance: Use Blocked Bloom Filter instead of basic bloom fitler impl (#34377) 
issue: #32995
pr: #33405
To speed up the construction and querying of Bloom filters, we chose a
blocked Bloom filter instead of a basic Bloom filter implementation.

WARN: This PR is compatible with old version bf impl, but if fall back
to old milvus version, it may causes bloom filter deserialize failed.

In single Bloom filter test cases with a capacity of 1,000,000 and a
false positive rate (FPR) of 0.001, the blocked Bloom filter is 5 times
faster than the basic Bloom filter in both querying and construction, at
the cost of a 30% increase in memory usage.

Block BF construct time {"time": "54.128131ms"}
Block BF size {"size": 3021578}
Block BF Test cost {"time": "55.407352ms"}
Basic BF construct time {"time": "210.262183ms"}
Basic BF size {"size": 2396308}
Basic BF Test cost {"time": "192.596229ms"}
In multi Bloom filter test cases with a capacity of 100,000, an FPR of
0.001, and 100 Bloom filters, we reuse the primary key locations for all
Bloom filters to avoid repeated hash computations. As a result, the
blocked Bloom filter is also 5 times faster than the basic Bloom filter
in querying.

Block BF TestLocation cost {"time": "529.97183ms"}
Basic BF TestLocation cost {"time": "3.197430181s"}

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-05 17:04:10 +08:00
zhenshan.cao
760b3fafd5
enhance: Refine compaction (#33982) (#34363) 
This PR cherry-picks the following commits related to data compaction:
- enhance: Refine compaction.
[#33982](https://github.com/milvus-io/milvus/pull/33982)
- fix l0 compaction may miss some sealed segments.
[#33838](https://github.com/milvus-io/milvus/pull/33980)

issue : https://github.com/milvus-io/milvus/issues/32939
https://github.com/milvus-io/milvus/issues/33955

pr : https://github.com/milvus-io/milvus/pull/33982

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-07-03 09:52:49 +08:00
congqixia
6b348e4e91
enhance: [2.4] Add go-deadlock as unittest only dependency (#33063) (#34322) 
Cherry-pick from master
pr: #33063
See also #33062

This PR:

- Add lock.RWMutex & lock.Mutex alias to switch implementation based on
build flags
- When build flags has test in it, use go-deadlock to detect possible
deadlocks
- Replace all sync.RWMutex & sync.Mutex in datacoord pkg

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-02 18:48:10 +08:00
zhenshan.cao
14a11e379c
enhance: Refactor Compaction to enable persistence(#33265) (#34268) 
pr : #33265 

issue #33586

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-07-01 19:32:07 +08:00
jaime
6423b6c718
enhance: move rocksmq from internal to pkg (#34165) 
pr:  https://github.com/milvus-io/milvus/pull/33881
issue:  https://github.com/milvus-io/milvus/issues/33956

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-06-26 13:36:05 +08:00
shaoting-huang
b3d425f50a
enhance: Upgrade go version from 1.20 to 1.21 (#33940) 
issue #32982

related pr in master:
pr: #33047 #33150 #33176 #33351 #33202 #33192

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2024-06-20 14:52:00 +08:00
congqixia
59144d8246
enhance: Bump milvus & proto version to v2.4.5 (#33852) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-14 22:27:59 +08:00
congqixia
86f3433053
enhance: [2.4]Use fastjson lib for unmarshal delete log (#33787) (#33802) 
Cherry-pick from master
pr: #33878
```
goos: linux
goarch: amd64
GOMAXPROC=1
cpu: Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
BenchmarkJsonSerdeStd             343872              3568 ns/op            1335 B/op         25 allocs/op
BenchmarkJsonSerdeFastjson       5124177               234.9 ns/op            16 B/op          1 allocs/op
```

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-13 10:27:57 +08:00
congqixia
b2d83d3354
enhance: Bump milvus version to v2.4.2 (#33048) 
Bumping version to v2.4.2. Also bump milvus-proto version to v2.4.3.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-15 13:57:35 +08:00
Xiaofan
36f1ea93a5
enhance: optimize plan parser pool to avoid unnessary recycle (#32869) 
fix #32868
plan parser takes too much cpu on high qps,this pr try to avoid create
lexer and parser too freequent

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-05-11 10:51:31 +08:00
wei liu
c35797c399
enhance: expose DescribeDatabase api in proxy (#32732) 
issue: #32707

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-05-09 22:51:30 +08:00
wayblink
42d0412e93
enhance: Add channelCPs in FlushResponse (#32044) 
#32609

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-04-30 09:45:27 +08:00
congqixia
f07e78ec91
enhance: Bump Milvus & proto version to v2.4.1 (#32693) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-29 14:37:26 +08:00
wei liu
07720f1a95
enhance: expose alter database api in proxy (#32639) 
issue: #30040

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-28 18:15:31 +08:00
SimFG
bed6363feb
enhance: update the go-api version for the list api (#32605) 
issue: https://github.com/milvus-io/milvus/issues/32550
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-26 17:49:33 +08:00
jaime
2c63f848bf
fix: upgrade nats server to fix security vulnerabilities (#32021) 
issue: #32022

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-04-11 16:07:18 +08:00