admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-02 00:45:30 +08:00
Code Issues Packages Projects Releases Wiki Activity
20,244 Commits 138 Branches 188 Tags
Commit Graph

551 Commits

Author SHA1 Message Date
aoiasd
c1e04534c3
enhance:change access log write cache default config (#34354) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-07-09 18:18:12 +08:00
congqixia
3333160b8d
enhance: Fix lint issues from recent PRs (#34482) 
See also #34483
Some lint issues are introduced due to lack of static check run. This PR
fixes these problems.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-09 10:06:24 +08:00
Gao
ae6d6f91e6
enhance: change autoindex default metric type (#34261) 
issue: #34304 
cosine is more widely used in float vectors, and cosine and hamming
distance are 'metrics' which have good geometric properties

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-07-08 19:52:24 +08:00
congqixia
145f4f919b
fix: Add nbits parameter check for IVF_PQ (#34451) 
See also #34426

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-08 11:16:10 +08:00
jaime
60be454db0
enhance: add disk quota and max collections into db properties (#34368) 
issue: #34385

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-05 18:22:17 +08:00
zhagnlu
18f2458385
fix: fix bitmap supported type (#34350) 
#34314

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-05 15:50:10 +08:00
jaime
21fc5f5d46
enhance: Remove datanode reporting TT based on MQ implementation (#34421) 
issue: #34420

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-05 15:48:09 +08:00
chyezh
7611128e57
enhance: wal adaptor implementation (#34122) 
issue: #33285

- add adaptor to implement walimpls into wal interface.
- implement timetick sorted and filtering scanner.
- add test for wal.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-07-04 15:23:08 +08:00
XuanYang-cn
8a2be8a457
fix: DataNode might OOM by estimating based on MemorySize (#34201) 
See also: #34136

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-04 10:16:17 +08:00
wei liu
9db897985b
enhance: refine misleading param name for bloom filter parallel factor (#34334) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-04 10:06:10 +08:00
congqixia
94fb5805d5
fix: Use raw parameter value to perform CAS (#34343) 
See also #34342

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-03 16:00:08 +08:00
cai.zhang
8165961223
enhance: Optimeize clustering compaction (#34313) 
issue: #30633

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-03 15:14:08 +08:00
Patrick Weizhi Xu
32c943f1bc
enhance: [skip e2e] update the version of MV (#34378) 
issue: #29892

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-07-03 14:28:16 +08:00
Buqian Zheng
fa8d641ce6
fix: update check for sparse hnsw index (#33713) 
issue: #29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-07-02 21:56:09 +08:00
chyezh
3563136c2a
enhance: timetick interceptor implementation (#34238) 
issue: #33285

- optimize the message package
- add interceptor package to achieve append operation intercepting.
- add timetick interceptor to attach timetick properties for message.
- add timetick background task to send timetick message.

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-07-02 14:42:08 +08:00
cai.zhang
a5be322ab2
enhance: Don't set the timeout for SyncSegments (#34226) 
issue: #32809

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-02 14:28:07 +08:00
yihao.dai
ff51c7e628
fix: Fix panic due to failed to seek (#34229) 
Converting the same msgposition's vchannel to a pchannel multiple times
would result in an invalid pchannel, leading to seek failure and panic.
This PR:
1. Make a copy of msgposition in msgdispatcher.
2. Check if channel is already a pchannel, no further channel conversion
is performed.

issue: https://github.com/milvus-io/milvus/issues/34221

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-01 16:08:12 +08:00
aoiasd
186757e622
enhance: support mark error as user error (#33498) 
relate: https://github.com/milvus-io/milvus/issues/33492

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-07-01 14:56:12 +08:00
jaime
d6afb31b94
enhance: make subfunctions of datanode component modular (#33992) 
issue: #33994

also remove deprecated channel manager based on the etcd implementation

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-01 14:46:07 +08:00
jaime
0426390f06
enhance: improve check health (#33800) 
issue: #34264

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-01 10:16:06 +08:00
wayblink
73ffc1b424
Refine clustering_compaction_task retry mechanism (#34194) 
#32939

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-06-30 20:22:09 +08:00
XuanYang-cn
dda70aa81b
fix: LegacyVersionWithoutRPCWatch default value to 2.4.1 (#34184) 
See also: #31933

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-06-27 19:36:06 +08:00
cqy123456
51ebe95be5
enhance: remove CheckVecIndexWithDataTypeExist function in pkg and remove some cgo call (#34102) 
issue: https://github.com/milvus-io/milvus/issues/22837
related pr: https://github.com/milvus-io/milvus/pull/34104

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-26 21:13:44 +08:00
cqy123456
381d43717d
enhance: update new mmap config parmeters version (#34143) 
issue: https://github.com/milvus-io/milvus/issues/32984
related pr: https://github.com/milvus-io/milvus/pull/34145

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-25 21:20:05 +08:00
wayblink
f9a0f7bb25
Add an option to enable/disable vector field clustering key (#34097) 
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-06-25 18:52:04 +08:00
jaime
d08cb885ca
enhance: enable flush rate limiter of collection level (#33837) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-06-24 10:52:03 +08:00
Patrick Weizhi Xu
b961767005
enhance: support integral type for MV and skip MV if there is only one category (#33161) 
issue: #29892

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-06-24 10:20:01 +08:00
Ted Xu
958ecd500b
feat: adding grouped virtual resource allocator (#33669) 
See #33559

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-06-21 15:58:01 +08:00
congqixia
cc77363b66
enhance: Set maxPartitionNum default value to 1024 (#33949) 
See also #30059

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-20 09:11:59 +08:00
smellthemoon
2a1356985d
enhance: support null in go payload (#32296) 
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-06-19 17:08:00 +08:00
cqy123456
298e50b834
enhance: check index with data type (#33880) 
issue: https://github.com/milvus-io/milvus/issues/22837
related: https://github.com/milvus-io/milvus/pull/33878

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-19 10:15:59 +08:00
chyezh
2f6f964bc8
enhance: [skip e2e] modify gc configuration document (#33946) 
issue: #31740

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-19 09:57:59 +08:00
Gao
a789c60380
enhance: autoindex for multi data type (#33868) 
issue: #22837 

contain https://github.com/milvus-io/milvus/pull/33625
https://github.com/milvus-io/milvus/pull/33867
https://github.com/milvus-io/milvus/pull/33911 which already merged to
2.4 branch

Signed-off-by: chasingegg <chao.gao@zilliz.com>
Co-authored-by: foxspy <xianliang.li@zilliz.com>
 2024-06-18 21:34:01 +08:00
congqixia
3fdaae8792
fix: Return record with largest timestamp for entires with same PK (#33936) 
See also #33883

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-18 15:55:59 +08:00
cqy123456
32f685ff12
enhance: growing segment support mmap (#32633) 
issue: https://github.com/milvus-io/milvus/issues/32984

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-18 14:42:00 +08:00
wei liu
b72026c859
enhance: Add rbac support on describe/alter database api (#33803) 
issue: #32707

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-17 10:07:57 +08:00
zhagnlu
d43ec4db0b
enhance: support array bitmap index (#33527) 
#32900

---------

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-06-16 21:51:58 +08:00
wei liu
4987067375
enhance: Execute bloom filter apply in parallel to speed up segment predict (#33792) 
issue: #33610

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-14 11:37:56 +08:00
wei liu
ab93d9c23d
enhance: Use BatchPkExist to reduce bloom filter func call cost (#33611) 
issue:#33610

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-06-13 17:57:56 +08:00
cqy123456
be3559e022
enhance: proxy check hnsw with sparse is legal (#33697) 
issue:https://github.com/milvus-io/milvus/issues/22837
related:https://github.com/milvus-io/milvus/pull/33653

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2024-06-12 20:13:55 +08:00
congqixia
b90999b741
fix: Compare config value then swap when caching param value (#33785) 
See also #33784

This PR change the behavior of `SetCacheValue` of config manager:

- Use mutex and map instead of concurrent map for `configCache`
- Compare config raw value before set cache value

With this implementation, concurrent caching & eviction shall always
have current output:

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |get     |        |old value|null     |
|t1  |CAS OK  |        |old value|old value|
|t2  |        |update  |new value|old value|
|t3  |        |eviction|new value|null     |

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |get     |        |old value|null     |
|t1  |        |update  |new value|null     |
|t2  |CAS fail|        |old value|null     |
|t3  |        |eviction|new value|null     |

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |        |update  |new value|null     |
|t1  |get     |        |new value|null     |
|t2  |CAS OK  |        |new value|new value|
|t3  |        |eviction|new value|null     |

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |        |update  |new value|null     |
|t1  |get     |        |new value|null     |
|t2  |        |eviction|new value|null     |
|t3  |CAS OK  |        |new value|new value|

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |        |update  |new value|null     |
|t1  |        |eviction|new value|null     |
|t2  |get     |        |new value|null     |
|t3  |CAS OK  |        |new value|new value|

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-12 18:51:56 +08:00
Chun Han
f7af323d1e
fix: sync partitiion stats blocking balance task(#33741) (#33742) 
related: #33741

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-06-11 14:21:56 +08:00
wayblink
a1232fafda
feat: Major compaction (#33620) 
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
 2024-06-10 21:34:08 +08:00
chyezh
f53ab54c5d
enhance: async cgo utility (#33133) 
issue: #30926, #33132

- implement future-based cgo utility.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-06-09 22:55:53 +08:00
XuanYang-cn
1629833060
enhance: Add consts of MsgDispatcher to configs (#33679) 
See also: #33676

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-06-07 14:21:59 +08:00
SimFG
ecee7d90d4
enhance: try to speed up the loading of small collections (#33570) 
- issue: #33569

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-06-07 08:25:53 +08:00
cai.zhang
27cc9f2630
enhance: Support analyze data (#33651) 
issue: #30633

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: chasingegg <chao.gao@zilliz.com>
 2024-06-06 17:37:51 +08:00
aoiasd
b9bafc76b4
enhance: access log support print with consistency level (#33503) 
relate: https://github.com/milvus-io/milvus/issues/33563

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-06-05 17:13:50 +08:00
zhenshan.cao
ac4f3997ce
enhance: Reconstructing Compaction to possess persistence capability (#33265) 
issue #33586

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-06-05 10:17:50 +08:00
Bingyi Sun
d610fdf033
enhance: change Allocator to generic (#33581) 
Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-06-04 19:05:49 +08:00