sthuang
a85e36bad2
fix: create collection task check failed after restart ( #40982 )
...
The fields and partitions information are stored and fetched with
different prefixes in the metadata. In the CreateCollectionTask, the
RootCoord checks the existing collection information against the
metadata. This check fails if the order of the fields or partitions info
differs, leading to an error after restarting Milvus. To resolve this,
we should use a map in the check logic to ensure consistency.
related: https://github.com/milvus-io/milvus/issues/40955
---------
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-04-05 06:58:22 +08:00
Zhen Ye
f18aa85083
enhance: vchannel fair balance policy for streaming ( #40959 )
...
issue: #40638
- Add `ChannelID` for streaming replica in future.
- Remove the pchannel count fair balance policy for streaming.
- Add Score based vchannel fair balance policy for streaming.
- Add pchannel stats manager to collect the stats of pchannel for
balancer.
- Add configuration and metrics for new balance policy
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-04-04 10:12:22 +08:00
cai.zhang
123b6588b6
feat: Support get segment binlogs info with new interface GetSegmentsInfo ( #40464 )
...
issue: #40341
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-03-16 21:04:07 +08:00
yihao.dai
b2a8694686
enhance: Merge IndexNode and DataNode ( #40272 )
...
Merge DataNode and IndexNode into DataNode.
issue: https://github.com/milvus-io/milvus/issues/39115
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-03-13 14:26:11 +08:00
Zhen Ye
f47ab31f23
enhance: remove redundant resource key watch operation, just keep consistency of wal ( #40235 )
...
issue: #38399
related PR: #39522
- Just implement exclusive broadcaster between broadcast message with
same resource key to keep same order in different wal.
- After simplify the broadcast model, original watch-based broadcast is
too complicated and redundant, remove it.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-03-03 14:40:05 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module ( #39990 )
...
Related to #39095
https://go.dev/doc/modules/version-numbers
Update pkg version according to golang dep version convention
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
SimFG
047254665d
feat: support to replicate import msg ( #39171 )
...
- issue: #39849
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: chyezh <chyezh@outlook.com>
Co-authored-by: chyezh <chyezh@outlook.com>
2025-02-16 00:08:13 +08:00
Zhen Ye
a9e0e0a852
enhance: broadcast with event-based notification ( #39522 )
...
issue: #38399
- broadcast message can carry multi resource key now.
- implement event-based notification for broadcast messages
- broadcast message use broadcast id as a unique identifier in message
- broadcasted message on vchannels keep the broadcasted vchannel now.
- broadcasted message and broadcast message have a common broadcast
header now.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-07 11:14:43 +08:00
Zhen Ye
bb8d1ab3bf
enhance: make new go package to manage proto ( #39114 )
...
issue: #39095
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-10 10:49:01 +08:00
sthuang
19572f5b06
enhance: RBAC new grant/revoke privilege ( #37785 )
...
issue: https://github.com/milvus-io/milvus/issues/37031
also fix issues: https://github.com/milvus-io/milvus/issues/37843 ,
https://github.com/milvus-io/milvus/issues/37842 ,
https://github.com/milvus-io/milvus/issues/37887
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-21 22:20:34 +08:00
congqixia
b0bd290a6e
enhance: Use internal json(sonic) to replace std json lib ( #37708 )
...
Related to #35020
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-18 10:46:31 +08:00
Zhen Ye
b5b003551e
enhance: use localhost for it and ut ( #37529 )
...
issue: #37528
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-12 11:36:27 +08:00
yihao.dai
994f52fab8
fix: Revert "enhance: Support db for bulkinsert ( #37012 )" ( #37420 )
...
This reverts commit 6e90f9e8d90440716d596a7fe8fe1db465d529b7.
issue: https://github.com/milvus-io/milvus/issues/31273
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-07 17:02:25 +08:00
yihao.dai
6e90f9e8d9
enhance: Support db for bulkinsert ( #37012 )
...
issue: https://github.com/milvus-io/milvus/issues/31273
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-25 14:31:39 +08:00
Zhen Ye
d29e01e284
fix: port listen racing in mix or standalone mode ( #36442 )
...
issue: #36441
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-26 21:23:16 +08:00
SimFG
731d45abbe
enhance: provide more general configuration to control mmap behavior ( #35359 )
...
- issue: #35273
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-21 00:22:54 +08:00
SimFG
b2cc4b0776
feat: add the rbac msg and send them to the replicate channel ( #35392 )
...
- issue: #35391
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-15 12:06:52 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 ( #34394 )
...
issue: #34252
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-29 11:31:51 +08:00
congqixia
4ee6c69217
enhance: Add Segment Level in milvus segment info APIs ( #34763 )
...
See also #34746
This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-26 10:01:46 +08:00
smellthemoon
07b94b4615
enhance: support upsert autoid==true ( #30342 )
...
related with: #29258
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-07-11 16:53:35 +08:00
aoiasd
1b4e28b97f
enhance: Check by proxy rate limiter when delete get data by query. ( #30891 )
...
relate: https://github.com/milvus-io/milvus/issues/30927
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-23 20:03:40 +08:00
Cai Yudong
4fc7915c70
enhance: unify data generation test APIs ( #32955 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-14 14:33:33 +08:00
SimFG
8594b55ad5
enhance: add max insert request size and must use partition key configs ( #32433 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-19 10:31:20 +08:00
SimFG
1af084ea6b
enhance: Make datanode exit and case TestProxy faster ( #32218 )
...
/kind improvement
issue: #32219
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-16 10:49:20 +08:00
SimFG
c012e6786f
feat: support rate limiter based on db and partition levels ( #31070 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
co-author: @jaime0815
---------
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-04-12 16:01:19 +08:00
SimFG
a63cf8625c
enhance: support to invalid the database meta cache ( #32078 )
...
issue: #32077
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-10 06:45:18 +08:00
zhenshan.cao
089c805e0a
enhance:Refactor hybrid search ( #32020 )
...
issue: https://github.com/milvus-io/milvus/issues/25639
https://github.com/milvus-io/milvus/issues/31368
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-04-09 14:21:18 +08:00
SimFG
ac26908cc4
enhance: Remove the storage info report ( #31772 )
...
issue: #30436
origin pr: #30438
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-01 20:50:59 -07:00
SimFG
b1a1cca10b
feat: add more operation detail info for better allocation ( #30438 )
...
issue: #30436
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-03-28 06:33:11 +08:00
yihao.dai
8cb06acfed
feat: Replacing the current import API with the v2 implementation ( #31046 )
...
Replacing the current import API v1 implementation with the v2
implementation.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-10 12:23:02 +08:00
cai.zhang
f619d792c0
enhance: Break down the granularity of collection info cache expired ( #29977 )
...
issue: #29772
1. `DropPartition` only invalidates the cache related to the partition.
2. `CreateAlias` does not invalidate the cache.
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-01-30 16:45:02 +08:00
SimFG
463765922e
enhance: support related privilege for grant api ( #30153 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-01-23 15:56:54 +08:00
wayblink
1df3f90696
feat: Implement DescribeAlias and ListAliases interfaces ( #29641 )
...
#22882
/kind feature
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-01-11 19:12:51 +08:00
xige-16
9702cef2b5
feat: Support multiple vector search ( #29433 )
...
issue #25639
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2024-01-08 15:34:48 +08:00
yah01
99e0f1e65a
enhance: unable to compile C++ tests ( #29616 )
...
The tests need to call a private method, Milvus uses `#define` to
replace private with public, the hack trick works but would be broken if
the including order changed.
This uses friend to make all things work well
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2024-01-04 13:20:46 +08:00
smellthemoon
ae640e7c80
fix: pass in undefined params ( #29591 )
...
fix pass in undefined params
issue: #29594
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-12-30 00:32:47 +08:00
xige-16
02673914a0
feat: Support multiple vector indexes in a collection ( #27700 )
...
issue: #25639
/kind improvement
Signed-off-by: xige-16 <xi.ge@zilliz.com>
---------
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-12-29 11:44:45 +08:00
xige-16
0a70e8b601
enhance: Remove multiple vector field limit ( #27827 )
...
issue: https://github.com/milvus-io/milvus/issues/25639
/kind improvement
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-12-28 16:40:46 +08:00
yah01
a0e1a1eb31
feat: support enable/disable mmap for index ( #29005 )
...
support enable/disable mmap for index, the user could alter the index's
mode by `AlterIndex` method
related: https://github.com/milvus-io/milvus/issues/21866
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-21 18:07:24 +08:00
Filip Haltmayer
6b1a106a31
Moving etcd client into session ( #27069 )
...
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
2023-10-27 07:36:12 +08:00
Futao Wei
599012a340
Implement SearchByPks path for Search ( #25882 )
...
Signed-off-by: unfode <forrest.futao.wei@gmail.com>
2023-10-16 15:34:08 +08:00
wei liu
310739d115
remove flush from ddl queue in proxy ( #27688 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-13 18:33:36 +08:00
yah01
be980fbc38
Refine state check ( #27541 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
yah01
3759857bc5
Refine Proxy errors ( #27499 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-09 10:09:33 +08:00
yah01
63ac43a3b8
Refine errors for import ( #27379 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-30 10:31:28 +08:00
yah01
6539a5ae2c
Refine DataCoord status ( #27262 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-26 17:15:27 +08:00
jaime
7f7c71ea7d
Decoupling client and server API in types interface ( #27186 )
...
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
aoiasd
0a0e4f0547
Remove old mockChannelsMgrs, use mockery MockChannelsMgr instead ( #27318 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-09-25 10:31:26 +08:00
SimFG
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
yah01
b4f86ea55e
Construct all success status with merr ( #27226 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-20 10:57:23 +08:00
