admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-04 18:02:08 +08:00
Code Issues Packages Projects Releases Wiki Activity
22,784 Commits 136 Branches 188 Tags
Commit Graph

57 Commits

Author SHA1 Message Date
sthuang
a85e36bad2
fix: create collection task check failed after restart (#40982) 
The fields and partitions information are stored and fetched with
different prefixes in the metadata. In the CreateCollectionTask, the
RootCoord checks the existing collection information against the
metadata. This check fails if the order of the fields or partitions info
differs, leading to an error after restarting Milvus. To resolve this,
we should use a map in the check logic to ensure consistency.

related: https://github.com/milvus-io/milvus/issues/40955

---------

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2025-04-05 06:58:22 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990) 
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-22 23:15:58 +08:00
Xianhui Lin
82f9689711
enhance: Add schema update time verification for insert and upsert to use cache (#39096) 
enhance: Add schema update time verification for insert and upsert to
use cache
issue: https://github.com/milvus-io/milvus/issues/39093

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-02-07 14:10:45 +08:00
yihao.dai
0df2c75b77
fix: Fix rootcoord meta mutex contention (#38799) 
RootCoord meta uses copy-on-write, allowing the removal of unnecessary
copies.

issue: https://github.com/milvus-io/milvus/issues/37630

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-01-15 20:11:08 +08:00
cai.zhang
6d45dd5666
fix: Add scalar index engine version for compatibility (#39204) 
issue: #39203

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-01-15 12:25:00 +08:00
Zhen Ye
3e788f0fbd
enhance: record memory size (uncompressed) item for index (#38770) 
issue: #38715

- Current milvus use a serialized index size(compressed) for estimate
resource for loading.
- Add a new field `MemSize` (before compressing) for index to estimate
resource.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-01-14 10:33:06 +08:00
Zhen Ye
bb8d1ab3bf
enhance: make new go package to manage proto (#39114) 
issue: #39095

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-01-10 10:49:01 +08:00
jaime
f03a85725a
enhance: add db name in replica (#38672) 
issue: #36621

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2025-01-09 19:40:59 +08:00
SimFG
2afe2eaf3e
feat: support to replicate collection when the services contains the system tt msg (#37559) 
- issue: #37105

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-12-17 09:08:46 +08:00
jaime
60dd55f292
enhance: add end time stats for compaction and index build tasks (#38048) 
issue: #36621

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-12-06 16:28:40 +08:00
SimFG
49ee46ec1d
enhance: support to config the default db properties (#38035) 
- issue: #38034

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-11-27 10:04:34 +08:00
jaime
1d06d4324b
fix: Int64 overflow in JSON encoding (#37657) 
issue: ##36621

- For simple types in a struct, add "string" to the JSON tag for
automatic string conversion during JSON encoding.
- For complex types in a struct, replace "int64" with "string."

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-11-14 22:52:30 +08:00
aoiasd
da227ff9a1
feat: Support create collection with functions (#35973) 
relate: https://github.com/milvus-io/milvus/issues/35853
Support create collection with functions. Prepare for support bm25
function.

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-09-12 10:43:06 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394) 
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-29 11:31:51 +08:00
jaime
4365308241
enhance: support setting properties in create database request (#34510) 
issue: #34493

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-09 18:16:18 +08:00
jaime
60be454db0
enhance: add disk quota and max collections into db properties (#34368) 
issue: #34385

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-05 18:22:17 +08:00
smellthemoon
b45798107a
enhance: add nullable in Field, check valid_data and fill data (#32086) 
1. add nullable in model.Field
   help to read nullable accurately.
2. check valid_data
a. if user pass default_value or the field is nullable, the length of
valid_data must be num_rows.
b. if passed valid_data, the length of passed field data must equal to
the number of 'true' in valid_data.
c. after fill default_value, only nullable field will still has
valid_data.
3. fill data in two situation
    a. has no default_value, if nullable,
will fill nullValue when passed num_rows not equal to expected num_rows.
    b. has default_value,
will fill default_value when passed num_rows not equal to expected
num_rows.
c. after fill data, the length of all field will equal to passed
num_rows.
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-05-16 11:57:35 +08:00
SimFG
b1a1cca10b
feat: add more operation detail info for better allocation (#30438) 
issue: #30436

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-03-28 06:33:11 +08:00
wei liu
e6d50def4f
enhance: Enable properites in database meta (#31394) 
issue: #30040

This PR add properties in database meta, so we can store some database
level info.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-03-19 19:21:06 +08:00
wayblink
635a7f777c
feat: add clustering key in create/describe collection (#29506) 
#28410
/kind feature

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-01-07 19:56:48 +08:00
Bingyi Sun
ad866d2889
feat: integrate storagev2 into index build process (#28995) 
issue: https://github.com/milvus-io/milvus/issues/28994

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-13 17:24:38 +08:00
Jiquan Long
370fdaf50d
Record engine version for segment index (#27384) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-28 18:03:28 +08:00
foxspy
5db4a0489e
dynamic index version control (#27335) 
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-25 21:39:27 +08:00
foxspy
370b6fde58
milvus support multi index engine (#27178) 
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-22 09:59:26 +08:00
SimFG
26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
cai.zhang
a362bb1457
Support array datatype (#26369) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-09-19 14:23:23 +08:00
yah01
213db490bd
Use pointer receiver for large struct (#26668) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-30 10:24:29 +08:00
jaime
18df2ba6fd
[Cherry-Pick] Support Database (#24769) 
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-06-25 17:20:43 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-09 01:28:37 +08:00
Jiquan Long
d531b17784
Add ut to check if two collections are identical (#24734) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-06-08 15:58:35 +08:00
Jiquan Long
8156317c9c
Fix creating collection not idempotent (#24721) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-06-07 20:08:34 +08:00
xige-16
732fe54775
Support partition Key (#24047) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-06 10:24:34 +08:00
smellthemoon
708c724ccc
fix default value store unknown type (#24477) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-05-30 14:51:28 +08:00
smellthemoon
473c65cd04
Make field in metastore contains default value (#24228) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-05-19 12:19:23 +08:00
cai.zhang
008285f849
Support dynamic schema for create collection (#24176) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-05-18 09:33:24 +08:00
yihao.dai
8c060215e9
Add collection number quota per DB (#23656) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-28 11:02:35 +08:00
zhenshan.cao
03ec804e68
Change default shard number to 1 (#23586) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-04-21 07:08:32 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
cai.zhang
e127cf7b99
Reset indexpb for upgrade (#21620) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-11 14:35:40 +08:00
cai.zhang
e5f408dceb
Merge IndexCoord and DataCoord (#21267) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-04 19:37:36 +08:00
Jiquan Long
0edb49209d
Fix rootcoord metrics (#20427) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-09 17:49:04 +08:00
yah01
aa6b8af80d
Migrate metadata of QueryCoord from 2.1 to 2.2 (#19986) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-24 23:41:35 +08:00
jaime
b79687687d
Replace segment index file path with index file id within segment index meta (#19866) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-19 16:55:27 +08:00
SimFG
a55f739608
Separate public proto files (#19782) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-16 20:49:27 +08:00
Jiquan Long
a8a074162f
Add meta migration tool (#19709) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-12 11:37:23 +08:00
jaime
31db01b6ae
Support TTL for collection level (#19383) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-10 20:31:22 +08:00
zhenshan.cao
0ac4bc32a5
Add logic of autoindex (#19570) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-10-08 15:38:58 +08:00
cai.zhang
fc3309b8b8
All indexes segments both write handoff (#19230) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-09-21 16:34:51 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-16 16:56:49 +08:00
Jiquan Long
a5e2d6b6fb
Refactor RootCoord (#18930) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>
 2022-09-05 13:29:11 +08:00