congqixia
12575885d3
enhance: [2.4] Add skip load validation for create collection task ( #35761 )
...
Cherry-pick from master
pr: #35737
Related to #35415
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-28 18:25:00 +08:00
congqixia
ab261d0f8b
feat: [2.4] Support field partial load collection ( #35416 ) ( #35696 )
...
Cherry-pick from master
pr: #35416
Related to #35415
---------
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-27 14:07:00 +08:00
SimFG
5b5119a51f
feat: [2.4] provide more general configuration to control mmap behavior ( #35609 )
...
- issue: #35273
- pr: #35359
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-23 12:35:02 +08:00
zhagnlu
946881e123
enhance: optimize retrieve on dynamic field ( #35602 )
...
pr: #35580
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-08-22 14:18:57 +08:00
wei liu
e2542a1bf5
enhance: Update protobuf-go to protobuf-go v2 ( #34394 ) ( #35555 )
...
issue: #34252
pr: #34394 #35072 #35084
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-21 18:50:58 +08:00
congqixia
555b7a6aa7
enhance: [2.4] Remove duplicated schema helper creation in proxy ( #35489 ) ( #35502 )
...
Cherry-pick from master
pr: #35489
Related to PRs of #35415
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-16 14:32:54 +08:00
Jiquan Long
34873a7f76
feat: record the duration waiting in the proxy queue ( #34744 ) ( #34788 )
...
fix: https://github.com/milvus-io/milvus/issues/34743
pr: #34744
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-07-19 15:37:44 +08:00
Patrick Weizhi Xu
3cf2e1831a
feat: [cherry-pick][2.4] support partition key isolation ( #34336 ) ( #34613 )
...
issue: #34332
pr: #34336
---------
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 104d0966b7df80a251437bed74f28fa47e138397)
2024-07-17 14:03:43 +08:00
wayblink
c62bf8a0b0
fix: [Cherry-pick]Pick major compaction fixs and optimizations ( #34360 )
...
This PR cherry-picks the following commits:
- fix: sync partitiion stats blocking balance task #33742
- fix: Fix meta prefix overlap bug #33830
- fix: Small fixs of major compaction #33929
- fix: Fix memory buffer error & some renaming #33850
- fix: sync part stats task cannot be finished #34027
- Add an option to enable/disable vector field clustering key #34097
- fix: fix error ignore in compactor #34169
- fix:load major compaction partial result #34052
- Use new stream segment reader in clustering compaction #34232
issue: #30633
pr: #33742 #33830 #33929 #33850 #34027 #34097 #34169 #34052 #34232
---------
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: Chun Han <116052805+MrPresent-Han@users.noreply.github.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-07-03 09:53:37 +08:00
wayblink
99586066f5
feat: [cherry-pick] Major compaction ( #34326 )
...
This PR cherry-picks the following commits:
fix: speed up segment lookup via channel name in datacoord (#33530 )
needed by the next commit
feat: Major compaction (#33620 )
issue: #30633
pr: #33620
---------
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: yiwangdr <80064917+yiwangdr@users.noreply.github.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
2024-07-02 18:29:01 +08:00
yihao.dai
ad4c1975bd
fix: Fix filtering by partition key fails for importing data ( #33274 ) ( #33277 )
...
Before executing the import, partition IDs should be reordered according
to partition names. Otherwise, the data might be hashed to the wrong
partition during import. This PR corrects this error.
issue: https://github.com/milvus-io/milvus/issues/33237
pr: https://github.com/milvus-io/milvus/pull/33274
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-23 11:25:40 +08:00
congqixia
e5db2f7e77
fix: [2.4] Check schema without vector field in proxy ( #33211 ) ( #33218 )
...
Cherry-pick from master
pr: #33211
Related to #33199
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-21 16:37:44 +08:00
Cai Yudong
4fc7915c70
enhance: unify data generation test APIs ( #32955 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-14 14:33:33 +08:00
wei liu
c35797c399
enhance: expose DescribeDatabase api in proxy ( #32732 )
...
issue: #32707
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-09 22:51:30 +08:00
SimFG
7da1ca9efb
enhance: check the partition num when creating collection with partition key ( #32670 )
...
issue: #30577
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-07 10:43:29 +08:00
wei liu
07720f1a95
enhance: expose alter database api in proxy ( #32639 )
...
issue: #30040
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-28 18:15:31 +08:00
SimFG
8594b55ad5
enhance: add max insert request size and must use partition key configs ( #32433 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-19 10:31:20 +08:00
Cai Yudong
00438f408f
enhance: Unify data type check APIs for go ( #31887 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-04-07 14:27:22 +08:00
cai.zhang
1f43be4a3c
enhance: Support auto index for scalar index ( #31255 )
...
issue: #29309
reopen pr : #29310
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-21 10:37:08 +08:00
Bingyi Sun
eb0cfe3e84
fix: set CollectionID field for AlterCollection request ( #30814 )
...
issue: https://github.com/milvus-io/milvus/issues/30800
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-07 14:49:02 +08:00
congqixia
9b3005f1be
enhance: Avoid create schema helper for each read task ( #30981 )
...
See also #30806
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-04 19:39:00 +08:00
Bingyi Sun
dcdf85977c
enhance: [skip e2e] add alter collection task ut ( #30447 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-02-02 15:25:04 +08:00
wayblink
1df3f90696
feat: Implement DescribeAlias and ListAliases interfaces ( #29641 )
...
#22882
/kind feature
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-01-11 19:12:51 +08:00
Xu Tong
e429965f32
Add float16 approve for multi-type part ( #28427 )
...
issue：https://github.com/milvus-io/milvus/issues/22837
Add bfloat16 vector, add the index part of float16 vector.
Signed-off-by: Writer-X <1256866856@qq.com>
2024-01-11 15:48:51 +08:00
xige-16
9702cef2b5
feat: Support multiple vector search ( #29433 )
...
issue #25639
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2024-01-08 15:34:48 +08:00
wayblink
635a7f777c
feat: add clustering key in create/describe collection ( #29506 )
...
#28410
/kind feature
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-01-07 19:56:48 +08:00
congqixia
4f8c540c77
enhance: cache collection schema attributes to reduce proxy cpu ( #29668 )
...
See also #29113
The collection schema is crucial when performing search/query but some
of the information is calculated for every request.
This PR change schema field of cached collection info into a utility
`schemaInfo` type to store some stable result, say pk field,
partitionKeyEnabled, etc. And provided field name to id map for
search/query services.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-04 17:28:46 +08:00
xige-16
0a70e8b601
enhance: Remove multiple vector field limit ( #27827 )
...
issue: https://github.com/milvus-io/milvus/issues/25639
/kind improvement
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-12-28 16:40:46 +08:00
aoiasd
a76e3b2813
Refine delete by expression for forbid proxy dml task scheduler hang ( #29340 )
...
relate: https://github.com/milvus-io/milvus/issues/29146
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-26 19:52:48 +08:00
yah01
e9ff7ed13d
enhance: remove legacy error related code ( #28385 )
...
/kind improvement
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-14 15:02:18 +08:00
yihao.dai
f9c630247d
Construct plan directly when search with vector output ( #27928 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-26 19:30:10 +08:00
yah01
be980fbc38
Refine state check ( #27541 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
jaime
7f7c71ea7d
Decoupling client and server API in types interface ( #27186 )
...
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
SimFG
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
yah01
b4f86ea55e
Construct all success status with merr ( #27226 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-20 10:57:23 +08:00
yah01
168e82ee10
Fix panic while handling with the nil status ( #27040 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-15 10:09:21 +08:00
aoiasd
e107d0794c
support complex delete expression ( #25752 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-09-12 10:19:17 +08:00
Xu Tong
9166011c4a
Add float16 vector ( #25852 )
...
Signed-off-by: Writer-X <1256866856@qq.com>
2023-09-08 10:03:16 +08:00
yah01
3349db4aa7
Refine errors to remove changes breaking design ( #26521 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00
wei liu
0bb68cac36
fix CollectionNotExist on describe rg ( #26541 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-23 09:56:21 +08:00
wei liu
f9823e039f
fix describe rg with non exist collection ( #26227 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-10 18:43:16 +08:00
Cai Yudong
73512c72fd
Remove calc_distance ( #25663 )
...
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-07-18 14:23:20 +08:00
jaime
18df2ba6fd
[Cherry-Pick] Support Database ( #24769 )
...
Support Database(#23742 )
Fix db nonexists error for FlushAll (#24222 )
Fix check collection limits fails (#24235 )
backward compatibility with empty DB name (#24317 )
Fix GetFlushAllState with DB (#24347 )
Remove db from global meta cache after drop database (#24474 )
Fix db name is empty for describe collection response (#24603 )
Add RBAC for Database API (#24653 )
Fix miss load the same name collection during recover stage (#24941 )
RBAC supports Database validation (#23609 )
Fix to list grant with db return empty (#23922 )
Optimize PrivilegeAll permission check (#23972 )
Add the default db value for the rbac request (#24307 )
Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
yah01
ebd0279d3f
Check error by Error() and NoError() for better report message ( #24736 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
xige-16
732fe54775
Support partition Key ( #24047 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-06 10:24:34 +08:00
cai.zhang
6209d5d717
Remove unused code for creating index ( #24442 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-26 16:49:26 +08:00
cai.zhang
7819a5733f
Hide dynamic field for describing collection ( #24312 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-23 14:27:25 +08:00
cai.zhang
d16e18fd34
Add output fields for search/query results ( #24302 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-23 10:19:26 +08:00
cai.zhang
98d86e2391
Return all dynamic field when retrieve json key ( #24205 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-19 09:41:25 +08:00
