admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
20,792 Commits 137 Branches 188 Tags
Commit Graph

52 Commits

Author SHA1 Message Date
yihao.dai
86ae07d705
enhance: [2.4] Limit the maximum number of segments restored and fail the job if saving the binlog fails (#39473) 
1. Limit the maximum number of restored segments to 1024.
2. Fail the import job if saving binlog fails.
3. Fail the import job if saving the import task fails to prevent
repeatedly generating dirty importing segments.
4. Update proto.

issue: https://github.com/milvus-io/milvus/issues/39331

pr: https://github.com/milvus-io/milvus/pull/39344

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-01-21 15:55:05 +08:00
Xianhui Lin
6111a0dc22
enhance: [2.4]Add schema update time verification for insert and upsert to use cache (#39405) 
enhance: Add schema update time verification for insert and upsert to
use cache
issue: https://github.com/milvus-io/milvus/issues/39093
Related to pr: https://github.com/milvus-io/milvus/pull/39096

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
 2025-01-21 10:19:11 +08:00
SimFG
df73f93126
enhance: [2.4] pick some master improvements to 2.4 branch (#38128) 
- issue: #38127

master pr list:
- #37759
- #37835
- #37845
- #37874
- #37894
- #37969
- #37983
- #38005
- #38035

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-12-13 10:24:45 +08:00
jaime
b92daa1532
fix: iaccurate size estimation for encoded array data (#36379) 
issue: #36029
pr: #36373

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-09-23 21:17:13 +08:00
congqixia
555b7a6aa7
enhance: [2.4] Remove duplicated schema helper creation in proxy (#35489) (#35502) 
Cherry-pick from master
pr: #35489
Related to PRs of #35415

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-16 14:32:54 +08:00
smellthemoon
23052f6ac2
fix: upsert use the previous pk in insert when autoid(#34672) (#35130) 
issue: #34668 
pr: #34672

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-08-12 18:58:25 +08:00
Jiquan Long
34873a7f76
feat: record the duration waiting in the proxy queue (#34744) (#34788) 
fix: https://github.com/milvus-io/milvus/issues/34743
pr: #34744

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-07-19 15:37:44 +08:00
smellthemoon
0fdb288de7
enhance: upsert support autoid(#30342) (#34633) 
pr: #30342 
issue: #29258

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
 2024-07-15 20:53:39 +08:00
aoiasd
9087b6f42e
enhance: [Cherry-Pick] support mark error as user error (#33498) (#34396) 
relate: https://github.com/milvus-io/milvus/issues/33492
pr: https://github.com/milvus-io/milvus/pull/33498
---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-07-04 10:08:10 +08:00
SimFG
8594b55ad5
enhance: add max insert request size and must use partition key configs (#32433) 
issue: https://github.com/milvus-io/milvus/issues/30577
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-19 10:31:20 +08:00
cai.zhang
40ca98f57f
enhance: Skip timestamp allocation when search/query consistency level is eventually (#29773) 
issue: #29772 
1. Skip timestamp allocation when search/query consistency level is
eventually.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-02-21 09:52:59 +08:00
congqixia
4f8c540c77
enhance: cache collection schema attributes to reduce proxy cpu (#29668) 
See also #29113

The collection schema is crucial when performing search/query but some
of the information is calculated for every request.

This PR change schema field of cached collection info into a utility
`schemaInfo` type to store some stable result, say pk field,
partitionKeyEnabled, etc. And provided field name to id map for
search/query services.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-01-04 17:28:46 +08:00
yah01
be980fbc38
Refine state check (#27541) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-11 21:01:35 +08:00
yah01
6539a5ae2c
Refine DataCoord status (#27262) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-26 17:15:27 +08:00
cai.zhang
a362bb1457
Support array datatype (#26369) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-09-19 14:23:23 +08:00
yah01
3349db4aa7
Refine errors to remove changes breaking design (#26521) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-04 09:57:09 +08:00
smellthemoon
87ecaac703
Add dynamic schema check in upsert (#26644) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-08-30 10:52:26 +08:00
MrPresent-Han
d30a920226
add log trace for segcore(#26277) (#26339) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-08-16 11:41:33 +08:00
SimFG
69d274d233
Improve the operation log (#25589) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-07-14 16:08:31 +08:00
Enwei Jiao
66fdc71479
Refactor logs in DataCoord & DataNode (#25574) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-07-14 15:56:31 +08:00
jaime
18df2ba6fd
[Cherry-Pick] Support Database (#24769) 
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-06-25 17:20:43 +08:00
Enwei Jiao
d143682d7d
Refactor logs in proxy package. (#24936) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-06-19 13:28:41 +08:00
yihao.dai
b62429070c
Set pchannels before dml enqueue to prevent panic (#24828) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-16 16:36:40 +08:00
smellthemoon
db31e88a73
Add length check when insert and upsert (#24759) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-06-15 10:24:38 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-09 01:28:37 +08:00
xige-16
732fe54775
Support partition Key (#24047) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-06 10:24:34 +08:00
cai.zhang
7819a5733f
Hide dynamic field for describing collection (#24312) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-05-23 14:27:25 +08:00
cai.zhang
008285f849
Support dynamic schema for create collection (#24176) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-05-18 09:33:24 +08:00
Jiquan Long
6965495b9d
Check overflow for inserted integer (#24142) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-05-16 20:19:22 +08:00
yihao.dai
0b354cbab1
Fix timetick block caused by dml task pop failed (#23277) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-11 11:00:31 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
zhenshan.cao
1287ca699a
Refine usage of TimeRecorder.Record (#23142) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-03-30 18:56:22 +08:00
smellthemoon
b1992e84cc
Delete dup check (#23076) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-03-29 10:06:01 +08:00
Jiquan Long
0be96ab610
Check if all columns aligned with same num_rows (#22968) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-03-24 17:09:58 +08:00
Jiquan Long
dff15c3488
Check dimension of inserted records (#22819) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-03-17 17:33:58 +08:00
MrPresent-Han
89618b7e12
fix wrong sequence of timetick stats on proxy #21855 (#22685) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2023-03-14 15:43:54 +08:00
groot
6f6bd98c27
Verify vector float data for bulkinsert and insert (#22728) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-03-14 14:03:58 +08:00
smellthemoon
ad6cbc990f
Combine similar function (#21752) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-01-17 17:53:42 +08:00
Enwei Jiao
fb42466c65
Use opentelemetry (#21509) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-12 16:09:39 +08:00
smellthemoon
bf3c02155a
Support update (#20875) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-01-04 17:21:36 +08:00
smellthemoon
18cad3a1fb
Optimization of delete and insert (#20990) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-12-08 18:37:19 +08:00
Enwei Jiao
89b810a4db
Refactor all params into ParamItem (#20987) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-07 18:01:19 +08:00
Jiquan Long
7f76e798e9
Fix metric ProxyApplyPrimaryKeyLatency (#20486) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-29 15:23:14 +08:00
Enwei Jiao
c05b9ad539
Add event dispatcher for config (#20393) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-17 18:59:09 +08:00
smellthemoon
7325b3e1c3
Substitute traceid for msgid in rpc (#20450) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-11-14 15:29:06 +08:00
Jiquan Long
2189530f5b
Fix some unreasonable logs in Proxy (#20411) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-09 15:05:08 +08:00
Enwei Jiao
956c5e1b9d
Make Params singleton (#20088) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-04 14:25:38 +08:00
Xiaofan
7d2ba60412
If autoID enabled, check to make sure user didn't specify autoID field (#19943) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-21 19:03:27 +08:00
smellthemoon
108e51b2f0
[test]Create index before load and fix error message (#19874) (#19857) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Co-authored-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2022-10-19 10:01:26 +08:00
SimFG
a55f739608
Separate public proto files (#19782) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-16 20:49:27 +08:00