admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-05 02:12:48 +08:00
Code Issues Packages Projects Releases Wiki Activity
23,724 Commits 136 Branches 188 Tags
Commit Graph

13 Commits

Author SHA1 Message Date
yihao.dai
5e525eb3bf
enhance: Retry reads from object storage on rate limit error (#46455) 
This PR improves the robustness of object storage operations by retrying
both explicit throttling errors (e.g. HTTP 429, SlowDown, ServerBusy).
These errors commonly occur under high concurrency and are typically
recoverable with bounded retries.

issue: https://github.com/milvus-io/milvus/issues/44772

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Configurable retry support for reads from object storage and improved
mapping of transient/rate-limit errors.
* Added a retryable reader wrapper used by CSV/JSON/Parquet/Numpy import
paths.

* **Configuration**
  * New parameter to control storage read retry attempts.

* **Tests**
* Expanded unit tests covering error mapping and retry behaviors across
storage backends.
* Standardized mock readers and test initialization to simplify test
setups.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-12-23 11:03:18 +08:00
groot
e48fe7f820
fix: Fix bulkimport bug for Struct field (#45474) 
issue: https://github.com/milvus-io/milvus/issues/45006

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2025-11-13 11:31:41 +08:00
Spade A
d8591f9548
fix: csv/json import with STRUCT adapts concatenated struct name (#45000) 
After https://github.com/milvus-io/milvus/pull/44557, the field name in
STRUCT field becomes STRUCT_NAME[FIELD_NAME]
This PR make import consider the change.

issue: https://github.com/milvus-io/milvus/issues/45006
ref: https://github.com/milvus-io/milvus/issues/42148

TODO: parquet is much more complex than csv/json, and I will leave it to
a separate PR.

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-10-24 10:22:15 +08:00
cai.zhang
d5ecb63f53
enhance: Support import geometry data by json/csv (#44826) 
issue: #44787

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2025-10-17 17:08:02 +08:00
Spade A
eb793531b9
feat: impl StructArray -- support import for CSV/JSON/PARQUET/BINLOG (#44201) 
Ref https://github.com/milvus-io/milvus/issues/42148

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
 2025-09-15 20:41:59 +08:00
groot
1ee8cea35b
enhance: bulkinsert handle nullable/defaultValue/functionOutput fields (#42956) 
issue: https://github.com/milvus-io/milvus/issues/42173

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2025-07-04 14:20:44 +08:00
yihao.dai
b4cb8a4b13
enhance: Add UTF-8 string validation for import (#40694) 
issue: https://github.com/milvus-io/milvus/issues/40684

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-04-01 19:04:21 +08:00
Xiaofan
fb48b3c7ac
fix: empty sparse row in importer (#40585) 
fix #40584

parquet bulk writer can not finish 0 dim sparse vector.

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2025-03-13 01:29:41 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990) 
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-22 23:15:58 +08:00
congqixia
b0bd290a6e
enhance: Use internal json(sonic) to replace std json lib (#37708) 
Related to #35020

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-11-18 10:46:31 +08:00
yihao.dai
b45cf2d49f
enhance: Add max length check for csv import (#37077) 
1. Add max length check for csv import.
2. Tidy import options.
3. Tidy common import util functions.

issue: https://github.com/milvus-io/milvus/issues/34150

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-10-25 14:37:29 +08:00
OxalisCu
3a381bc247
enhance: Bulkinsert supports null in csv formats (#35912) 
see details in this issue
https://github.com/milvus-io/milvus/issues/35911

---------

Signed-off-by: OxalisCu <2127298698@qq.com>
 2024-09-09 19:17:07 +08:00
OxalisCu
ed4eaffc9d
enhance: add csv support for bulkinsert (#34938) 
See this issue for details: #34937

---------

Signed-off-by: OxalisCu <2127298698@qq.com>
 2024-08-21 17:47:01 +08:00