Cai Yudong
b560602885
enhance: Store SparseFloatVector into parquet as JSON string ( #33101 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-17 15:01:37 +08:00
Cai Yudong
4ef163fb70
enhance: Support readable JSON file import for Float16/BFloat16/SparseFloat ( #33064 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-16 14:47:35 +08:00
Cai Yudong
dc89c6f810
enhance: remove duplicated data generation APIs for bulk insert test ( #32889 )
...
Issue: #22837
including following changes:
1. Add API CreateInsertData() and BuildArrayData() in
internal/util/testutil
2. Remove duplicated test APIs from importutilv2 unittest and bulk
insert integration test
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-10 15:27:31 +08:00
Cai Yudong
bcdbd1966e
feat: Support sparse float vector bulk insert for binlog/json/parquet ( #32649 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-07 18:43:30 +08:00
yihao.dai
53874ce245
fix: Fix cannot specify partition name in binlog import ( #32730 )
...
issue: https://github.com/milvus-io/milvus/issues/32807
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-07 17:19:30 +08:00
yihao.dai
4de063ae14
fix: Make the dynamic column optional in parquet import ( #32738 )
...
issue: https://github.com/milvus-io/milvus/issues/32729
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-07 11:21:29 +08:00
congqixia
ecd8e52b53
fix: Use default integration case timeout for TestBinlogImport ( #32701 )
...
See also #32700
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-29 19:07:27 +08:00
yihao.dai
1594122c0a
enhance: Make the dynamic field file optional during numpy import ( #32596 )
...
1. Make the dynamic field file optional during numpy import
2. Add integration importing test with dynamic
3. Disallow file of pk when autoID=true during numpy import
issue: https://github.com/milvus-io/milvus/issues/32542
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-28 19:39:25 +08:00
Cai Yudong
5fc439c600
feat: Bulk insert support fp16/bf16 ( #32157 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-04-22 10:05:22 +08:00
yihao.dai
558feed5ed
fix: Use pk from binlog during import ( #32118 )
...
During binlog import, even if the primary key's autoID is set to true,
the primary key from the binlog should be used instead of being
reassigned.
issue: https://github.com/milvus-io/milvus/discussions/31943 ,
https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-16 14:51:20 +08:00
yihao.dai
273df98e20
enhance: Add binlog import intergration test ( #32112 )
...
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-11 10:31:18 +08:00
yihao.dai
c408a32db6
feat: Add disk quota checks for import V2 ( #31131 )
...
Return quota error when the files to be imported exceed the disk quota.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-15 14:43:03 +08:00
Bingyi Sun
5c0bb40549
fix: merge index params when creating index ( #31127 )
...
issue: https://github.com/milvus-io/milvus/issues/31102
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-11 17:31:03 +08:00
yihao.dai
a434d33e75
feat: Add import scheduler and manager ( #29367 )
...
This PR introduces novel managerial roles for importv2:
1. ImportMeta: To manage all the import tasks;
2. ImportScheduler: To process tasks and modify their states;
3. ImportChecker: To ascertain the completion of all tasks and instigate
relevant operations.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-01 18:31:02 +08:00
