admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
19,923 Commits 137 Branches 188 Tags
Commit Graph

2849 Commits

Author SHA1 Message Date
Cai Yudong
ebd0af14f4
enhance: Handle Float16Vector/BFloat16Vector numpy bulk insert as same as BinaryVector (#33760) (#33788) 
pr: #33760
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-06-13 10:49:57 +08:00
yihao.dai
396f8608dd
fix: Fix multiple vector fields import (#33723) (#33724) 
1. Fix dim mismatch with multi-vector fields and JSON import
2. Enhance: do not display file ID in GetImportResponse.

issue: https://github.com/milvus-io/milvus/issues/33681,
https://github.com/milvus-io/milvus/issues/33682

pr: https://github.com/milvus-io/milvus/pull/33723

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-10 21:55:55 +08:00
yihao.dai
ed1dee9e38
enhance: Support L0 import (#33514) (#33712) 
issue: https://github.com/milvus-io/milvus/issues/33157

pr: https://github.com/milvus-io/milvus/pull/33514

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-08 11:17:52 +08:00
zhuwenxing
29efd69857
test: refine import test (#33691) 
pr: #33600 #33356
 2024-06-07 09:21:53 +08:00
zhuwenxing
b78d7edca6
test: add coo format sparse vector in restful test (#33689) 
pr: https://github.com/milvus-io/milvus/pull/33677

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-06-07 09:19:58 +08:00
zhuwenxing
c39d7c3f49
test: wait all tasks finished in import test (#33629) 
pr: None

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-06-05 15:51:52 +08:00
zhuwenxing
6344b788fd
test: [cherry-pick]add sparse vector datatype for import test (#33166) (#33436) 
pr: #33166

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-06-04 08:59:47 +08:00
Buqian Zheng
d6a994038c
enhance: [2.4] add sparse float vector support to restful v2 (#33555) 
issue: https://github.com/milvus-io/milvus/issues/29419
pr: #33231

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-06-03 19:31:46 +08:00
yihao.dai
8ff5d2793c
fix: Fill stats log id and check validity (#33477) (#33478) 
1. Fill log ID of stats log from import
2. Add a check to validate the log ID before writing to meta

issue: https://github.com/milvus-io/milvus/issues/33476

pr: https://github.com/milvus-io/milvus/pull/33477

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-05-31 14:13:46 +08:00
zhuwenxing
6de80a0c19
test: add different nq for restful v2 test (#33469) 
pr: https://github.com/milvus-io/milvus/pull/33456

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-05-31 13:54:07 +08:00
Cai Yudong
68e2d532d8
enhance: Cherry-pick following SparseFloatVector bulk insert PRs to Milvus2.4 (#33391) 
Cherry pick from master
pr: #33064 #33101 #33187 #33259 #33224
#33064 Support readable JSON file import for
Float16/BFloat16/SparseFloat
  #33101 Store SparseFloatVector into parquet as JSON string
  #33187 Fix SparseFloatVector data parse error for parquet
  #33259 Fix SparseFloatVector data parse error for json
  #33224 Optimize bulk insert unittest

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-05-30 10:31:45 +08:00
PowderLi
aaecc92244
fix: [cherry-pick] [restful v2] search result be cut while nq > 1 (#33363) 
issue: #33099 #32837 #32419
master pr: #33339

1. len(search result) may be nq * topk, we need return all rather than
topk
2. the  in restful response payload keep the same with milvus error code

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-05-27 00:57:45 +08:00
nico
ecdaeda6a8
enhance: update nightly groovy (#33362) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-05-24 18:43:41 +08:00
zhuwenxing
512f57e65a
test: [cherry-pick]add check for partition key for import test (#33323) 
pr:https://github.com/milvus-io/milvus/pull/33253

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-05-24 10:57:40 +08:00
yanliang567
8c9afd5f07
test: [cherry pick] Add index name tests for multiple vectors (#33315) 
Related issue: https://github.com/milvus-io/milvus/issues/32653
Related pr: #33250

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-05-23 17:17:41 +08:00
yihao.dai
ad4c1975bd
fix: Fix filtering by partition key fails for importing data (#33274) (#33277) 
Before executing the import, partition IDs should be reordered according
to partition names. Otherwise, the data might be hashed to the wrong
partition during import. This PR corrects this error.

issue: https://github.com/milvus-io/milvus/issues/33237

pr: https://github.com/milvus-io/milvus/pull/33274

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-05-23 11:25:40 +08:00
sammy.huang
7013d2f906
feat: support arm-based image build and pull request (#33225) 
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-05-21 16:54:43 +08:00
sre-ci-robot
780f3137f3
[automated] Update Pytest image changes (#33091) 
Update Pytest image changes
See changes:
9536e79363
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-05-17 15:09:36 +08:00
binbin
9536e79363
test: update pymilvus version (#33077) 
issue: #17614 
pr: #33079

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
 2024-05-16 14:51:34 +08:00
yihao.dai
6fc00e42d7
fix: Fix binlog import (#33027) 
binlog import should only import into one partition.

issue: https://github.com/milvus-io/milvus/issues/32807

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-05-14 21:45:32 +08:00
yanliang567
ba3b2a91a0
test: Remove useless common types and refine error assert in negative cases (#33023) 
Related issue: #32653 

1. Remove some meaningless common types
2. Refine error assertion in negative cases
3. Remove some dup tests

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-05-14 15:03:33 +08:00
Cai Yudong
4fc7915c70
enhance: unify data generation test APIs (#32955) 
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-05-14 14:33:33 +08:00
zhuwenxing
3871009cf5
test: add timeout for GetMetrics grpc interface (#33024) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-05-14 10:29:32 +08:00
zhuwenxing
9e3f3d99f7
test: add timeout for import testcases (#33006) 
add timeout for import testcases

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-05-13 19:35:32 +08:00
SimFG
4031abd2fa
enhance: change default partition num to 16 when using partition key (#32950) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-05-13 14:19:31 +08:00
wei liu
e2332bdc17
enhance: Enable channel exclusive balance policy (#32911) 
issue: #32910  
* split replica's node list to channels when create replicas
 * balance nodes among channels when node change happens
 * implement channel level balance, let balance happens in channel level

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-05-10 17:27:31 +08:00
Cai Yudong
dc89c6f810
enhance: remove duplicated data generation APIs for bulk insert test (#32889) 
Issue: #22837

including following changes:
1. Add API CreateInsertData() and BuildArrayData() in
internal/util/testutil
2. Remove duplicated test APIs from importutilv2 unittest and bulk
insert integration test

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-05-10 15:27:31 +08:00
yanliang567
82a1e1ebe1
test: Update insert data tests and refactor some data gen functions (#32924) 
related issue: #32653

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-05-10 14:57:32 +08:00
zhuwenxing
e776c807c4
test: add more json datatype in restful test cases (#32930) 
add more json datatype in restful test cases

related to https://github.com/milvus-io/milvus/issues/31051

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-05-10 14:11:30 +08:00
zhuwenxing
9a269f1489
test: add import checker to chaos test (#32908) 
add import checker to chaos test

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-05-10 11:43:30 +08:00
Cai Yudong
bcdbd1966e
feat: Support sparse float vector bulk insert for binlog/json/parquet (#32649) 
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-05-07 18:43:30 +08:00
yihao.dai
53874ce245
fix: Fix cannot specify partition name in binlog import (#32730) 
issue: https://github.com/milvus-io/milvus/issues/32807

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-05-07 17:19:30 +08:00
yiwangdr
b1eacb2ae8
feat: datacoord/node watch based on rpc (#32036) 
issue: https://github.com/milvus-io/milvus/issues/25309

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-05-07 15:49:30 +08:00
zhuwenxing
29f3cb692b
test:fix retry for network error (#32804) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-05-07 14:01:29 +08:00
zhuwenxing
e1eb03ad0d
test: add testcase for import without meta when enabled dynamic (#32706) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-05-07 13:59:36 +08:00
yihao.dai
4de063ae14
fix: Make the dynamic column optional in parquet import (#32738) 
issue: https://github.com/milvus-io/milvus/issues/32729

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-05-07 11:21:29 +08:00
SimFG
7da1ca9efb
enhance: check the partition num when creating collection with partition key (#32670) 
issue: #30577

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-05-07 10:43:29 +08:00
sre-ci-robot
c5191e9b28
[automated] Update Pytest image changes (#32697) 
Update Pytest image changes
See changes:
1a6293df09
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-05-06 21:03:29 +08:00
Buqian Zheng
37a99ca23e
fix: remove flaky sparse integration test (#32767) 
issue: https://github.com/milvus-io/milvus/issues/32766

this test is outdated, thus removing it instead of fixing it.

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-05-06 19:19:29 +08:00
nico
b0ebc50097
test: update test cases (#32780) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-05-06 17:41:29 +08:00
yanliang567
b9713f96fd
test: Fix a deep copy issue when getting search params (#32781) 
related issue: #32653

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-05-06 17:11:29 +08:00
yanliang567
5bb672d70d
test: Add a new range search test for all indexes and align some index params (#32724) 
related issue: https://github.com/milvus-io/milvus/issues/32653

1. align some default index params
2. add a new range search tests for all indexes and float vectors

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-04-30 17:37:27 +08:00
nico
c70c21e70b
test: modify test cases (#32727) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-04-30 17:13:31 +08:00
zhuwenxing
e346714bf2
test: add reruns params in pytest command (#32720) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-04-30 17:11:26 +08:00
congqixia
ecd8e52b53
fix: Use default integration case timeout for TestBinlogImport (#32701) 
See also #32700

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-29 19:07:27 +08:00
binbin
083bd38c77
test: add test cases for code change (#32289) 
issue: #31368

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
 2024-04-29 14:29:26 +08:00
sammy.huang
4e01591158
enhance: use soft node affinity instead of hard nodeSelector (#32677) 
issue: #32627

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-29 14:25:36 +08:00
nico
1a6293df09
test: update some test cases (#32687) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-04-29 14:23:26 +08:00
yihao.dai
1594122c0a
enhance: Make the dynamic field file optional during numpy import (#32596) 
1. Make the dynamic field file optional during numpy import
2. Add integration importing test with dynamic
3. Disallow file of pk when autoID=true during numpy import

issue: https://github.com/milvus-io/milvus/issues/32542

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-04-28 19:39:25 +08:00
congqixia
8c4fc1e61c
enhance: Close singleton etcd client in integration teardown (#32664) 
Found lots of `failed to updateTimeTick` with error `skip
ChannelTimeTickMsg from un-recognized session 1` The reason was etcd
client became singleton and used last root path in multiple cases are
run in one suite.

This PR add close singleton client invocation to fix this problem.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-28 18:17:26 +08:00