Gao
3777eb2f68
enhance: update knowhere new version ( #33043 )
...
for milvus new release, knowhere update to latest release version
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-05-14 20:45:32 +08:00
jaime
f48a7ff8ff
enhance: use Delete instead of DeletePartialMatch to remove metrics ( #33029 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-05-14 18:49:33 +08:00
smellthemoon
89a7c34c7a
fix: exceed etcd limit ( #33041 )
...
#32974
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-05-14 18:45:32 +08:00
congqixia
dea7faf394
fix: Remove RLock re-entry in GetNumRowsOfPartition ( #33045 )
...
Resolves #33044
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-14 17:51:32 +08:00
Jiquan Long
3d85e6e028
fix: etcd txn exceeds limit due to too many fields ( #33040 )
...
fix : #33038
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-14 17:05:33 +08:00
yanliang567
ba3b2a91a0
test: Remove useless common types and refine error assert in negative cases ( #33023 )
...
Related issue: #32653
1. Remove some meaningless common types
2. Refine error assertion in negative cases
3. Remove some dup tests
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-05-14 15:03:33 +08:00
SimFG
1d48d0aeb2
enhance: use different value to get related data size according to segment type ( #33017 )
...
issue: #30436
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-14 14:59:33 +08:00
Cai Yudong
4fc7915c70
enhance: unify data generation test APIs ( #32955 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-14 14:33:33 +08:00
chyezh
96489b814d
fix: remove busy log ( #33042 )
...
issue: #32963
Signed-off-by: chyezh <chyezh@outlook.com>
2024-05-14 14:20:32 +08:00
congqixia
861977ab60
fix: Start LeaderCacheObserver before SyncAll ( #33035 )
...
Related to #33033
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-14 13:25:32 +08:00
XuanYang-cn
40e5c0be70
fix: Enable to balance released standby channel ( #32986 )
...
See also: #32879
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-14 11:51:32 +08:00
aoiasd
dc058eaf61
fix: should init access logger before init server ( #32976 )
...
relate: https://github.com/milvus-io/milvus/issues/32968
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-14 10:47:32 +08:00
zhuwenxing
3871009cf5
test: add timeout for GetMetrics grpc interface ( #33024 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-14 10:29:32 +08:00
congqixia
4ae7cabb04
fix: Remove channel when create flowgraph timeout ( #33014 )
...
See also #33013
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-14 10:07:33 +08:00
wei liu
cba2c7a3be
enhance: clean channel node info in meta store ( #32988 )
...
issue: #32910
see also: #32911
when channel exclusive mode is enabled, replica will record channel node
info in meta store, and if the balance policy changes, which means
channel exclusive mode is disabled, we should clean up the channel node
info in meta store, and stop to balance node between channels.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-14 10:05:40 +08:00
yihao.dai
a984e46a29
enhance: Remove rootcoord from datanode broker ( #32818 )
...
issue: https://github.com/milvus-io/milvus/issues/32827
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-14 10:03:32 +08:00
sre-ci-robot
09cca1fb51
[automated] Update Knowhere Commit ( #33030 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-14 01:47:32 +08:00
zhuwenxing
9e3f3d99f7
test: add timeout for import testcases ( #33006 )
...
add timeout for import testcases
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-13 19:35:32 +08:00
XuanYang-cn
efdbd8e7c1
enhance: Enable to upload by batch ( #32788 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-13 19:07:32 +08:00
XuanYang-cn
29b621f759
fix: Make compactor able to clear empty segments ( #32821 )
...
See also: #32553
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-13 18:21:32 +08:00
jaime
ba625835bc
enhance: Add metrics for segment index files size ( #32979 )
...
issue:#32980
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-05-13 17:59:32 +08:00
chyezh
293f14a8b9
fix: remove redundant replica recover ( #32985 )
...
issue: #22288
- replica recover should be only triggered by replica recover
Signed-off-by: chyezh <chyezh@outlook.com>
2024-05-13 15:25:32 +08:00
foxspy
f6777267e3
enhance: add score compute consistency config for knowhere ( #32997 )
...
issue: https://github.com/milvus-io/milvus/issues/32583
related: #32584
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-05-13 14:21:31 +08:00
SimFG
4031abd2fa
enhance: change default partition num to 16 when using partition key ( #32950 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-13 14:19:31 +08:00
congqixia
12ec3d61d9
fix: Fill deltalog entry num & time range in L0 compactions ( #33004 )
...
Resolves #33003
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-13 14:13:31 +08:00
Bingyi Sun
4724779b3b
enhance: remove fallback keys for config generator ( #32946 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-13 13:33:31 +08:00
Xiaofan
230c88b1de
enhance: improve reduce performance on very large topk ( #32871 )
...
fix #32870
reduce cpu usage for reduce on large k
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-13 11:55:31 +08:00
congqixia
ef41f80bd0
enhance: Sync go.mod version & dep to main pkg ( #32973 )
...
Related to #31293
See also #32951 #32952 #32953 #32954
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-13 10:25:32 +08:00
congqixia
0e5765b116
enhance: Utilize TestLocations ability to accelerate write & compaction ( #32948 )
...
See also #32642
This PR reuses hash locations for bloom filter prediction utilizing
`storage.Location`, like enhancement #32642 .
Also adds a utility struct in storage: `LocationCache` to storage
locations for variable K (numbers of hash functions)
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-13 10:15:32 +08:00
wei liu
8a9a42198d
fix: Proxy crash due to shard leader cache data race ( #32971 )
...
issue: #32970
cause InvalidateShardLeaderCache use wrong lock, which may cause data
race in meta cache, then proxy may crash
This PR fixed that use leaderMut when try to access shard leader cache.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-11 14:32:12 +08:00
Xiaofan
36f1ea93a5
enhance: optimize plan parser pool to avoid unnessary recycle ( #32869 )
...
fix #32868
plan parser takes too much cpu on high qps,this pr try to avoid create
lexer and parser too freequent
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-11 10:51:31 +08:00
Xiaofan
b044e5503e
enhance:Improve load speed ( #32898 )
...
fix #32897
add memory check when load collection
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-11 10:29:31 +08:00
congqixia
c0e62e6629
enhance: Use collection default consistency level for restv2 ( #32956 )
...
Set `UseDefaultConsistency` to true so that restv2 read API shall use
collection consistency level setting correctly.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-11 00:47:31 +08:00
sre-ci-robot
4577394f4f
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-05-10 12:01:00 +00:00
chyezh
1c84a1c9b6
fix: lru related issue fixup patch ( #32916 )
...
issue: #32206 , #32801
- search failure with some assertion, segment not loaded and resource
insufficient.
- segment leak when query segments
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-05-10 19:17:30 +08:00
wei liu
25689859a1
fix: Load index metric use wrong time unit ( #32935 )
...
issue:#32899
This PR fix the wrong metric value of load index, which introduced by
pr#32567, use wrong time unit for load index metrics
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-10 18:07:30 +08:00
congqixia
244d2c04f6
feat: Add milvusclient package and migrate GoSDK ( #32907 )
...
Related to #31293
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-10 18:01:30 +08:00
yiwangdr
855192eb3d
fix: sync milvus.yaml ( #32920 )
...
issue: https://github.com/milvus-io/milvus/issues/25309
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-05-10 17:29:31 +08:00
wei liu
e2332bdc17
enhance: Enable channel exclusive balance policy ( #32911 )
...
issue: #32910
* split replica's node list to channels when create replicas
* balance nodes among channels when node change happens
* implement channel level balance, let balance happens in channel level
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-10 17:27:31 +08:00
Gao
f7d29118a5
enhance: update knowhere version ( #32931 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-05-10 16:13:31 +08:00
Cai Yudong
dc89c6f810
enhance: remove duplicated data generation APIs for bulk insert test ( #32889 )
...
Issue: #22837
including following changes:
1. Add API CreateInsertData() and BuildArrayData() in
internal/util/testutil
2. Remove duplicated test APIs from importutilv2 unittest and bulk
insert integration test
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-10 15:27:31 +08:00
yanliang567
82a1e1ebe1
test: Update insert data tests and refactor some data gen functions ( #32924 )
...
related issue: #32653
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-05-10 14:57:32 +08:00
wei liu
04a8ec69f6
fix: Segment on stopping query node can't be release successfully ( #32929 )
...
issue: #32901
Cause release segment request need be send to delegator, but it need
replica to info find segment's delegator. but the stopping query node
will be marked as read only in replica, then `replica.Contains()` just
return true for rwNode in replica. then it can't get replica info by
stopping query node and release segment will be blocked.
This PR make `replica.Contains()` return true for both roNode and
rwNode.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-10 14:33:30 +08:00
aoiasd
cbbfb5b6d6
enhance: Add update milvus yaml command to makefile ( #32857 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-10 14:31:37 +08:00
Jiquan Long
0783582e2e
fix: temporarily disable two-phase retrieval when lru is enabled ( #32927 )
...
issue: #31822
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-10 14:19:45 +08:00
zhuwenxing
e776c807c4
test: add more json datatype in restful test cases ( #32930 )
...
add more json datatype in restful test cases
related to https://github.com/milvus-io/milvus/issues/31051
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-10 14:11:30 +08:00
Bingyi Sun
5cbf081111
fix: fix index resource estimation ( #32842 )
...
issue: #32820
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-10 11:53:30 +08:00
Bingyi Sun
b7ef8da360
fix: set channel checkpoint to delta position ( #32878 )
...
issue: https://github.com/milvus-io/milvus/issues/32853
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-10 11:51:30 +08:00
zhuwenxing
9a269f1489
test: add import checker to chaos test ( #32908 )
...
add import checker to chaos test
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-10 11:43:30 +08:00
Buqian Zheng
bb7765cbd6
fix: fix Indexing.Iterator ut: build index with all data at once ( #32844 )
...
issue: #32843
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-05-10 11:31:30 +08:00
