381 Commits

Author SHA1 Message Date
Bingyi Sun
86728490a2
Fix partition not found (#17132)
issue: #16842

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-05-20 19:37:57 +08:00
yah01
7746a5b742
Add NodeIds field for QuerySegmentInfo (#17121)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-20 18:03:58 +08:00
congqixia
599763d9bf
Fix replicas info is not removed after release (#17111)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-19 20:05:57 +08:00
yah01
dcfe472586
Fix LoadBalance doesn't save the modification to replicas' shards (#17064)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-19 16:51:57 +08:00
bigsheeper
9eeec4a2d5
Add collection load cache and InvalidateCollMetaCache by collID (#16882)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-19 10:13:56 +08:00
yah01
33c855dcd2
Fix LoadBalance doesn't remove the source nodes from segment (#17051)
If the triggerCondition isn't NodeDown, the removing won't happen.

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-18 11:55:56 +08:00
yah01
960d35e517
Add MockCluster, make unit tests reliable (#17032)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-17 12:57:56 +08:00
Bingyi Sun
59bc0a7000
Add some log. (#16978)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-05-16 17:15:55 +08:00
yah01
e38c6f6c44
Fix load the same segments multiple times for manual LoadBalance (#16921)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-16 15:41:56 +08:00
yah01
a382133a8a
Add new node into the replica which has the most of offline nodes (#16907)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-13 18:31:54 +08:00
congqixia
ae717bf991
Fix channelUnsubscribe data race and logic (#16946)
- Add a RWMutex for container/list which is not goroutine-safe
- Fix the element in list is never removed

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-12 18:09:53 +08:00
congqixia
5c98329f7c
Return error when no replica available (#16886)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-10 19:47:53 +08:00
Xiaofan
000c5ff3de
Fix msgstream unsubscription (#16883)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-10 19:43:52 +08:00
yah01
2d0f908dba
Fix updating segments' NodeIds correctly after LoadBalance (#16854)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-10 15:47:53 +08:00
Xiaofan
62658dcda6
Fix pulsar unscubsribe fail because of consumer not found (#16839)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-09 12:07:52 +08:00
Xiaofan
92b6293be4
Fix QueryNode log level (#16604)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-07 10:27:51 +08:00
Ten Thousand Leaves
1acd256481
Add DataQueryable and DataIndexed states for bulk load tasks (#16725)
issue: #16607

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-05 21:17:50 +08:00
yah01
c82e2453eb
Modify the replicas' shard info after load balance (#16785)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-05 21:15:50 +08:00
congqixia
8c004fa981
Fix replicaID is allow zero from watch channel task (#16788)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-05 18:11:51 +08:00
yah01
20936f78da
Fix LoadBalance failed to sync segments to shard leader when the leader is offline (#16770)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-05 16:25:50 +08:00
yah01
7dfab0aa13
Add unit test for case of failed to sync segments to shard leader (#16712)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-29 15:51:49 +08:00
yah01
93416d5524
Fix LoadBalance loads the same segment multiple times (#16734)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-29 15:19:47 +08:00
yah01
415a0e5978
Fix miss segments/DmChannel when query by nodeID (#16719)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-29 14:11:47 +08:00
xige-16
515d0369de
Support string type in segcore (#16546)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
2022-04-29 13:35:49 +08:00
jaime
68b1b82faf
Remove DataKV interface (#16692)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-28 21:03:47 +08:00
yah01
206b554d49
Sync the distribution of segments to shard leaders (#16685)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-28 16:01:48 +08:00
yah01
7017756601
Validate the number of replicas for load collection/partitions request (#16697)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-28 10:54:00 +08:00
yah01
bb6cd4b484
Disable the unit test for the case of filter out unavailable leaders of GetShardLeaders (#16688)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-27 22:15:47 +08:00
yah01
68ef720d6b
Filter out the unavailable shard leaders (#16677)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-27 16:27:46 +08:00
congqixia
0bf176a7e1
Add SyncReplicaSegments interface for QueryNode (#16672)
Add SyncReplicaSegments for LoadCollection/LoadPartition child task
This API notifies related ShardCluster to record segment Loaded
Preventing the Etcd events is delayed, which will cause some segment not search

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-27 10:41:46 +08:00
yah01
60f7fef3b3
Remove offline nodes from replica info after LoadBalanceTask done (#16653)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-26 18:23:46 +08:00
Xiaofan
c1ff0cec8f
Change QueryCoord Log level (#16590)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-26 11:29:54 +08:00
Bingyi Sun
1682dc49fc
Fix unit test timeout (#16537)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-04-25 14:49:46 +08:00
Ten Thousand Leaves
4ef2df8cb9
Fix an issue where query coord calls MinioChunkManager during init (#16629)
It causes Milvus not runnable with local storage.

issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-25 11:11:46 +08:00
Xiaofan
89b4a34892
Fix Data race in NodeID (#16603)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-24 22:03:44 +08:00
yah01
5c66300408
Always add the shard leader into the nodes list of shard (#16616)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-24 17:37:46 +08:00
yah01
b82f6a1a0a
Fix GetReplicas() returns all nodes in ShardReplicas (#16597)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-24 11:35:45 +08:00
congqixia
69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470)
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-20 16:15:41 +08:00
jaime
ed3c29eae7
Support Kafka (#16149)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-12 19:47:33 +08:00
godchen
bb7a0766fe
Add dependency factory (#16204)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-04-07 22:05:32 +08:00
XuanYang-cn
78200009a3
Impl GetReplicas in Proxy (#16390)
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-06 14:57:31 +08:00
Letian Jiang
028c5cb882
Modify grpc interface for replica Search/Query in QueryNode (#16326)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-04-02 14:15:31 +08:00
yah01
c005f07ccc
Modify proto to add GetReplicas() interface (#16328)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-01 16:15:29 +08:00
XuanYang-cn
58addbf9f4
Add GetShardLeaders rpc in querycoord proto (#16299)
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-31 16:39:29 +08:00
Bingyi Sun
baa745b83e
Improve log (#16307)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-31 11:33:28 +08:00
Bingyi Sun
40fab91448
improve log (#16236)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-30 23:05:28 +08:00
Jiquan Long
ba37531456
Add support for loading multiple indexes (#16138)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-30 21:11:28 +08:00
congqixia
18a3e9f265
Add grpc interface for replica Search/Query in QueryNode (#16197)
Resolves #16195
Add Search and Query grpc interface in query proto and types/types.go

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-30 12:03:27 +08:00
yah01
178eb8cce1
Modify proto to support replicas (#16218)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-28 16:43:27 +08:00
Bingyi Sun
e0ffe54547
Add interface to get replications by collection ID (#16213)
issue: #16158
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-28 14:35:27 +08:00