admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
18,496 Commits 137 Branches 188 Tags
Commit Graph

275 Commits

Author SHA1 Message Date
SimFG
74e72ce27e
enhance: [2.3] Support to get the param value in the runtime (#29298) 
pr: #29297
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-12-21 20:36:43 +08:00
congqixia
9050b236e4
fix: [cherry-pick] delegator may mark segment offline by mistake (#29344) 
cherry-pick from master
pr: #29343
See also #29332

The segment may be released before or during the request when delegator
tries to forward delete request to yet. Currently, these two situation
returns different error code.

In this particular case, ErrSegmentNotLoaded and ErrSegmentNotFound
shall both be ignored preventing return search service unavailable by
mistake.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-20 21:16:41 +08:00
yah01
cd9e3c4837
fix: creating growing segments may introduce many threads (#29314) 
many growing segments may be created in a short time and there is no
restriction to the process, the CGO call will leave many threads

related: https://github.com/milvus-io/milvus/issues/29282
pr: #29306

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-19 18:02:40 +08:00
wei liu
9092b1ae8a
feat: enable balance based on growing segment row count (#28623) (#29184) 
issue: #28622 
pr: #28623
query node with delegator will has more rows than other query node due
to delgator loads all growing rows.
This PR enable the balance segment which based on the num of growing
rows in leader view.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-14 15:26:37 +08:00
MrPresent-Han
5f4ac437b2
enhance: [Cherry-pick] Moving etcd client into session (#27069) (#28996) 
relate: #26694
pr: https://github.com/milvus-io/milvus/pull/27069

Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
Co-authored-by: Filip Haltmayer <81822489+filip-halt@users.noreply.github.com>
 2023-12-07 16:22:34 +08:00
aoiasd
8502037cff
fix: [Cherry-pick] sync action load segment with lack collection index info list (#28956) 
relate: https://github.com/milvus-io/milvus/issues/28779
https://github.com/milvus-io/milvus/issues/28637
pr: https://github.com/milvus-io/milvus/pull/28788

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-07 14:14:42 +08:00
cqy123456
8fd38c8eea
enhance:[cherry-pick] Use binlog index for better search performance (#29012) 
this pr is cherry-pick from master:
pr: https://github.com/milvus-io/milvus/pull/28528
pr: https://github.com/milvus-io/milvus/pull/27673
related issue:
issue: https://github.com/milvus-io/milvus/issues/27678

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-12-07 09:52:34 +08:00
congqixia
63e1ac0846
fix: [cherry-pick] schema->size() check logic with system field (#28802) (#28841) 
Cherry pick from master
pr: #28802

Now segcore load system field info as well, the growing segment
assertion shall not pass with "+ 2" value
This will cause all growing segments load failure
Fix #28801
Related to #28478
See also #28524

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-01 13:28:30 +08:00
Gao
ccca932cc6
fix: [2.3] correct autoindex segment num (#28429) 
issue: #28386 
pr: #28387

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-11-28 19:24:26 +08:00
congqixia
5a962a631a
fix: [cherry-pick] Change schema to atomic.Pointer to avoid data race (#28739) (#28759) 
Cherry-pick from master
pr: #28739
See also #28738

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-27 19:50:27 +08:00
yihao.dai
8520ee7552
enhance: Print nq (#28507) (#28639) 
Log nq in search path.

pr: https://github.com/milvus-io/milvus/pull/28507

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-11-27 10:18:26 +08:00
yah01
a1b861ed7a
enhance: improve load speed (#28518) (#28719) 
This check rejects load request if running out the pool workers, but
small segment would be loaded soon, another segments would been loading
again after a check interval, which leads to slow loading for collection

Block the request by go pool

pr: #28518

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-26 22:10:26 +08:00
congqixia
e4ea148c0f
fix: [cherry-pick] Add IndexList check for load segment request (#28601) (#28700) 
Cherry-pick from master
pr: #28601
See also #28022 #28034
The load segment may reaches before watch dml channel, so the index meta
may be empty as well

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-24 15:02:39 +08:00
congqixia
0fbd610e89
fix: [cherry-pick] querynodev2 local worker failed to maintain collection ref (#28631) 
Cherry-pick from master
pr: #28590 #28598
See also #28589 #28596 
Increase ref for collection during load and unref after load completed.
Use the same logic protection from services.go `LoadSegments`
Perform `Unref` after release sealed segments

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-22 10:04:23 +08:00
SimFG
598788e6b8
Delay the cancellation of ctx when stopping the node (#28249) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-11-08 01:46:20 +08:00
yah01
d10a82dba4
Fix getting incorrect CPU num (#28178) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-11-07 11:52:22 +08:00
wei liu
87e8d04ed7
fix sync distribution with wrong version (#28130) (#28170) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-06 11:38:18 +08:00
yah01
5c444218a2
Limit max thread num for pool (#28018) (#28115) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-06 10:50:17 +08:00
yah01
0ab13c935a
Fix QueryNode panic while upgrading (#28034) (#28114) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-03 17:24:18 +08:00
wei liu
4558af94d5
fix retry on offline node (#28079) (#28139) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-03 16:00:17 +08:00
congqixia
994bb6991b
Refine offline segments logic in shard delegator (#28073) (#28084) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-01 23:18:17 +08:00
congqixia
f492e33343
Refine log level when request resource fail for loading segments (#28004) (#28077) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-01 23:16:26 +08:00
congqixia
3c716717b6
Add ctx parameter for tsafe pkg & NewDelegator method (#27877) (#28075) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-01 23:14:35 +08:00
wei liu
8041fc3c75
avoid add empty growing segment to delegator distribution (#27930) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-26 10:10:10 +08:00
Gao
c8415453a1
Support prepare config for index (#27920) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-10-25 19:54:12 +08:00
aoiasd
9ba79a97e4
Add some log for delete task when it need query (#27317) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-10-24 14:54:12 +08:00
wei liu
eadd48ef7c
fix grpc error judge logic (#27800) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-20 04:28:09 +08:00
MrPresent-Han
af61a80f73
connect traceID for retrieve (#27804) 
/kind improvement

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-10-19 19:22:10 +08:00
zhenshan.cao
020ad9a6bc
Rectify wrong exception messages associated with Array datatype (#27769) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-10-19 17:24:07 +08:00
congqixia
bcbe98aba1
Add querynode client wrapper and avoid grpc in standalone mode (#27781) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-19 11:10:07 +08:00
Enwei Jiao
e98e56f75d
Fix SIGSEGV if run within gdb (#27736) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-10-18 02:16:11 +08:00
congqixia
1f2a76d04d
Handle legacy querynode Delete Unimplemented (#27749) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-17 19:06:06 +08:00
jaime
ec1fe3549e
Add a stop hook to clean session (#27564) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-16 10:24:10 +08:00
yah01
50672f0d60
Skip empty segment while query/search (#27711) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-15 16:10:04 +08:00
yihao.dai
d9431266bb
Fix panic at loadSegment during rolling upgrade (#27671) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-13 16:31:35 +08:00
yah01
be980fbc38
Refine state check (#27541) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-11 21:01:35 +08:00
wei liu
0695c88844
refine_log_of_update_target_version (#27572) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-11 06:51:33 +08:00
yihao.dai
7d0dd0047d
Use path.Join when init ChunkCache path (#27433) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-30 17:05:27 +08:00
zhenshan.cao
dbdb9e15d8
Update Knowhere version (#27445) 
Signed-off-by: Li Liu <li.liu@zilliz.com>
Co-authored-by: Li Liu <li.liu@zilliz.com>
 2023-09-29 14:23:28 +08:00
Jiquan Long
370fdaf50d
Record engine version for segment index (#27384) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-28 18:03:28 +08:00
congqixia
6ff4ffa6c2
Fix scheduler may panic during close (#27431) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-28 16:53:26 +08:00
yihao.dai
106c17f304
Make read ahead policy in ChunkCache configurable (#27291) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-28 15:47:27 +08:00
congqixia
258e1ccd66
Refine querynode scheduler lifetime (#26915) 
This PR refines scheduler lifetime control:
- Move private tri-state into lifetime package
- Make scheduler block incoming "Add" task
- Make scheduler Stop wait until all previously accepted task done

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-28 10:21:26 +08:00
congqixia
8c59dba329
Refine queryHook mockery (#27394) 
This PR move `QueryHook` interface to `optimizers` pkg
Update all mockery generated files to latest
Add makefile entry for `QueryHook`

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-28 10:01:26 +08:00
yah01
2d6a968233
Refine QueryNode errors (#27380) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-26 22:09:28 +08:00
yah01
6539a5ae2c
Refine DataCoord status (#27262) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-26 17:15:27 +08:00
cai.zhang
30a6c6072e
Fix bug for loading array data panic (#27343) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-09-26 12:23:26 +08:00
jaime
7f7c71ea7d
Decoupling client and server API in types interface (#27186) 
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-09-26 09:57:25 +08:00
foxspy
5db4a0489e
dynamic index version control (#27335) 
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-25 21:39:27 +08:00
xige-16
fa06265756
Fix cgo pointer param error (#27313) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-09-22 18:49:26 +08:00