1507 Commits

Author SHA1 Message Date
wayblink
da339535d5
enhance: Merge flowgraph goroutines into 1 (#28654)
/kind enhancement
#24826

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-11-23 19:52:25 +08:00
smellthemoon
73f2bab454
enhance:add some log when create client and get component states (#28160)
/kind improvement

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-22 09:12:22 +08:00
PowderLi
c238bff9fb
fix: symbol 'GetStorageMetrics' and 'enableDynamicField' (#28580)
/kind bug
to #28579 #28504

1. replace enableDynamic with enableDynamicField
2. cgo directly link to milvus_storage

Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-21 10:20:22 +08:00
Bingyi Sun
d7145e2c06
enhance: Update golangci_lint version (#28535)
Update golangci lint and fix some warnings

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-11-21 10:04:21 +08:00
PowderLi
a1c505dbd5
add internal storage metrics (#28278)
/kind improvement
issue: #28277

Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-19 17:22:25 +08:00
XuanYang-cn
40d5c902b6
Enable getting multiple segments in plan result (#28350)
Compaction plan result contained one segment for one plan. For l0
compaction would write to multiple segments, this PR expand the segments
number in plan results and refactor some names for readibility.

- Name refactory: - CompactionStateResult -> CompactionPlanResult -
CompactionResult -> CompactionSegment

See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-14 15:56:19 +08:00
smellthemoon
0aa90de141
Reduce the goroutine in flowgraph to 2 (#28233)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-13 10:50:17 +08:00
wei liu
bce1054f92
Fix retry when proxy stopped (#28264)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-09 18:58:21 +08:00
groot
3f6b203018
Fix bulkinsert bug that segments are compacted after import (#28192)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-11-07 15:14:26 +08:00
wei liu
5b45a138b1
disable auto balance when old node exists (#28191)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-07 14:02:20 +08:00
wei liu
da41a5b51e
fix check grpc error logic (#28182)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-07 11:54:18 +08:00
Xiaofan
da19e49daf
Support purge old session for standalone (#28184)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-11-06 21:21:42 +08:00
wei liu
68a86471ba
fix grpc client retry on node server not match error (#28169)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-03 23:42:16 +08:00
wayblink
00ae019ff0
Use go 1.20 csv_reader to keep milvus go=1.18 limitation (#28080)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-11-03 10:40:16 +08:00
wei liu
ecec5dfcfd
fix retry on offline node (#28079)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-03 10:14:16 +08:00
groot
abd5b199cc
Bulkinsert support pure list json (#27990)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-11-01 19:02:13 +08:00
Enwei Jiao
8ae9c947ae
Use OpenDAL to access object store (#25642)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-11-01 09:00:14 +08:00
KumaJie
e88212ba4b
Add CSV file import function (#27149)
Signed-off-by: kuma <675613722@qq.com>
Co-authored-by: kuma <675613722@qq.com>
2023-10-31 22:47:23 +08:00
yah01
9658367a3c
Refine chunk manager errors (#27590)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-31 12:18:15 +08:00
Enwei Jiao
4a33391b8f
rename createindex (#27903)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-10-27 10:12:14 +08:00
Filip Haltmayer
6b1a106a31
Moving etcd client into session (#27069)
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
2023-10-27 07:36:12 +08:00
zhagnlu
6060dd7ea8
Add chunk manager request timeout (#27692)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-10-23 20:08:08 +08:00
SimFG
9b0ecbdca7
Support to replicate the mq message (#27240)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-10-20 14:26:09 +08:00
congqixia
bcbe98aba1
Add querynode client wrapper and avoid grpc in standalone mode (#27781)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-19 11:10:07 +08:00
jaime
ac2d1bb5c2
Support receive signals from parent process (#27756)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-18 20:20:11 +08:00
yihao.dai
49b3a12804
Return newly defined merr instead of grpc unimplemented err (#27751)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-18 15:32:11 +08:00
congqixia
2f201c25e2
Remove deprecated io/ioutil usage (#27747)
`io/ioutil` package is deprecated, use `io`,`os` package replacement
also added golangci-lint rule to block future reference

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-17 20:32:09 +08:00
wei liu
7aa862c0ea
fix retry on unimplemented rpc error (#27639)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-16 10:50:09 +08:00
jaime
ec1fe3549e
Add a stop hook to clean session (#27564)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-16 10:24:10 +08:00
yah01
be980fbc38
Refine state check (#27541)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
Jiquan Long
e4f73cc805
Add host & enable_disk to session (#27507)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-10-08 20:05:31 +08:00
yah01
8394b3a1ec
Block creating new error from status reason (#27426)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-07 11:29:32 +08:00
Jiquan Long
5c1abfa2cc
Print the server id when active-standby switch (#27119)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-10-07 10:01:31 +08:00
Jiquan Long
0f14d18201
Optimize the codec code of session (#27360)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-10-01 10:33:30 +08:00
yihao.dai
106c17f304
Make read ahead policy in ChunkCache configurable (#27291)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-28 15:47:27 +08:00
SimFG
c9653b1683
Add some log and improve TestSessionProcessActiveStandBy test case (#27403)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-28 09:35:27 +08:00
XuanYang-cn
5c5f9aa05e
Enhance newDataSyncService (#27277)
- Add flowgraph.Assemble assembles nodes in flowgraph.go
- remove fgCtx in newDataSyncService
- Add newServiceWithEtcdTickler func, reduce param numbers to 3
- Remove unnecessary params
  - config.maxQueueLength, config.maxParallelish

See also: #27207

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-27 11:07:25 +08:00
yah01
6539a5ae2c
Refine DataCoord status (#27262)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-26 17:15:27 +08:00
wayblink
7dd0be1b2c
Enable bulkinsert binlog data with partitionkey (#27241)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-09-26 10:25:25 +08:00
jaime
7f7c71ea7d
Decoupling client and server API in types interface (#27186)
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
foxspy
5db4a0489e
dynamic index version control (#27335)
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-09-25 21:39:27 +08:00
XuanYang-cn
676024ff38
Add rpc to notify channel operation (#27172)
- Add NotifyChannelOperation
- Add CheckChannelOperationProgress

See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-25 14:05:27 +08:00
wei liu
9433a24f5d
fix component not exit when liveness check failed (#27236)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-09-22 19:13:25 +08:00
foxspy
370b6fde58
milvus support multi index engine (#27178)
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-09-22 09:59:26 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
congqixia
16b35e07b3
Fix TestSessionSuite/TestKeepAliveRetryActiveCancel unit test logic (#27231)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-20 18:59:23 +08:00
yah01
9baff1b81c
Rename error to match the nameing rules (#27214)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-20 10:59:24 +08:00
yah01
b4f86ea55e
Construct all success status with merr (#27226)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-20 10:57:23 +08:00
congqixia
cc9974979f
Add staticcheck linter and fix existing problems (#27174)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-19 10:05:22 +08:00
PowderLi
4feb3fa7c6
support azure (#26398)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-09-19 10:01:23 +08:00