admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-01-07 19:31:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
18,617 Commits 137 Branches 188 Tags
Commit Graph

1516 Commits

Author SHA1 Message Date
congqixia
14aa20b7f7
enhance: [cherry-pick] fix otel config param type & leak (#30068) 
cherry pick from master
pr: #29810 #30055 

`SampleFraction` shall be float and all `C.CString` shall be freed

Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-01-18 12:43:05 +08:00
chyezh
df9b3376dc
fix: Use determined order to lock in BlockAll to avoid deadlock (#29972) 
issue: #29104
pr: #29246

Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2024-01-15 14:32:51 +08:00
wayblink
e1446da83c
feat: [Cherry-pick] Implement DescribeAlias and ListAliases interfaces (#29896) 
#22882
pr: #29641

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-01-12 16:30:51 +08:00
cai.zhang
ba1d0556ba
fix: Import data from parquet file in streaming way (#29528) 
issue:  #29292 
master pr: #29514

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-29 11:56:45 +08:00
congqixia
85f17ada74
fix: [cherry-pick] grpc client check session skipped due to role not match (#29358) 
Cherry pick from master
pr: #29356
Related to #28815

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-21 10:12:51 +08:00
wei liu
7ef49adbd2
enhance: Use mockery to replace manual mock code (#29074) (#29182) 
issue: #29043
pr: #29074
This PR remove mannul mock code for proxy and data coord

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-14 16:44:43 +08:00
wayblink
7946c2a8dc
fix: [cherry-pick] bulkinsert binlog didn't consider ts order when processing delta data (#29176) 
issue: #29162
pr: #29163

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-14 14:38:38 +08:00
cai.zhang
bd46dc2d1b
enhance: [Pick]Support implicit type conversion for parquet (#29119) 
issue: https://github.com/milvus-io/milvus/issues/29019
master pr: #29046

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-12 16:14:43 +08:00
congqixia
0b532b42ea
enhance: [cherry-pick] Add client connect wrapper to keep connection alive (#29061) 
Cherry-pick from master
pr: #29058
See also #29057
Add wrapper to maintain client&connection
When reset operation is needed, close method shall wait until all
on-going request return

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-08 21:12:35 +08:00
zhenshan.cao
bba0e159ba
feat: [pick] Add bypassing logic for ttMsg in flowgraph of DataNode(#28756) (#29036) 
In order to minimize the CPU usage of the coroutine and avoid frequent
execution of time-consuming operations in the flowgraph when the message
stream consists solely of "ttMsg," it is recommended to implement a
mechanism for quickly bypassing the subsequent flowgraph node processing
logic.

If "ttMsg" is continuously received for a certain period of time
(coldTime), the flowgraph enters skipMode. Once in skipMode, every
skipNum "ttMsg" messages are merged into one for processing. If a
non-"ttMsg" message is received while in skipMode, the flowgraph exits
skipMode.

pr: #28756

Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
 2023-12-08 12:10:39 +08:00
MrPresent-Han
c21a1d59e4
fix: createCollection failed occasionally (#28592) (#28989) 
related: #28592
pr:#https://github.com/milvus-io/milvus/pull/28712

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-12-08 11:34:39 +08:00
cai.zhang
7f59826089
enhance: [Pick]Make import-related error message clearer (#28979) 
issue: #28976 
master pr: #28978

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-08 10:16:35 +08:00
MrPresent-Han
5f4ac437b2
enhance: [Cherry-pick] Moving etcd client into session (#27069) (#28996) 
relate: #26694
pr: https://github.com/milvus-io/milvus/pull/27069

Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
Co-authored-by: Filip Haltmayer <81822489+filip-halt@users.noreply.github.com>
 2023-12-07 16:22:34 +08:00
cai.zhang
00461145b3
enhance: [Pick]Support importing data with parquet file (#28820) 
issue: https://github.com/milvus-io/milvus/issues/28272
master pr: #28608 

Numpy does not support array type import.
Array type data is imported through parquet.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-29 22:16:29 +08:00
cai.zhang
fda452ea4d
enhance: [Pick] Revert import support csv format (#28766) 
issue: #28778 
master pr: #28760

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-28 18:26:28 +08:00
wayblink
1ed92da414
enhance:[cherry-pick] Merge flowgraph goroutines into 1 (#28728) 
see also: #28654 #28233

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-11-27 21:04:27 +08:00
cai.zhang
ef115dd7df
enhance: Support Array DataType for bulk_insert (#28578) 
issue: https://github.com/milvus-io/milvus/issues/28272
master pr: #28341 
Support array DataType for bulk_insert with json, binlog files.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-27 16:56:27 +08:00
smellthemoon
288844e3cf
enhance: Reduce the goroutine in flowgraph to 2 (#28233) (#28545) 
each node in flow graph alloc a goroutine, but it is actually executed
sequentially and can be placed in one goroutine. InputNode will consume
msg form msgstream, alloc one goroutine.
issue: #24826 
pr: #28233

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-11-24 15:00:26 +08:00
yah01
c6ba4fa102
enhance: enhance the accuracy of memory usage (#28554) (#28559) 
before this, Milvus use container/system's memory info to get the memory
usage, which could be inaccurate.

we allocates the memory by private anon mmap,
then rss - shared would be the accurate memory usage

resolve https://github.com/milvus-io/milvus/issues/28553
pr: #28554

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-24 14:12:25 +08:00
PowderLi
92e5f96064
fix: symbol 'GetStorageMetrics' and 'enableDynamicField' (#28581) 
issue: #28579 #28504
pr: #28580

1. replace enableDynamic with enableDynamicField
2. cgo directly link to milvus_storage

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-21 15:30:23 +08:00
PowderLi
1383680b3a
add internal storage metrics (#28279) 
/kind improvement
issue: #28277

pr: #28278

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-19 17:22:25 +08:00
KumaJie
cbd3e100b8
[cherry-pick]Add CSV file import function (#28161) 
issue: #27148

from pr: #27149

Signed-off-by: kuma <675613722@qq.com>
Co-authored-by: kuma <675613722@qq.com>
 2023-11-15 18:32:20 +08:00
wei liu
16dc26833b
Fix retry when proxy stopped (#28263) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-09 10:10:19 +08:00
groot
29e66ed46b
Fix bulkinsert bug that segments are compacted after import (#28227) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-11-08 10:18:20 +08:00
wei liu
918333817e
Disable auto balance when old node exists (#28191) (#28224) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-08 07:10:17 +08:00
wei liu
62e585d8db
fix check grpc error logic (#28182) (#28218) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-07 14:22:16 +08:00
Xiaofan
bbcaf7a703
Fix coordinator fast restart (#28205) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-11-07 10:04:16 +08:00
wei liu
e8950a529e
fix grpc client retry on node server not match error (#28169) (#28177) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-07 00:44:17 +08:00
groot
f11e99efed
Bulkinsert support pure list json (#28126) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-11-03 16:02:24 +08:00
wei liu
4558af94d5
fix retry on offline node (#28079) (#28139) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-03 16:00:17 +08:00
zhagnlu
6060dd7ea8
Add chunk manager request timeout (#27692) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-10-23 20:08:08 +08:00
SimFG
9b0ecbdca7
Support to replicate the mq message (#27240) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-10-20 14:26:09 +08:00
congqixia
bcbe98aba1
Add querynode client wrapper and avoid grpc in standalone mode (#27781) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-19 11:10:07 +08:00
jaime
ac2d1bb5c2
Support receive signals from parent process (#27756) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-18 20:20:11 +08:00
yihao.dai
49b3a12804
Return newly defined merr instead of grpc unimplemented err (#27751) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-18 15:32:11 +08:00
congqixia
2f201c25e2
Remove deprecated io/ioutil usage (#27747) 
`io/ioutil` package is deprecated, use `io`,`os` package replacement
also added golangci-lint rule to block future reference

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: guoguangwu <guoguangwu@magic-shield.com>
 2023-10-17 20:32:09 +08:00
wei liu
7aa862c0ea
fix retry on unimplemented rpc error (#27639) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-16 10:50:09 +08:00
jaime
ec1fe3549e
Add a stop hook to clean session (#27564) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-16 10:24:10 +08:00
yah01
be980fbc38
Refine state check (#27541) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-11 21:01:35 +08:00
Jiquan Long
e4f73cc805
Add host & enable_disk to session (#27507) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-10-08 20:05:31 +08:00
yah01
8394b3a1ec
Block creating new error from status reason (#27426) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-07 11:29:32 +08:00
Jiquan Long
5c1abfa2cc
Print the server id when active-standby switch (#27119) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-10-07 10:01:31 +08:00
Jiquan Long
0f14d18201
Optimize the codec code of session (#27360) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-10-01 10:33:30 +08:00
yihao.dai
106c17f304
Make read ahead policy in ChunkCache configurable (#27291) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-28 15:47:27 +08:00
SimFG
c9653b1683
Add some log and improve TestSessionProcessActiveStandBy test case (#27403) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-28 09:35:27 +08:00
XuanYang-cn
5c5f9aa05e
Enhance newDataSyncService (#27277) 
- Add flowgraph.Assemble assembles nodes in flowgraph.go
- remove fgCtx in newDataSyncService
- Add newServiceWithEtcdTickler func, reduce param numbers to 3
- Remove unnecessary params
  - config.maxQueueLength, config.maxParallelish

See also: #27207

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-09-27 11:07:25 +08:00
yah01
6539a5ae2c
Refine DataCoord status (#27262) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-26 17:15:27 +08:00
wayblink
7dd0be1b2c
Enable bulkinsert binlog data with partitionkey (#27241) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-09-26 10:25:25 +08:00
jaime
7f7c71ea7d
Decoupling client and server API in types interface (#27186) 
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-09-26 09:57:25 +08:00
foxspy
5db4a0489e
dynamic index version control (#27335) 
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-25 21:39:27 +08:00