admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2025-12-08 01:58:34 +08:00
Code Issues Packages Projects Releases Wiki Activity
18,538 Commits 133 Branches 183 Tags
Commit Graph

875 Commits

Author SHA1 Message Date
smellthemoon
adce6aab4f
fix:handle err when ManualCompaction (#28804) 
handle err when ManualCompaction
#28644

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-11-30 17:06:31 +08:00
wayblink
b592f08db3
enhance: Use smaller lock in datacoord.meta.UpdateChannelCheckpoint (#28753) 
/kind enhancement

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-11-30 11:00:28 +08:00
congqixia
3f9da2b8a0
fix: Check whether compaction plan exist before update (#28826) 
Fix: #28825

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-29 18:54:27 +08:00
XuanYang-cn
321c5c32e3
fix: Separate schedule and check results loop (#28692) 
This PR:

- Separates compaction scheduler and check results loop So that slow in
check-loop doesn't influence execution.

- Cleans compaction tasks when drop a vchannel so dropped-channel's
compaction tasks won't be checked over and over again.

  - Skips meta change when meta's already changed, avoid panic
  - Remove not inuse injectDone(bool) parameter

See also: #28628, #28209

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-29 10:50:29 +08:00
XuanYang-cn
606ec77b66
enhance: Unify levelzero segment config in DN (#28720) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-28 18:04:26 +08:00
congqixia
5025c10e8b
enhance: Remove channel cp lag metrics when DropChannel (#28784) 
See also #28765
Remove metric when Drop channel grpc execute succeed

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-28 14:24:25 +08:00
congqixia
dd682658da
enhance: Add channel checkpoint ts lag metric (#28767) 
See also #28765

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-28 07:00:28 +08:00
jaime
b1e0a27f31
enhance: Add logs for each step during service initialization (#28624) 
/kind improvement

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-11-27 16:30:26 +08:00
aoiasd
8a4cfb7d6a
enhance: add l0 metric and fix datacoord no need drop l0 segment when flush (#28373) 
relate: https://github.com/milvus-io/milvus/issues/27675

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-24 15:58:24 +08:00
smellthemoon
79c0edb1d8
enhance:Remove msgbase unnecessary assignments (#28511) 
remove some unnecessary assignments, for the reason that
commonpbutil.NewMsgBase has default value.

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-11-24 15:02:39 +08:00
congqixia
a2fe9dad49
enhance: Make etcd kv request timeout configurable (#28661) 
See also #28660
This pr add request timeout config item for etcd kv request timeout
 Sync the default timeout value to same value for etcdKV & tikv config

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-23 19:34:23 +08:00
XuanYang-cn
9b371067d2
feat: Add Compaction views and triggers (#27906) 
- Add Compaction l0 views
- Add Compaction scheduler
- Add Compaction triggerv2
- Add Compaction view manager

See also: #27606

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-23 17:30:25 +08:00
Bingyi Sun
4fedff6d47
feat: integrate storage v2 into the write path (#28440) 
#28378

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-11-23 17:26:24 +08:00
cai.zhang
31f442915b
fix: Fix bug for parsing expression that include quotes (#28416) 
issue: #28365 
Fix bug for parsing error when a string enclosed in single quotes in an
expression contains multiple double quotes.
such as:
```
expr = "tag == '\"blue\"'"
```

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-23 17:18:32 +08:00
smellthemoon
73f2bab454
enhance:add some log when create client and get component states (#28160) 
/kind improvement

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-11-22 09:12:22 +08:00
yah01
cc952e0486
enhance: optimize forwarding level0 deletions by respecting partition (#28456) 
- Cache the level 0 deletions after loading level0 segments
- Divide the level 0 deletions by partition
related: #27349

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-21 18:24:22 +08:00
aoiasd
b1fbc1f4e7
fix: datacoord l0 segment bugs (#28490) 
relate: https://github.com/milvus-io/milvus/issues/27675

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-21 16:48:22 +08:00
Enwei Jiao
7445d3711c
feat: trigger compaction to handle index version (#28442) 
issue: https://github.com/milvus-io/milvus/issues/28441

---------

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-11-21 09:26:22 +08:00
wei liu
fdbca10e23
fix: Fix channel manager bg checker exit when disable auto balance (#28459) 
issue: #28454

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-20 18:20:22 +08:00
aoiasd
13a5b9f64a
fix: query l0 segment bugs (#28558) 
relate: https://github.com/milvus-io/milvus/issues/27675

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-20 17:26:23 +08:00
PowderLi
a1c505dbd5
add internal storage metrics (#28278) 
/kind improvement
issue: #28277

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-19 17:22:25 +08:00
XuanYang-cn
0be22ccacd
Fix compaction unable to notify datanode (#28409) 
See also: #28214, #28368

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-14 16:38:18 +08:00
XuanYang-cn
40d5c902b6
Enable getting multiple segments in plan result (#28350) 
Compaction plan result contained one segment for one plan. For l0
compaction would write to multiple segments, this PR expand the segments
number in plan results and refactor some names for readibility.

- Name refactory: - CompactionStateResult -> CompactionPlanResult -
CompactionResult -> CompactionSegment

See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-14 15:56:19 +08:00
XuanYang-cn
87465f07a7
Fix unstable ConsistencyHashPolicy ut (#28375) 
Fixes: #28372

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-13 19:06:20 +08:00
XuanYang-cn
a153950b10
Change channel to Interface (#27839) 
This PR changes `*channel` into RWChannel interface

See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-13 11:16:18 +08:00
wei liu
14c8a90517
Fix auto balance block channel reassign after datanode restart (#28275) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-09 19:00:25 +08:00
wei liu
b9bf910039
fix unstable auto balance config ut (#28288) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-09 10:00:22 +08:00
wei liu
7f78e1dd46
fix datacoord unstable ut (#28281) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-08 18:43:31 +08:00
cai.zhang
fd68837f84
Fix bug for setting index state when indexnode connecting failed (#28220) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-07 23:34:26 +08:00
wei liu
5b45a138b1
disable auto balance when old node exists (#28191) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-07 14:02:20 +08:00
congqixia
c41df18b6d
Add compaction meta in SyncSegmentsRequest (#28199) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-07 10:06:17 +08:00
XuanYang-cn
22c089894e
Add scheduler (#27938) 
See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-07 03:18:18 +08:00
XuanYang-cn
2bbc20c7b8
Update session manager (#27761) 
See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-06 22:10:19 +08:00
aoiasd
1d4be0d257
Adjust datacoord for L0 Delta (#28021) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-06 15:26:16 +08:00
chyezh
86973a8871
[Fixup] error root path for catalog when using tikv (#28163) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-11-06 12:06:19 +08:00
yah01
90e2c63d9e
Fix getting incorrect CPU num (#28146) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-11-06 06:02:16 +08:00
wei liu
7e0bda8b4c
remove binlog/delta log from getRecoveryInfoV2 (#27895) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-30 14:28:15 +08:00
MrPresent-Han
ccaec44930
enable parameter for dataCoord broker timeout(#26972) (#27825) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-10-28 06:14:11 +08:00
Filip Haltmayer
6b1a106a31
Moving etcd client into session (#27069) 
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
 2023-10-27 07:36:12 +08:00
jaime
6749957e71
Refine RPC call in unwatch drop channel (#27864) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-24 17:46:15 +08:00
Xiaofan
2ea7579dbb
Reduce rpc size for GetRecoveryInfoV2 (#27483) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-23 21:44:09 +08:00
zhagnlu
6060dd7ea8
Add chunk manager request timeout (#27692) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-10-23 20:08:08 +08:00
jaime
d2dbbbc11b
Reduce write lock scope in channel manager (#27823) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-21 07:58:16 +08:00
SimFG
9b0ecbdca7
Support to replicate the mq message (#27240) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-10-20 14:26:09 +08:00
congqixia
49516d44b4
Add ctx parameter and log tracer for watch and selectNodes (#27809) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-20 04:22:11 +08:00
zhenshan.cao
c0edc22a6e
Limiting the idempotence check for index to user index params (#27615) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-10-18 20:18:24 +08:00
yihao.dai
49b3a12804
Return newly defined merr instead of grpc unimplemented err (#27751) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-18 15:32:11 +08:00
congqixia
efe8984eca
Fix Flush error message typo (#27729) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-17 17:18:09 +08:00
congqixia
b7bfccaf21
Add channel name paramter in FlushSegmentsRequest (#27725) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-17 12:00:10 +08:00
wayblink
e3f2122618
Expose metrics of stanby coordinators (#27698) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-10-16 15:04:09 +08:00