See also #28660
This pr add request timeout config item for etcd kv request timeout
Sync the default timeout value to same value for etcdKV & tikv config
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
See also #28628
Previous compaction task blocked the segment sync task and may block the
flowgraph when sync task is generated by auto sync policy This
`BlockAll` call will block forever and cause whole fg stuck
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
create goroutine only once when getOrCreateMergedTimeTickerSender
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
fix#28629
orignal unit test close channel before setting the segment id, so there
is a chance that test read segment id before setting it change unit test
behavior to wait future return now
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
See also #27675, comment from #27874
This PR changes the `IsFull` logic of insert buffer
Changing the limit from rowNum to buffer size,
this shall help form better binlog file when schema has variable-length
field
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
See also #28575
Add zero-length check for `storage.NewPrimaryKeyStats`. This function
shall return error when non-positive rowNum passed.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
See also #27675
- Fix LevelZero segment cannot be flushed
- Add level option for syncTask
- Invoke `AddSegment` when new LevelZero segment is allocated
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
See also #27675
Since `MetaWriter` need `*datapb.FieldBinlog` struct, sync task now
generate FieldBinlog directly
Also fix merged statslog not generated if last task has no insert
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
This PR removes all the commented code and files from PR #28320
For naming issue:
- Renaming `MinCheckpoint` to `EarliestPosition`, see #28320 comment
- Renaming `writebuffer.Mananger` to `BufferMananger`, see #27874
comment
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
See also #27675
This PR make previously merged refactory of datanode go online
- Use write node to replace insert/delete node
- Use write buffer manager to control all buffers
- Use sync manager to control sync tasks instead of flush manager
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Compaction plan result contained one segment for one plan. For l0
compaction would write to multiple segments, this PR expand the segments
number in plan results and refactor some names for readibility.
- Name refactory: - CompactionStateResult -> CompactionPlanResult -
CompactionResult -> CompactionSegment
See also: #27606
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
- Use explicit lifetime control methods: `Start` and `Stop`
- Allow control retry option
- Make sure tt sender worker exit after `Stop` return
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
