mirror of
https://gitee.com/milvus-io/milvus.git
synced 2025-12-08 01:58:34 +08:00
f01ff57f3f
Related to #39173 `null_bitmap_data()` returns raw pointer of null bitmap of Array. While after slicing, this bitmap is not rewritten due to zero copy implementation, so the current start pos maybe non-zero while FillFieldData generating column `valid_data` array. This PR add `offset` param for `FillFieldData` method, and force all invocation pass correct offset of `null_bitmap_data` ptr. Also update milvus-storage commit fixing reader failed to return data when buffer size smaller than row group size problem. --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
29 lines
1.3 KiB
Go
29 lines
1.3 KiB
Go
// Copyright 2023 Zilliz
|
|
//
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
package packed
|
|
|
|
const (
|
|
// ColumnGroupSizeThreshold is the threshold of column group size per row.
|
|
ColumnGroupSizeThreshold = 1024 // 1KB
|
|
// DefaultBufferSize is the default buffer size for writing data to storage.
|
|
DefaultWriteBufferSize = 32 * 1024 * 1024 // 32MB
|
|
// DefaultBufferSize is the default buffer size for reading data from storage.
|
|
DefaultReadBufferSize = -1 // use -1 for unlimited
|
|
// DefaultMultiPartUploadSize is the default size of each part of a multipart upload.
|
|
DefaultMultiPartUploadSize = 10 * 1024 * 1024 // 10MB
|
|
// Arrow will convert these field IDs to a metadata key named PARQUET:field_id on the appropriate field.
|
|
ArrowFieldIdMetadataKey = "PARQUET:field_id"
|
|
)
|