pr: https://github.com/milvus-io/milvus/pull/31097
* When the collection is created using an SDK and includes array and
JSON datatypes in the schema, data can be inserted using the RESTful
API.
* When the collection is created using the RESTful API and includes JSON
and array datatypes in dynamic fields, data can also be inserted using
the RESTful API.
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
See also #31024#31058
Flush cost boosted from 2 seconds to 5 or more after the change of
channel updater. This PR add a manual trigger method to accelerate flush
procedure.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Cherry pick from master
pr: #31021
- Change load field log from "dy pool" to "load pool"
- Also defer delete when there is no error
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
This PR includes the following adjustments:
1. To prevent channelCP update task backlog, only one task with the same
vchannel is retained in the updater. Additionally, the lastUpdateTime is
refreshed after the flowgraph submits the update task, rather than in
the callBack function.
2. Batch updates of multiple vchannel checkpoints are performed in the
UpdateChannelCheckpoint RPC (default batch size is 128). Additionally,
the lock for channelCPs in DataCoord meta has been switched from key
lock to global lock.
3. The concurrency of UpdateChannelCheckpoint RPCs in the datanode has
been reduced from 1000 to 10.
issue: https://github.com/milvus-io/milvus/issues/30004
pr: https://github.com/milvus-io/milvus/pull/30941
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Cherry-pick from master
pr: #30994
Segment load memory usage is underestimated due to removing the load
memroy factor. This PR adds it back to protect querynode OOM during some
extreme memory cases.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
If DC restarted, those unkonwn compaction tasks
will never get call back in DN, so that the segments in the compaction
task will be locked, unable to sync and compaction again, blocking cp
advance and compaction executing.
See also: #30137
pr: #30850
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
issue: #30723
pr: #30724
This PR skip generate balance task when collection's target isn't ready.
also refine the check stale logic in query coord's scheduler, if channel
exist in current or next target, task won't be canceled.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
pr: #30654
This PR add retry on all interface which belong to indexcoord in milvus
2.2 and. move to data coord in milvus 2.3, to prevent meet unimplemented
error during rolling upgrade from milvus 2.2 to 2.3.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
master pr: #30871 issue: #30870
fix: vector field cannot be empty while insert
did a check whether the vector field is empty in advance
master pr: #30740
fix:
1. spelling mistake about metricsType #30643
2. int64 percious #20415
3. insert into collection which has multi vector fields #30674
enhance: support dataType: Float16Vector & BFloat16Vector #22837
#30980(master pr: #30969)
enhance: describe collection will show the field is partition key or not
#30789
---------
Signed-off-by: PowderLi <min.li@zilliz.com>
issue: #30950
pr: #30951
due to segment version doesn't update as expected.
This PR will update segment version until segment become loaded
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Cherry pick from master
pr: #30757
See also #30756
This PR:
- Request disk resource when index type, version loaded with disk
- Add attribute cache for index utility
- Add `typeutil.Pair`
---------
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
flush rate control at collection level to avoid generate too much
segment.
0.1 qps by default.
issue: #29477
pr: #29567
Signed-off-by: chyezh <ye.zhen@zilliz.com>
Cherry-pick from master
pr: #30833
See also #30832
This PR removes time tick delay metrics when rootcoord GetMetrics
response does not have previously existed querynode/datanode
Also add unit tests for this case
---------
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi.Xia <congqi.xia@zilliz.com>
Revert "enhance: reduce many I/O operations while loading disk index
(#30189) (#30690)" This reverts commit
d4c4bf946b15bc537acd170dfd1d938bea237c7a.
Revert "enhance: limit the max pool size to 16 (#30371) (#30415)" This
reverts commit 52ac0718f059d4aa45c5908ec8507e6045b24e1f.
Revert "enhance: convert the `GetObject` util to async (#30166)
(#30197)" This reverts commit 4b7c5baab773366aa8084762e7321130c4f894b7.
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
issue: #30715
pr: #30754
- Bug: Set nil struct pointer to describe nil interface.
Panic with segment violation when calling method on this nil struct
pointer.
Signed-off-by: chyezh <chyezh@outlook.com>
before this, every time writting the index chunk data into the disk,
there are 4 I/O operations:
- open the file
- seek to the offset
- write the data
- close the file
this optimized this to open only once and continiously write all data.
This also makes it concurrent to load the files from object storage
pr: #30189
Signed-off-by: yah01 <yang.cen@zilliz.com>
