issue: https://github.com/milvus-io/milvus/issues/32262
The old implementation always takes metric type from the first sub
result, which may not always be valid. The fixed implementation returns
initialized metric type from sub results.
Signed-off-by: smdsbz <smdsbz@qq.com>
Bumps [deepdiff](https://github.com/seperman/deepdiff) from 6.7.1 to
8.6.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/seperman/deepdiff/releases">deepdiff's
releases</a>.</em></p>
<blockquote>
<h2>8.5.0</h2>
<ul>
<li>Updating deprecated pydantic calls</li>
<li>Switching to pyproject.toml</li>
<li>Fix for moving nested tables when using iterable_compare_func.
by</li>
<li>Fix recursion depth limit when hashing numpy.datetime64</li>
<li>Moving from legacy setuptools use to pyproject.toml</li>
</ul>
<h2>8.4.1</h2>
<ul>
<li>pytz is not required.</li>
</ul>
<h2>8.4.0</h2>
<ul>
<li>Adding BaseOperatorPlus base class for custom operators</li>
<li>default_timezone can be passed now to set your default timezone to
something other than UTC.</li>
<li>New summarization algorithm that produces valid json</li>
<li>Better type hint support</li>
</ul>
<h2>8.1.1</h2>
<p>Adding Python 3.13 to setup.py</p>
<h2>8.1.0</h2>
<ul>
<li>Removing deprecated lines from setup.py</li>
<li>Added <code>prefix</code> option to <code>pretty()</code></li>
<li>Fixes hashing of numpy boolean values.</li>
<li>Fixes <strong>slots</strong> comparison when the attribute doesn't
exist.</li>
<li>Relaxing orderly-set reqs</li>
<li>Added Python 3.13 support</li>
<li>Only lower if clean_key is instance of str <a
href="https://redirect.github.com/seperman/deepdiff/issues/504">#504</a></li>
<li>Fixes issue where the key deep_distance is not returned when both
compared items are equal <a
href="https://redirect.github.com/seperman/deepdiff/issues/510">#510</a></li>
<li>Fixes exclude_paths fails to work in certain cases</li>
<li>exclude_paths fails to work <a
href="https://redirect.github.com/seperman/deepdiff/issues/509">#509</a></li>
<li>Fixes to_json() method chokes on standard json.dumps() kwargs such
as sort_keys</li>
<li>to_dict() method chokes on standard json.dumps() kwargs <a
href="https://redirect.github.com/seperman/deepdiff/issues/490">#490</a></li>
<li>Fixes accessing the affected_root_keys property on the diff object
returned by DeepDiff fails when one of the dicts is empty</li>
<li>Fixes accessing the affected_root_keys property on the diff object
returned by DeepDiff fails when one of the dicts is empty <a
href="https://redirect.github.com/seperman/deepdiff/issues/508">#508</a></li>
</ul>
<p>8.0.1 - extra import of numpy is removed</p>
<h2>8.0.0</h2>
<p>With the introduction of <code>threshold_to_diff_deeper</code>, the
values returned are different than in previous versions of DeepDiff. You
can still get the older values by setting
<code>threshold_to_diff_deeper=0</code>. However to signify that enough
has changed in this release that the users need to update the parameters
passed to DeepDiff, we will be doing a major version update.</p>
<ul>
<li><input type="checkbox" checked="" disabled="" />
<code>use_enum_value=True</code> makes it so when diffing enum, we use
the enum's value. It makes it so comparing an enum to a string or any
other value is not reported as a type change.</li>
<li><input type="checkbox" checked="" disabled="" />
<code>threshold_to_diff_deeper=float</code> is a number between 0 and 1.
When comparing dictionaries that have a small intersection of keys, we
will report the dictionary as a <code>new_value</code> instead of
reporting individual keys changed. If you set it to zero, you get the
same results as DeepDiff 7.0.1 and earlier, which means this feature is
disabled. The new default is 0.33 which means if less that one third of
keys between dictionaries intersect, report it as a new object.</li>
<li><input type="checkbox" checked="" disabled="" /> Deprecated
<code>ordered-set</code> and switched to <code>orderly-set</code>. The
<code>ordered-set</code> package was not being maintained anymore and
starting Python 3.6, there were better options for sets that ordered. I
forked one of the new implementations, modified it, and published it as
<code>orderly-set</code>.</li>
<li><input type="checkbox" checked="" disabled="" /> Added
<code>use_log_scale:bool</code> and
<code>log_scale_similarity_threshold:float</code>. They can be used to
ignore small changes in numbers by comparing their differences in
logarithmic space. This is different than ignoring the difference based
on significant digits.</li>
<li><input type="checkbox" checked="" disabled="" /> json serialization
of reversed lists.</li>
<li><input type="checkbox" checked="" disabled="" /> Fix for iterable
moved items when <code>iterable_compare_func</code> is used.</li>
<li><input type="checkbox" checked="" disabled="" /> Pandas and Polars
support.</li>
</ul>
<h2>7.0.1</h2>
<ul>
<li><input type="checkbox" checked="" disabled="" /> When verbose=2,
return <code>new_path</code> when the <code>path</code> and
<code>new_path</code> are different (for example when ignore_order=True
and the index of items have changed).</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/seperman/deepdiff/commits">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/milvus-io/milvus/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Related to #39173
This PR:
- Bump milvus-storage commit to handle bucket name concatenation logic
in multipart s3 fs
- Remove all user-side bucket name concatenation code
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #43858
Reduce CompactionDropToleranceInSeconds from 24 hours to 1 hour to
improve memory efficiency and faster task metadata cleanup. Changes
include:
- Update default value from 86400s (24h) to 3600s (1h) in
component_param.go
- Update corresponding configuration in milvus.yaml
- Faster cleanup of completed compaction task metadata
- Reduce memory footprint by shorter retention period
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: https://github.com/milvus-io/milvus/issues/42032
- Use bytes to estimate load resource in the whole estimation procedure
- Add num_rows and dim info for vector index to better estimate
- Disable eviction for tiered index's meta
---------
Signed-off-by: chasingegg <chao.gao@zilliz.com>
issue: https://github.com/milvus-io/milvus/issues/42148
Optimized from
Go VectorArray → VectorArray Proto → Binary → C++ VectorArray Proto →
C++ VectorArray local impl → Memory
to
Go VectorArray → Arrow ListArray → Memory
---------
Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
issue: https://github.com/milvus-io/milvus/issues/44011
namespace is an alias for tenant. if this property is enabled, milvus
will add a __namespace_id field.
Modifications in the future will use this property to do compaction and
search.
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
https://github.com/milvus-io/milvus/issues/44011
this is to support compaction that sorts records by partition key and pk
in the future
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
issue: #43980
Fix panic issue caused by incorrect nullable field merging logic when
upsert converts to insert operation on empty tables.
- Add AppendFieldDataWithNullData to handle nullable field merging
- Fix existing data merge with skipAppendNullData=false
- Fix insert data merge with skipAppendNullData=true
- Add unit tests for nullable field data appending scenarios
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: #42942
This pr includes the following changes:
1. Added checks for index checker in querycoord to generate drop index
tasks
2. Added drop index interface to querynode
3. To avoid search failure after dropping the index, the querynode
allows the use of lazy mode (warmup=disable) to load raw data even when
indexes contain raw data.
4. In segcore, loading the index no longer deletes raw data; instead, it
evicts it.
5. In expr, the index is pinned to prevent concurrent errors.
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
Related to #44150
This PR make enabling `dynamic schema` feature for an existing
collection possible.
This related API is to reuse `AlterCollection` and underhood its
redirected to `adding nullable json field`
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
related: #42417
- Add NameValidationAllowedChars and RoleNameValidationAllowedChars
configuration parameters to specify additional characters allowed
respectively in (generic) names and a role names
- All validations in validateName method is moved to a the new method
validateNameWithCustomChars which is called by both validateName
and ValidateRoleName while specifying characters allowed
Signed-off-by: Jean-Francois Weber-Marx <jfwm@hotmail.com>
Signed-off-by: Jean-Francois Weber-Marx <jf.webermarx@criteo.com>
