23273 Commits

Author SHA1 Message Date
Spade A
476cf61d98
fix: random sample consider empty input (#40201)
issue: #40198

Fix random sample does not consider empty input, that is no data is hit
by filter expression.

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-02-26 16:15:58 +08:00
yihao.dai
8f077089ba
enhance: Accelerate listing objects during binlog import (#40047)
issue: https://github.com/milvus-io/milvus/issues/40030

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-26 15:55:57 +08:00
zhuwenxing
8fd39779f7
test: add restful v2 api testcases (#39558)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-26 14:03:57 +08:00
zhuwenxing
ac7550da82
test: fix duplicate import file names in tests (#40181)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-26 14:02:05 +08:00
Chun Han
190ac11cd1
fix: cancel sub contexts casade when http request timeout(#40030) (#40059)
related: #40030

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-02-26 11:33:57 +08:00
junjiejiangjjj
162d241063
feat: Add siliconflow text embedding (#39867)
https://github.com/milvus-io/milvus/issues/35856

Signed-off-by: junjie.jiang <junjie.jiang@zilliz.com>
2025-02-26 11:01:56 +08:00
zhuwenxing
01966280da
test: add run_analyzer api test and lindera tokenizer test (#40160)
/kind improvement


/hold

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-26 10:29:55 +08:00
zhuwenxing
828ecacadc
test: fix checker function name, release mistake and add nullable (#40135)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-26 10:27:56 +08:00
Bingyi Sun
f05e9628f6
fix: Fix search failure of null expression (#40129)
issue: #40095

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-02-25 20:43:55 +08:00
congqixia
e0b028ade5
enhance: Integrate holmes as pprof dumper (#40151)
Related to #40150

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-25 19:01:55 +08:00
XuanYang-cn
315cfb7f32
fix: Negative -1 executing compaction tasks (#39954)
See also: #39675

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-02-25 18:07:55 +08:00
Zhen Ye
84df80b5e4
enhance: refactor metrics of streaming (#40031)
issue: #38399

- add metrics for broadcaster component.
- add metrics for wal flusher component.
- add metrics for wal interceptors.
- add slow log for wal.
- add more label for some wal metrics. (local or remote/catcup or
tailing...)

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-25 12:25:56 +08:00
congqixia
ce480c3f57
enhance: [GoSDK] Use pkg v2 module path (#40134)
Related to #39095
Previous PR #39990 update pkg module path using "/v2" package name, this
PR update milvusclient go sdk dependency for this update

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-25 10:33:56 +08:00
sthuang
90acc8a58f
enhance: upgrade go arrow version from 12.0.1 to 17.0.0 (#39916)
related: https://github.com/milvus-io/milvus/issues/39915

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-02-25 10:30:02 +08:00
congqixia
36a4fada02
enhance: Sync proxy.maxTaskNum defaual value to 1024 (#40072)
Sync milvus.yaml value to default value for maxTaskNum

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-24 16:28:06 +08:00
Bingyi Sun
db4769281c
fix: Fall back to a brute-force search if json index type unmatched (#40076)
issue: https://github.com/milvus-io/milvus/issues/35528
If the query data type does not match the index type, fall back to a
brute-force search

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-02-24 16:25:57 +08:00
aoiasd
38f1608910
enhance: pack analyzer code and support lindera tokenizer (#39660)
relate: https://github.com/milvus-io/milvus/issues/39659

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-02-24 12:13:55 +08:00
congqixia
dd68814c15
enhance: Remove hardcoded partition num in restful handler (#40112)
The partition num shall be determined by core logic if user did not
specifiy the partition num in request.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-23 15:33:54 +08:00
cai.zhang
9f5b488f9a
enhance: Export request timeout interval in config (#40119)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-23 15:15:54 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990)
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
congqixia
a774f05ea7
fix: Add sub task pool for multi-stage tasks (#40079)
Related to #40078

Add a subTaskPool to execute sub task in case of logic deadlock
described in issue.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 16:37:54 +08:00
congqixia
e1b5b37195
enhance: Avoid stringtoslicebytes copy for BatchPKExists (#40096)
Using unsafe.Slice to convert string to []byte by directly using
underlying data could avoid lots of copy and cpu time

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 15:23:52 +08:00
Ted Xu
8562a102ec
enhance: API integration with storage v2 in mix-compactions (#40008)
See #39173

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2025-02-22 14:23:54 +08:00
SimFG
ad36347fb3
fix: add BeginTimestamp and EndTimestamp to insert and upsert messages (#40110)
- issue: #40109
- caused by: #38656

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-02-22 12:29:53 +08:00
smellthemoon
8b974c5742
enhance: support compact if lack of binlog (#40000)
https://github.com/milvus-io/milvus/issues/39718

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-02-22 10:51:56 +08:00
yihao.dai
d72d2281ca
fix: Fix concurrent map (#39775)
issue: https://github.com/milvus-io/milvus/issues/39778

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-22 09:51:54 +08:00
sre-ci-robot
dd1347d041
[automated] Update Knowhere Commit (#40103)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-02-22 01:01:53 +08:00
sthuang
3eb3af5f08
feat: explicitly specify column groups for storage v2 api (#39790)
* use the new packed reader and writer api to be compatible with current
etcd meta
* For the new packed writer API: column groups and paths are explicitly
defined by users and won't split column groups by memory in storage v2.
Packed writer follows the user-defined column groups to split arrow
record and write into the corresponding file path.
* For the new packed reader API: read paths are explicitly defined by
users.
related: #39173

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-02-21 22:03:54 +08:00
yihao.dai
2a037a97f1
enhance: Add get vector latency metric and refine request limit error message (#40083)
issue: https://github.com/milvus-io/milvus/issues/40078

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-21 19:41:55 +08:00
Chun Han
d6699b5f50
enhance: support return configable properties when describing index(#39951) (#40042)
related: #39951

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-02-21 19:07:53 +08:00
XuanYang-cn
fb969cf636
fix: A segment may never transfer from sealed to flushing (#39993)
See also: #39717

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-02-21 16:51:54 +08:00
wei liu
7d2c948c69
fix: task delta cache leak on reduce task (#40055)
issue: #40052

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-21 16:47:54 +08:00
Spade A
d34d70582d
fix: fix misleading name *_add_multi_* (#39997)
fix: #39995

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-02-21 16:45:55 +08:00
zhuwenxing
0d873719d3
test: fix phrase match verication in testcase (#40069)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-21 13:19:53 +08:00
yanliang567
afbe02009e
test: Add test for search iterator external filter function (#39986)
related issue: #39985
1. add some test for external filter function
2. combine search iterator tests into one test file

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-02-21 11:25:53 +08:00
wei liu
07578041ba
fix: querycoord panic in cornor case (#40057)
issue: #40050

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-21 11:19:58 +08:00
SimFG
b562f8e644
fix: add filter to exclude L0 import jobs in compaction trigger (#40045)
- issue: #39849

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-02-21 10:45:53 +08:00
sre-ci-robot
f0d3d98c3f
[automated] Update Knowhere Commit (#40063)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-02-21 01:19:54 +08:00
Chun Han
1dc31619f8
enhance: support create collection with description(#40022) (#40023)
related: #40022

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-02-20 22:31:53 +08:00
SimFG
aba39ff98f
fix: enhance isBalanced function to correctly count quote pairs (#40001)
- issue: #39999

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-02-19 20:19:00 +08:00
Zhen Ye
fd701eca71
fix: local wal perform different with remote wal (#39967)
issue: #38399

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-19 19:12:51 +08:00
sthuang
f47320e0e7
enhance: clean up legacy storage v2 (#39987)
related: https://github.com/milvus-io/milvus/issues/39173

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-02-19 15:42:52 +08:00
congqixia
5d83deb3f8
fix: Use start pos ts instead for sealSegmentByLifetime policy (#39982)
Related to #39981

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-19 15:41:06 +08:00
zhuwenxing
1e5d6c1545
test: remove xfail for text match testcases (#39920)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-19 14:16:50 +08:00
zhuwenxing
0cb0525130
test: add phrase match testcases in restful client (#39785)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-19 14:15:11 +08:00
zhuwenxing
9d37f0f9ee
test: add fts and text match verification in second test (#39970)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-19 14:12:58 +08:00
Patrick Weizhi Xu
04fff74a56
feat: introduce Text data type (#39874)
issue: https://github.com/milvus-io/milvus/issues/39818

This PR mimics Varchar data type, allows insert, search, query, delete,
full-text search and others.
Functionalities related to filter expressions are disabled temporarily. 

Storage changes for Text data type will be in the following PRs.

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2025-02-19 11:04:51 +08:00
yihao.dai
c917fe4782
fix: Fix failed to seek to earliest position (#39965)
If it is the earliest message ID, skip the seek to prevent failure.

issue: https://github.com/milvus-io/milvus/issues/39964

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-19 00:30:51 +08:00
Spade A
52c7d7dd80
fix: offset combined with term should be based on Token positions in phrase match (#39931)
fix: #39711

Unlike English sentence where each words are parsed exactly once and one
after one with position length 1, one Chinese word may be parsed to
multiple words with position length larger than 1.

For example, "badminton and skiing" will be parsed to Token{ start: 0,
length: 1, text: "badminton" }, Token{ start: 1, length: 1, text: "and"
}, and Token{ start: 2, length: 1, text: "tennis" }.

While for exmaple for Chinsese: "羽毛球和滑雪" may be parsed to Token{ start:
0, length: 2, text: "羽毛" }, Token{ start: 0, length: 3, text: "羽毛球" },
Token{ start: 3, length: 1, text: "和" }, and Token{ start: 4, length: 2,
text: "滑雪" }.

This PR fix that the code not recognizes this situation.

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-02-18 20:38:51 +08:00
congqixia
59881a7f73
fix: Remove load field & schema column size check (#39833)
Related to #39788

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-18 16:24:51 +08:00