aoiasd
0203aefad1
enhance: add concurrency pool for analyzer ( #46185 )
...
relate: https://github.com/milvus-io/milvus/issues/42589
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## New Features
- Added `concurrency_per_cpu_core` configuration parameter for the
analyzer component, enabling customizable per-CPU concurrency tuning
(default: 8).
## Tests
- Added test coverage for batch analysis operations.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-12-23 16:01:18 +08:00
aoiasd
7e4f87e351
fix: Init analyzer at delegator for all field with enable analyzer ( #46361 )
...
To support text match highlight
relate: https://github.com/milvus-io/milvus/issues/46308
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-12-19 10:23:18 +08:00
aoiasd
294282f1d2
enhance: support use nullable field as bm25 function input field ( #44586 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-09-29 10:25:05 +08:00
aoiasd
f98dd84dec
fix: add utf8 check before bm25 functoin run ( #44220 )
...
relate: https://github.com/milvus-io/milvus/issues/44219
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-09-08 14:43:56 +08:00
aoiasd
208a345a3d
enhance: package analyzer code in Go and fix named analyzer as tokenizer ( #43694 )
...
relate: https://github.com/milvus-io/milvus/issues/43687
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-08-27 10:59:52 +08:00
aoiasd
13330bd466
fix: add concurrency and close protect for bm25 function ( #42597 )
...
relate: https://github.com/milvus-io/milvus/issues/42576
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-06-10 11:36:34 +08:00
aoiasd
2eb24fbe7c
fix: analyzer memory leak because function runner not close ( #41839 )
...
relate: https://github.com/milvus-io/milvus/issues/41213
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-06-05 14:24:40 +08:00
aoiasd
2ae4d80120
enhance: support run analyzer by loaded collection field ( #42113 )
...
relate: https://github.com/milvus-io/milvus/issues/42094
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-05-29 10:54:30 +08:00
aoiasd
f52c2909c4
feat: support multi analyzer for bm25 function ( #41351 )
...
relate: https://github.com/milvus-io/milvus/issues/41213
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-04-23 18:22:38 +08:00
SimFG
91d40fa558
fix: Update logging context and upgrade dependencies ( #41318 )
...
- issue: #41291
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-23 10:52:38 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module ( #39990 )
...
Related to #39095
https://go.dev/doc/modules/version-numbers
Update pkg version according to golang dep version convention
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
aoiasd
f16232f834
fix: sparse inc mast less than uint32 max ( #38250 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-12-06 15:14:41 +08:00
Buqian Zheng
00edec2ebd
fix: incorrect dim set when creating BM25 doc sparse array ( #37717 )
...
issue: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-11-16 16:36:30 +08:00
aoiasd
1c5b5e1e3d
feat: Add chinese and english analyzer with refactor jieba tokenizer ( #37494 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-14 10:34:31 +08:00
aoiasd
12951f0abb
enhance: rename tokenizer to analyzer and check analyzer params ( #37478 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-10 16:12:26 +08:00
aoiasd
d67853fa89
feat: Tokenizer support build with params and clone for concurrency ( #37048 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
https://github.com/milvus-io/milvus/issues/36751
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-06 17:48:24 +08:00
aoiasd
139787371e
feat: support embedding bm25 sparse vector and flush bm25 stats log ( #36036 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-09-19 10:57:12 +08:00
