21776 Commits

Author SHA1 Message Date
Ted Xu
53a4207f46
enhance: improve sort performance by writing with batch record (#39685)
See #37234

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2025-02-12 15:08:47 +08:00
congqixia
95f3a248b3
enhance: [GoSDK] Add range & sparse ann param (#39751)
Related to #38846

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-12 14:54:46 +08:00
yanliang567
5fdc7578bb
test: Add sparse invert index algo check tests (#39691)
related issue: #39332

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-02-12 12:14:46 +08:00
wei liu
c12c4b4fff
fix: [skip e2e] pr conflict cause ut failed (#39811)
Related to https://github.com/milvus-io/milvus/pull/39701 &
https://github.com/milvus-io/milvus/issues/39681

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-12 11:44:51 +08:00
sre-ci-robot
796478e89e Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2025-02-12 00:01:32 +00:00
cai.zhang
9e6e477c5d
fix: Fix modulo for long type (#39722)
issue: #39640

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-11 20:04:46 +08:00
nico
5cdd906d4b
test: update test cases and sdk version (#39443)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2025-02-11 18:42:45 +08:00
congqixia
7b51e4839f
fix: Resolve conflict on qc task test (#39796)
Related to #39701 & #39681

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-11 18:40:45 +08:00
gofastasf
664a8a25f4
fix: Remove getModTime() to use ModTime() from os.FileInfo directly (#39623)
- Use ModTime from os.FileInfo directly instead of custom getModTime
implementation.
- Removed test case for getModTime().

Signed-off-by: Gofastasf gofastasf@gmail.com

Signed-off-by: Go gofastasf@gmail.com
Signed-off-by: Gofastasf <gofastasf@gmail.com>
2025-02-11 16:55:11 +08:00
sthuang
15c8798b93
feat: storage v2 serde reader and writer (#39667)
related: https://github.com/milvus-io/milvus/issues/39173

---------

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-02-11 16:00:46 +08:00
wei liu
ff5c680c99
fix: load collection stucks if compaction/gc happens (#39701)
issue: #39680
if compaction/gc happens, load collection may stuck due to
SegmentNotFound, we should trigger UpdateNextTarget to get a new data
view to execute loading operation.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-11 15:48:50 +08:00
wei liu
85c9f92ff4
fix: uneven distribution caused by executing task delta cache leak (#39702)
issue: #39681 

this PR maintain workload effect in action instead of computing workload
effect from target, which may cause leak if target changes.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-11 14:30:46 +08:00
zhuwenxing
31fe8cc1c3
test: add phrase match in chaos test (#39765)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-11 13:56:45 +08:00
ThreadDao
c1b4bae130
test: add go-sdk cases for collection related APIs (#39537)
issue: 33419
- add cases for: RenameCollection, CollectionProperties,
GetCollectionStats, fast collection

---------

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2025-02-11 13:54:45 +08:00
congqixia
b2d6dbf836
fix: Return early when skip load pk index (#39762)
Previous PR #39437 only print log and add index while load operation is
still executed. This PR return early when segment decides not to load PK
index.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-11 11:04:50 +08:00
Spade A
78b2d205af
fix: phrase match return more readable error msg with invalid slop parameter (#39746)
fix: #39737

---------

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-02-10 23:28:45 +08:00
MimoN
ee716a53f1
fix: [skip e2e] update the collection detail page logic (#39740)
issue: #36621

Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2025-02-10 19:58:45 +08:00
Zhen Ye
d3e32bb599
enhance: make pchannel level flusher (#39275)
issue: #38399

- Add a pchannel level checkpoint for flush processing
- Refactor the recovery of flushers of wal
- make a shared wal scanner first, then make multi datasyncservice on it

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-10 16:32:45 +08:00
wei liu
7f119a7997
enhance: Optimize the result format of GetQueryNodeDistribution (#39664)
Use string array for SealedSegmentIDs to prevent precision loss in JSON
parsers. Large integers (int64) may be incorrectly rounded when parsed
as double.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-10 15:48:51 +08:00
zhuwenxing
13cffafca1
test: add phrase match testcases (#39689)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-10 15:28:45 +08:00
Zhen Ye
82bdf8584c
enhance: keep consistent of memory and meta of broadcaster (#39720)
issue: #38399

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-10 10:44:44 +08:00
Ted Xu
3c8acbce59
enhance: remove useless memory copies on type conversion during sync operations (#39703)
See: #39697

In sync operations, the type conversions from message to insert data
always result in a memory copy, which is not necessary if the converting
type is identical.

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2025-02-10 10:40:44 +08:00
zhuwenxing
ee87e4d0b6
test: add go-sdk cases for full text search (#39570)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Signed-off-by: zhuwenxing <wxzhuyeah@gmail.com>
2025-02-10 10:32:45 +08:00
XuanYang-cn
1f14053c70
enhance: Enable to observe write amplification (#39661)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-02-08 18:38:43 +08:00
zhuwenxing
a3df2782d6
test: [skip e2e]add streaming node chaos test (#39674)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-08 15:34:43 +08:00
Bingyi Sun
c13fc8cd19
enhance: update tantivy version (#39253)
https://github.com/milvus-io/milvus/issues/39254

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-02-08 14:08:43 +08:00
MimoN
cd56d64ec4
fix: [skip e2e] update web ui tasks and query segments list (#39693)
issue: #36621

Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2025-02-08 09:50:48 +08:00
sre-ci-robot
90dead3887 Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2025-02-08 00:01:33 +00:00
sre-ci-robot
ba312427f2
[automated] Update Knowhere Commit (#39696)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-02-08 01:36:43 +08:00
sparknack
2d9bef44d4
fix: sparse: add inverted_index_algo and dim_max_score_ratio config (#39358)
issue: #39332

Signed-off-by: Shawn Wang <shawn.wang@zilliz.com>
2025-02-07 16:40:44 +08:00
yihao.dai
8288c06e3d
fix: Fix flowgraph leak (#39656)
Consider lag targets when determining whether to remove the dispatcher
manager.

issue: https://github.com/milvus-io/milvus/issues/39642

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-07 16:16:44 +08:00
Xianhui Lin
82f9689711
enhance: Add schema update time verification for insert and upsert to use cache (#39096)
enhance: Add schema update time verification for insert and upsert to
use cache
issue: https://github.com/milvus-io/milvus/issues/39093

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-02-07 14:10:45 +08:00
jaime
8a4ac8cccd
enhance: expose more metrics data (#39456)
issue: #36621 #39417
1. Adjust the server-side cache size.
2. Add source information for configurations.
3. Add node ID for compaction and indexing tasks.
4. Resolve localhost access issues to fix health check failures for
etcd.

Signed-off-by: jaime <yun.zhang@zilliz.com>
2025-02-07 11:50:50 +08:00
Zhen Ye
a9e0e0a852
enhance: broadcast with event-based notification (#39522)
issue: #38399

- broadcast message can carry multi resource key now.
- implement event-based notification for broadcast messages
- broadcast message use broadcast id as a unique identifier in message
- broadcasted message on vchannels keep the broadcasted vchannel now.
- broadcasted message and broadcast message have a common broadcast
header now.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-07 11:14:43 +08:00
congqixia
0bac4d3f2a
fix: [GoSDK] Unify AnnRequest creation and fix nil map (#39665)
Related to #39559

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-07 10:34:44 +08:00
congqixia
d7c07d16d3
fix: [GoSDK] Use client config db name when connect (#39662)
Related to #34137

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-07 10:26:43 +08:00
aoiasd
74890dabc9
enhance: skip load bm25 sparse row data (#39078)
Skip load row data of bm25 sparse field, because it's forbidden to
return.

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-02-06 18:40:44 +08:00
Ted Xu
427b6a4c94
enhance: reduce stats task cost by skipping ser/de (#39568)
See #37234

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2025-02-06 17:14:45 +08:00
aoiasd
2b4caba76e
fix: check utf-8 format for varchar with analyzer open (#39299)
relate: https://github.com/milvus-io/milvus/issues/39285

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-02-06 17:11:51 +08:00
Zhen Ye
5669016af0
enhance: erase the rpc level when wal is located at same node (#38858)
issue: #38399

- Make the wal scanner interface same with streaming scanner.
- Use wal if the wal is located at current node.
- Otherwise fallback the old logic.

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-05 22:25:10 +08:00
Gao
c1794cc490
enhance: update knowhere version and IsAdditionalScalarSupported interface (#39573)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2025-02-05 19:51:10 +08:00
cai.zhang
1d54ff157f
fix: Restore the compacting state for stats task during recovery (#39459)
issue: #39333

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2025-02-05 17:11:12 +08:00
Ted Xu
f007465942
fix: unstable UT pack_writer_test (#39641)
See #39601

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2025-02-05 16:39:11 +08:00
wei liu
05ac4041aa
enhance: use rated logger for high frequency log in dist handler (#39452)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-05 15:31:10 +08:00
yihao.dai
f0b7446e6b
enhance: Remove unnecessary collection and partition label from the metrics (#39536)
/kind improvement

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-05 11:01:10 +08:00
Zhen Ye
b1cee78a55
fix: segment stats may be inconsistent after wal closing (#39593)
issue: #38399

- The stats may be kept after wal closing if the growing segment is not
dirty.
- Change the error handling of wal open to avoid redundant manager api
call.

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-05 10:53:10 +08:00
Zhen Ye
a816a03351
fix: unsafe concurrent consuming api of rocksmq (#39544)
issue: #38966

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-05 09:51:10 +08:00
congqixia
b3791a6f90
fix: Use param item formatter to avoid SetConfig to overlay (#39597)
Related to #39596

When updating the build param configuration, the `Formatter` could be
used to do so and completed avoid touching the `overlay` config items

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-27 10:51:07 +08:00
congqixia
f7b7bf69d1
enhance: Add dsl log field for HybridSearch (#39584)
The dsl fields are separated in sub request structs and cannot be
easiliy printed before. This PR adds a log helper to print the dsl
expressions of HybridSearchRequest.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-26 15:57:06 +08:00
sthuang
5f4bad6b23
fix: metastore privilege name check with privilege name all (#39476)
related: https://github.com/milvus-io/milvus/issues/39365

---------

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-01-26 12:17:20 +08:00