32 Commits

Author SHA1 Message Date
aoiasd
96d0e780ac
fix: segcore collection schema update not concurrent safe. (#45337)
relate: https://github.com/milvus-io/milvus/issues/45345

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-11-14 17:51:37 +08:00
congqixia
b76478378a
feat: [Tiered] Make load list work as warmup hint (#42490)
Related to #42489
See also #41435

This PR's main target is to make partial load field list work as caching
layer warmup policy hint. If user specify load field list, the fields
not included in the list shall use `disabled` warmup policy and be able
to lazily loaded if any read op uses them.

The major changes are listed here:
- Pass load list to segcore and creating collection&schema
- Add util functions to check field shall be proactively loaded
- Adapt storage v2 column group, which may lead to hint fail if columns
share same group

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-06-04 10:28:32 +08:00
congqixia
b5443ddbd0
enhance: [AddField] Reopen loaded segments after AddField (#41529)
Related to #39718

This PR:
- Add reopen logic for growing & sealed segments
- Lazy reopen when schema version increases
- Add FinishLoad api for loading progress

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-26 08:48:39 +08:00
congqixia
b36c88f3c8
enhance: [AddField] Broadcast schema change via WAL (#41373)
Related to #39718

Add Broadcast logic for collection schema change and notifies:
- Streamnode - Delegator
- Streamnode - Flush component
- QueryNodes via grpc

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-22 16:28:37 +08:00
yah01
aef483806d
enhance: improve the segcore logs (#29372)
- remove the streaming logging
- refine existing logs

fix #29366

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-23 21:52:43 +08:00
Bingyi Sun
36f69ea031
feat: integrate storagev2 in building index of segcore (#28768)
issue: https://github.com/milvus-io/milvus/issues/28655

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-12-05 16:48:54 +08:00
Enwei Jiao
f8dd589755
Refactor collection's cgo call (#28055)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-11-02 13:02:13 +08:00
foxspy
6f4ed517de
add growing segment index (#23615)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-04-26 10:14:41 +08:00
yah01
a4031da634
Refine string parameters, avoid coping or deref (#22708)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-13 17:53:53 +08:00
yah01
bdd6bc7695
Re-format cpp code (#22513)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-02 15:55:49 +08:00
Cai Yudong
04d34bfcd7
Remove redundant code (#10635)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-26 13:12:21 +08:00
Cai Yudong
4cd83e78ed
Reorder header files for segcore/Collection.cpp (#10634)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-26 13:10:30 +08:00
FluorineDog
2cec04ed90 Fix empty schema proto hack
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-03-17 11:35:28 +08:00
GuoRentong
1104f059ee Update doc:module interfaces
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-13 11:08:03 +08:00
quicksilver
8e9d8e36e1 Update run_go_unittest.sh
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-13 10:40:46 +08:00
FluorineDog
4cd42c553f Rename field_name, make field_id strongly typed, skip multithread test
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-12 18:31:52 +08:00
bigsheeper
762b1e1f1f Add ddService, fieldID, requestID and refactor params table, fix nil hits search result bug
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-10 16:31:09 +08:00
XuanYang-cn
e6f726e73a Add cache for thirdparty files cache
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-08 18:51:07 +08:00
neza2017
70710dee47 Add parquet payload
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 16:11:03 +08:00
FluorineDog
6412ebc0d4 Add support of metric type in schema, enable binary vector, fix segfault
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-05 06:46:01 +08:00
cai.zhang
85a544b79b Add dockerfile for each component
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-28 19:06:48 +08:00
dragondriver
568cef0730 Fix InsertTask and SearchTask
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-28 10:48:29 +08:00
FluorineDog
c9fb34142c Enable primary_key switch
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-28 09:16:00 +08:00
FluorineDog
df1dcf81b9 Update cpplint.py and lint all files
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-24 21:28:38 +08:00
neza2017
ce89f64bb3 Set master config
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-19 10:46:17 +08:00
sunby
9e4ce8454e Implement segment management in master
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-11-18 15:04:17 +08:00
FluorineDog
e7dd30a884 Add framework of ExecExprVisitor
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-18 17:32:52 +08:00
FluorineDog
f0ec7a8c78 Support Placeholder Parser and refine CreatePlan on predicate-less
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-12 12:18:07 +08:00
bigsheeper
be409b29c8 Refactor cgo collection, partition and segment
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-09 18:07:15 +08:00
FluorineDog
d7ef21c644 Remove message.pb and reformat
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-03 12:42:13 +08:00
FluorineDog
9d2fa4e430 Add SyntaxTree of QueryNode and Expr
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-03 11:45:48 +08:00
GuoRentong
15b6963be0 Replace pdf figs with png figs
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2020-10-31 15:11:47 +08:00