congqixia
b5443ddbd0
enhance: [AddField] Reopen loaded segments after AddField ( #41529 )
...
Related to #39718
This PR:
- Add reopen logic for growing & sealed segments
- Lazy reopen when schema version increases
- Add FinishLoad api for loading progress
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-26 08:48:39 +08:00
congqixia
b36c88f3c8
enhance: [AddField] Broadcast schema change via WAL ( #41373 )
...
Related to #39718
Add Broadcast logic for collection schema change and notifies:
- Streamnode - Delegator
- Streamnode - Flush component
- QueryNodes via grpc
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-04-22 16:28:37 +08:00
Spade A
736512a59e
fix: change log info to debug for collection ref ( #41267 )
...
issue: #41268
Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-04-14 15:16:30 +08:00
smellthemoon
cb1e86e17c
enhance: support add field ( #39800 )
...
after the pr merged, we can support to insert, upsert, build index,
query, search in the added field.
can only do the above operates in added field after add field request
complete, which is a sync operate.
compact will be supported in the next pr.
#39718
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-04-02 14:24:31 +08:00
congqixia
cb7f2fa6fd
enhance: Use v2 package name for pkg module ( #39990 )
...
Related to #39095
https://go.dev/doc/modules/version-numbers
Update pkg version according to golang dep version convention
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
congqixia
bca2a62b78
enhance: Handle PutOrRef collection schema failure error ( #39310 )
...
Related to previous pr #39279
When NewCollection returns nil, the error shall be returned and handled
by caller
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-16 10:13:06 +08:00
Zhen Ye
bb8d1ab3bf
enhance: make new go package to manage proto ( #39114 )
...
issue: #39095
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-10 10:49:01 +08:00
SimFG
2afe2eaf3e
feat: support to replicate collection when the services contains the system tt msg ( #37559 )
...
- issue: #37105
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-17 09:08:46 +08:00
congqixia
7c55649585
enhance: Refine querynode collection number metrics ( #38350 )
...
Related to #37630
Previously the loaded collection metrics was calculated via scanning all
loaded segment in segment manager, which is slow and buggy
implementation.
This PR:
- Move collection num metrics to collection manager
- Remove deprecated loaded partition metrics update logic
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-11 11:36:43 +08:00
Zhen Ye
c6dcef7b84
enhance: move segcore codes of segment into one package ( #37722 )
...
issue: #33285
- move most cgo opeartions related to search/query into segcore package
for reusing for streamingnode.
- add go unittest for segcore operations.
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-29 10:22:36 +08:00
yihao.dai
ed37c27bda
fix: Fix collection leak in querynode ( #37061 )
...
Unref the removed L0 segment count.
issue: https://github.com/milvus-io/milvus/issues/36918
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-25 19:59:29 +08:00
aoiasd
22b917a1e6
enhance: Add collection name label for some metric ( #36951 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-25 14:29:47 +08:00
congqixia
0d8f20f7ce
fix: Pass full field list when partial load enabled ( #37053 )
...
Related to #37038
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-22 18:43:27 +08:00
foxspy
3de57ec4fa
enhance: add vector index mgr to remove vector index type dependency ( #36843 )
...
issue: #34298
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-10-17 22:15:25 +08:00
aoiasd
db34572c56
feat: support load and query with bm25 metric ( #36071 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-11 10:23:20 +08:00
Zhen Ye
a773836b89
enhance: optimize milvus core building ( #35610 )
...
issue: #35549,#35611,#35633
- remove milvus_segcore milvus_indexbuilder..., add libmilvus_core
- core building only link once
- move opendal compilation into cmake
- fix odr
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-23 12:35:02 +08:00
congqixia
2fbc628994
feat: Support field partial load collection ( #35416 )
...
Related to #35415
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-20 16:49:02 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 ( #34394 )
...
issue: #34252
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-29 11:31:51 +08:00
congqixia
efa0c73c62
fix: Unify querynode metrics cleanup in collection release ( #32805 )
...
Related to #32803
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-07 15:41:29 +08:00
aoiasd
31dca3249e
enhance: add type info for payload writer error message and add log when querynode find new collection ( #32522 )
...
relate: https://github.com/milvus-io/milvus/issues/32668
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-07 14:45:29 +08:00
chyezh
9f9ef8ac32
enhance: transfer resource group and dbname to querynode when load ( #30936 )
...
issue: #30931
Signed-off-by: chyezh <chyezh@outlook.com>
2024-03-21 11:59:12 +08:00
congqixia
08aba2e05f
fix: Remove QueryNodeEntitiesSize after segment/collection released ( #31290 )
...
See also #31289
This PR:
- Set collection level `QueryNodeEntitiesSize` to zero if all segment
released
- Delete `QueryNodeEntitiesSize` metrics value after collection ref is
zero
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-15 15:43:04 +08:00
xige-16
9702cef2b5
feat: Support multiple vector search ( #29433 )
...
issue #25639
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2024-01-08 15:34:48 +08:00
yah01
9e0163e12f
enhance: use GPU pool for gpu tasks ( #29678 )
...
- this much improve the performance for GPU index
Signed-off-by: yah01 <yang.cen@zilliz.com>
2024-01-04 17:50:46 +08:00
congqixia
d344336a13
fix: Change schema to atomic.Pointer to avoid data race ( #28739 )
...
Fix #28738
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-25 12:42:24 +08:00
yah01
dc89730a50
Support collection-level mmap control ( #26901 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-02 23:52:16 +08:00
Enwei Jiao
f8dd589755
Refactor collection's cgo call ( #28055 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-11-02 13:02:13 +08:00
congqixia
a7e6f08183
Fix DeleteCollection not protected ( #25599 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-14 17:04:31 +08:00
yah01
d216f9abda
Clear collection meta after all channels/segments released ( #25486 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-14 10:28:30 +08:00
yihao.dai
c56a26a556
Add SetMetricType and set metric only for shard leader ( #25285 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-07-04 10:20:25 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
yah01
fb66924c90
Set metric type while init the collection ( #24665 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-06 13:24:36 +08:00
yihao.dai
b0ce4cff83
Assign metric type in search request when user has not explicitly assigned ( #24591 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-01 18:10:32 +08:00
yihao.dai
1a3dca9b5e
Fix dynamic partitions loading ( #24112 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-18 09:17:23 +08:00
foxspy
6f4ed517de
add growing segment index ( #23615 )
...
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-04-26 10:14:41 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
yah01
081572d31c
Refactor QueryNode ( #21625 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00
