admin/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2026-02-02 01:06:41 +08:00
Code Issues Packages Projects Releases Wiki Activity
18,621 Commits 146 Branches 196 Tags
Commit Graph

46 Commits

Author SHA1 Message Date
chyezh
c8e3a48214
fix: querynode num entity metric is broken by illegal label (#29949) 
issue: #29766
also see pr: #29825
pr: #29948

Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2024-01-14 10:22:59 +08:00
wei liu
86cddd24b5
enhance: Add ctx for load index logs (#29686) (#29905) 
pr: #29686
This PR add ctx for load index logs

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-01-12 18:56:58 +08:00
yah01
4edcd4d22b
fix: the insert count is zero after set the pointer to nil (#29870) (#29881) 
this leads to the EntitiesNum metric would be never reduced

fix: #29766
pr: #29870

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2024-01-12 10:20:51 +08:00
zhenshan.cao
7cf2be09b5
fix: Restore the MVCC functionality. (#29749) (#29802) 
When the TimeTravel functionality was previously removed, it
inadvertently affected the MVCC functionality within the system. This PR
aims to reintroduce the internal MVCC functionality as follows:

1. Add MvccTimestamp to the requests of Search/Query and the results of
Search internally.
2. When the delegator receives a Query/Search request and there is no
MVCC timestamp set in the request, set the delegator's current tsafe as
the MVCC timestamp of the request. If the request already has an MVCC
timestamp, do not modify it.
3. When the Proxy handles Search and triggers the second phase ReQuery,
divide the ReQuery into different shards and pass the MVCC timestamp to
the corresponding Query requests.

issue: #29656
pr: #29749

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-01-11 14:42:49 +08:00
yah01
e7e4561da8
fix: the entities num metric may be contributed more than once (#29767) (#29825) 
the growing segments contribute to this metric while inserting and
putting into the manager, but the current impl inserts data before
putting the segments into manager, which leads to double contributions

fix: #29766
pr: #29767

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2024-01-11 10:24:51 +08:00
yah01
58410d8b62
enhance: skip loading duplicated index (#29715) (#29716) 
this protect the loading index from failure, and speed up the loading
progress
pr: #29715

Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2024-01-07 17:00:48 +08:00
congqixia
fc65f01ddd
enhance: [Cherry-pick] Cache segment row num, size, and insert count to reduce CGO calls (#28007) (#29679) 
Cherry pick from master
pr: #28007
See also #29650

Signed-off-by: yah01 <yah2er0ne@outlook.com>
Co-authored-by: yah01 <yah2er0ne@outlook.com>
 2024-01-04 23:04:47 +08:00
congqixia
687eb3955e
enhance: [Cherry-pick] Refine C.NewSegment response and handle exception (#28952) (#29550) 
Cherry-pick from master
pr: #28952
See also #28795

Orignal `C.NewSegment` may panic if some condition is not met, this pr
changes response struct to `CNewSegmentResult`, which contains
`C.CStatus` and may return catched exception

---------

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-28 14:48:47 +08:00
congqixia
dd52a674aa
enhance: [cherry-pick] add ctx for HandleCStatus and callers (#29517) (#29546) 
Cherry-pick from master
pr: #29517 
See also #29516

Make `HandleCStatus` print trace id for better logging

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-12-28 10:20:47 +08:00
yah01
e422a62a80
enhance: improve the handling for segcore error (#29471) (#29521) 
- fix lost exception details in segcore
- improve the logs of handling errors from segcore

pr: #29471

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-27 19:06:46 +08:00
yah01
cd9e3c4837
fix: creating growing segments may introduce many threads (#29314) 
many growing segments may be created in a short time and there is no
restriction to the process, the CGO call will leave many threads

related: https://github.com/milvus-io/milvus/issues/29282
pr: #29306

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-19 18:02:40 +08:00
yah01
a1b861ed7a
enhance: improve load speed (#28518) (#28719) 
This check rejects load request if running out the pool workers, but
small segment would be loaded soon, another segments would been loading
again after a check interval, which leads to slow loading for collection

Block the request by go pool

pr: #28518

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-26 22:10:26 +08:00
MrPresent-Han
af61a80f73
connect traceID for retrieve (#27804) 
/kind improvement

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-10-19 19:22:10 +08:00
yihao.dai
8f4aaa2da8
Remove ValidateIndexedFieldsData (#27254) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-22 18:31:25 +08:00
SimFG
26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
yah01
0a750408d0
Skip delta logs have been applied (#26971) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-19 16:21:23 +08:00
yihao.dai
bb6711f28c
Add ChunkCache: support get vector from storage (#26142) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-15 10:21:20 +08:00
congqixia
c6116d1819
Remove segment to LocalSegment type assertion (#26931) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-09 10:35:16 +08:00
MrPresent-Han
8330c18dc9
add log for loading segment(#26564) (#26640) 
/kind improvement

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-08-31 12:03:00 +08:00
congqixia
1cf6e00fa6
Improve segment manager interface (#26637) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-29 15:46:27 +08:00
yah01
5dae6a654a
Protect segment from being released while query/search (#26322) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-16 18:38:17 +08:00
yah01
3d8871b30e
Fix data race while updating segment version (#26387) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-16 16:30:18 +08:00
wei liu
b47a72bfcf
fix set dirty segment distribution to leader view (#26180) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-11 11:21:32 +08:00
Enwei Jiao
ca1349708b
Remove time travel ralted testcase (#26119) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-08-10 18:53:17 +08:00
xige-16
1055c90456
Add default retrieve limit (#24782) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-08-10 14:11:15 +08:00
Jiquan Long
5c1f79dc54
Push down the limit operator to segcore (#25959) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-08-01 20:29:05 +08:00
congqixia
8b5e276361
Use different cgo pool for SQ and other operations (#26021) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-01 09:19:05 +08:00
yah01
168f346620
Remove deprecated interface (#25864) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-07-24 17:47:00 +08:00
congqixia
cc71e065e3
Support load index after segment loaded (#25567) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-18 10:51:19 +08:00
yah01
205a7c430a
Fix QueryNode panic while inserting to released segment (#25495) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-07-12 19:48:29 +08:00
yihao.dai
29b9b26de8
Improve get vector error msg (#24838) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-27 16:34:45 +08:00
xige-16
04082b3de2
Migrate the ability to upload and download binlog to cpp (#22984) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-25 14:38:44 +08:00
yah01
a413842e38
Fix deleted data is still visible (#24849) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-06-16 17:16:41 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-09 01:28:37 +08:00
Enwei Jiao
d3af451d92
Upgrade golangci-lint (#24707) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-06-07 19:34:36 +08:00
yihao.dai
014387fd94
Forbid to get quantized vector from ChunkManager (#24334) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-24 23:03:27 +08:00
Bingyi Sun
0c8045d66c
Fix crash caused by segment function concurrency (#24351) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-05-24 19:23:27 +08:00
yihao.dai
45aa9779e0
Retrieve segments concurrently (#24245) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-19 18:19:24 +08:00
yihao.dai
7e0d1492c7
Load delete from channel checkpoint (#23961) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-09 19:10:41 +08:00
yihao.dai
092d743917
Add support for getting vectors by ids (#23450) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-23 09:00:32 +08:00
Enwei Jiao
967a97b9bd
Support json & array types (#23408) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
 2023-04-20 11:32:31 +08:00
congqixia
4fe363c4b2
Add KNN cgo pool (#23526) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-19 23:56:31 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
Enwei Jiao
50dab2d394
Add tracing for retrieve (#23033) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-03-29 18:10:02 +08:00
yah01
826c1fb850
Support mmap on QueryNodeV2 (#23073) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-28 21:30:05 +08:00
yah01
081572d31c
Refactor QueryNode (#21625) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-27 00:42:00 +08:00