1343 Commits

Author SHA1 Message Date
xige-16
1ea477151e
Add disk metric info (#25678)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-22 16:58:22 +08:00
yihao.dai
2c32d0dbae
Add server id validation interceptor (#26395) (#26424)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-18 12:42:20 +08:00
Bingyi Sun
86fd1cbf10
Fix leader view return loading shard cluster (#26263)
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-08-10 17:55:16 +08:00
Jiquan Long
2ffd3a3808
Fix useless log (#26059)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-03 15:35:09 +08:00
zhagnlu
caf62033cb
Upgrade minio-go and add region and virtual host config for segcore chunk manager (#25811)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-08-02 14:47:07 +08:00
yah01
9857319483
Improve memory/disk usage prediction logs (#26025)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-08-01 14:07:05 +08:00
congqixia
1e800bf610
Add back segment CGO pool and separate sq/dm ops (#26035)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-01 09:21:05 +08:00
Bingyi Sun
8dc48aeed9
Fix misleading log info (#25726)
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-07-28 17:41:03 +08:00
Enwei Jiao
c2ba63e9d8
Fix sc.distribution is nil (#25904)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-26 11:01:02 +08:00
congqixia
8542b26629
Use normal msgstream instead of mqttmsgstream for delta channel (#25849)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-24 09:09:05 +08:00
Bingyi Sun
31b3391fb3
Remove debug log (#25779)
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-07-20 14:00:59 +08:00
Buqian Zheng
bf28a01927
Add minCPUParallelTaskNumRatio config to allow better parallelism when estimated cpu usage of a single task is higher than total cpu usage (#25772)
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2023-07-20 10:04:50 +08:00
yah01
07441faeed
Fix delete messages unsorted (#25757)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-19 21:32:59 +08:00
congqixia
48e30aa1dd
Fix querynode always retry on corrupted statslog (#25104)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-19 20:16:58 +08:00
yah01
7d6fae42d1
Stop blocking until all channels & segments released (#25352)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-19 11:44:58 +08:00
xige-16
7d2b1a6f08
Add current used disk log when load (#25590)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-07-17 11:56:35 +08:00
Bingyi Sun
605d1aa630
Fix deleted data may be returned by search/query (#25513)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-07-13 15:24:30 +08:00
SimFG
c14a2511ee
Prevent exclusive consumer exception in pulsar (#25376) (#25378)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-07-13 13:28:29 +08:00
cai.zhang
ca3d70bd4e
Fix bug of missing JSON type when sorting retrieve results (#25412)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-07-10 17:15:41 +08:00
yah01
8cd758e3db
Fix QueryNode always retries to load stats even file corrupted (#25029)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-10 14:04:31 +08:00
yah01
56c32d3b32
Fix dup deletions make deleted records visible (#25369)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-07 14:24:26 +08:00
SimFG
afd1c66de9
Fix the data race in waitgroup for graceful stop (#25224)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-07-06 15:14:27 +08:00
yihao.dai
e2952c6d0b
Add support for getting vectors by ids (#23450) (#25090)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-07-04 17:00:27 +08:00
Jiquan Long
68b671a1bd
Skip unnecessary reduce phase during search (#25166) (#25192)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-07-03 20:14:25 +08:00
yah01
7c1d35dcd6
Fix delete record may be re-applied (#24858)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-03 15:04:25 +08:00
zhenshan.cao
62e9e4a266
Fixbug: metric_type not set in shard member (#25266)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-07-03 09:54:24 +08:00
SimFG
bd17544fa5
Fix the kafka panic when sending the message to a closed channel (#25116)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-06-28 18:36:26 +08:00
Xiaofan
2273e2a45a
Limit the go routine numbers (#25171)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-06-28 10:54:46 +08:00
Xiaofan
c302f10635
Fix compaction stucked due to channel rebalance (#25098)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-06-27 10:35:47 +08:00
Xiaofan
83367219bb
Fix frequent stats log (#25094)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-06-25 20:35:46 +08:00
yihao.dai
9f3e421427
Check metric type at watchChannelTask (#24861)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-13 20:34:38 +08:00
yah01
022162ae6a
Fix deleted data is still visible (#24796)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-13 09:14:38 +08:00
yihao.dai
a03ebcff22
Check metric type before search (#24652) (#24716)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-08 21:38:36 +08:00
congqixia
9de97b7766
Fix confusing log while plugin is not enabled (#24735)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-08 16:06:36 +08:00
congqixia
5973f5aeb9
Use go-api/v2 for milvus-proto (#24723)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-07 21:32:37 +08:00
yihao.dai
4434d001fd
Assign metric type in search request when user has not explicitly assigned (#24583)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-01 13:44:35 +08:00
Gao
8da5b6a614
Disable search params rewrite while searching with filter (#24556)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-05-31 19:52:33 +08:00
Bingyi Sun
b341f43e68
Add default panic handler for ants pool (#24536)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-05-31 14:43:39 +08:00
xige-16
793864c12d
Clean tmp disk data when start milvus (#24400)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-05-26 14:13:27 +08:00
yihao.dai
bbb4d5d463
Revert "Retrieve segments concurrently (#24358)" (#24383)
This reverts commit a6b82733ad60bf3b0068af18c138650ad7e6239b.

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-25 19:45:28 +08:00
yihao.dai
a6b82733ad
Retrieve segments concurrently (#24358)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-25 09:57:27 +08:00
wei liu
266bf418d4
fix real time task num metrics (#24349)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-24 14:27:26 +08:00
yihao.dai
a9c05d1eb0
Support rate limit based on growing segments size (#24157)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-16 22:03:24 +08:00
chyezh
20054dc42b
Support configurable policy for query node, Add user level schedule policy (#23718)
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-05-16 10:55:22 +08:00
yah01
62cabbfce2
Support JSON data type (#23839)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-09 22:11:24 +08:00
cqy123456
f7159189fd
modify diskann memory estimation method (#23892)
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-05-09 20:29:09 +08:00
congqixia
bf8e6376db
Make querynode lazy load statslog if SkipBFLoad is true (#23904)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-08 14:20:41 +08:00
Gao
21762d9c23
Seperate reading build and load params for diskann (#23722)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-04-27 10:54:36 +08:00
yah01
1a773b8db1
Refine the merge algorithm (#23721)
For given two search tasks t1, t2:
- same topk: merge them if nq1 + nq2 <= MaxNQ
- otherwises: merge them if the merge cost doesn't exceeded merge ratio

the merge cost is defined as:
total cost - min(cost1, cost2)

Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-27 09:52:36 +08:00
congqixia
5aad58d318
Fix shard detector rewatch always returns closed channel (#23734)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-26 19:26:39 +08:00