12 Commits

Author SHA1 Message Date
wei liu
4987067375
enhance: Execute bloom filter apply in parallel to speed up segment predict (#33792)
issue: #33610

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-06-14 11:37:56 +08:00
jaime
0d3272ed6d
enhance: refine logs of cgo pool (#33373)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-05-27 19:06:11 +08:00
yihao.dai
760223f80a
fix: use seperate warmup pool and disable warmup by default (#33348)
1. use a small warmup pool to reduce the impact of warmup
2. change the warmup pool to nonblocking mode
3. disable warmup by default
4. remove the maximum size limit of 16 for the load pool

issue: https://github.com/milvus-io/milvus/issues/32772

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-27 01:25:40 +08:00
yah01
878c4c9463
enhance: limit the max pool size to 16 (#30371)
according to our benchmark, concurrency level 16 is enough to fully
utilize the object storage network bandwidth

Signed-off-by: yah01 <yang.cen@zilliz.com>
2024-01-31 14:13:06 +08:00
congqixia
88b4b8b77c
enhance: make segments SQPool & LoadPool resizable (#29239)
See also #29223

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-15 20:22:39 +08:00
yah01
d2f53aefa5
enhance: improve load speed (#28518)
This check rejects load request if running out the pool workers, but
small segment would be loaded soon, another segments would been loading
again after a check interval, which leads to slow loading for collection

Block the request by go pool

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-17 19:56:21 +08:00
yah01
90e2c63d9e
Fix getting incorrect CPU num (#28146)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-11-06 06:02:16 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
congqixia
8b5e276361
Use different cgo pool for SQ and other operations (#26021)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-01 09:19:05 +08:00
congqixia
efdd71c640
Make cgo pool size larger than worker pool size (#25318)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-05 16:56:26 +08:00
congqixia
31880ab427
Use conc.PoolOption instead of ants.Option (#24585)
- Add conc.PoolOption to setup conc.Pool
- Change panic default behavior
- Make future has error when job panicks

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-01 15:52:31 +08:00
congqixia
4fe363c4b2
Add KNN cgo pool (#23526)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-19 23:56:31 +08:00