sparknack
9b4081e110
enhance: cachinglayer: some performance optimization (#42858)
issue: #41435
We compared the performance using the modified test_sealed.cpp, which
randomly accesses all rows in all chunks and counts the number of runs
within 3s.
## performance data comparison (ops/second)
chunk config: 1x1000
| Field Type | w/o cachinglayer (commit 640f526301) | w/ cachinglayer |
w/ cachinglayer + opt |
|---|---|---|---|
| Bool field | 82428 | -63.6% (29983) | +2.7% (84675) |
| Int8 field | 82228 | -63.3% (30166) | +2.4% (84163) |
| Int16 field | 82572 | -63.8% (29867) | +1.8% (84036) |
| Int32 field | 82797 | -63.7% (30031) | +1.5% (84043) |
| Int64 field | 81077 | -62.9% (30107) | +0.6% (81604) |
| Float field | 82678 | -63.4% (30266) | +1.8% (84146) |
| Double field | 81925 | -63.4% (29974) | +0.2% (82097) |
| Varchar field | 19933 | -19.6% (16027) | +18.9% (23690) |
| JSON field | 16519 | -96.8% (533) | +2.5% (16927) |
| Int array field | 7325 | -13.7% (6321) | -1.4% (7220) |
| Long array field | 6347 | -8.9% (5781) | -0.1% (6344) |
| Bool array field | 8275 | -14.0% (7116) | +0.4% (8311) |
| String array field | 2281 | -5.0% (2168) | +0.2% (2287) |
| Double array field | 6427 | -13.3% (5574) | -2.0% (6301) |
| Float array field | 7291 | -13.0% (6346) | -1.5% (7183) |
| Vector field | 27487 | -40.4% (16371) | -4.7% (26192) |
| Float16 vector field | 49773 | -54.6% (22601) | -5.9% (46834) |
| BFloat16 vector field | 49783 | -53.1% (23350) | -5.7% (46934) |
| Int8 vector field | 63871 | -59.0% (26179) | -6.2% (59926) |
---
chunk config: 10x1000
| Field Type | w/o cachinglayer (commit 640f526301) | w/ cachinglayer |
w/ cachinglayer + opt |
|---|---|---|---|
| Bool field | 3659 | -48.6% (1879) | +110.1% (7686) |
| Int8 field | 3410 | -45.3% (1864) | +123.9% (7636) |
| Int16 field | 3647 | -48.6% (1874) | +110.1% (7661) |
| Int32 field | 3647 | -48.8% (1866) | +109.6% (7645) |
| Int64 field | 3645 | -48.9% (1863) | +107.8% (7573) |
| Float field | 3647 | -49.0% (1861) | +109.5% (7639) |
| Double field | 3640 | -45.1% (1998) | +108.4% (7586) |
| Varchar field | 1594 | -23.9% (1213) | +20.6% (1922) |
| JSON field | 1202 | -26.5% (884) | +16.1% (1396) |
| Int array field | 602 | -12.3% (528) | +12.7% (678) |
| Long array field | 529 | -12.2% (465) | +7.5% (569) |
| Double array field | 537 | -13.0% (467) | +6.4% (571) |
| Vector field | 1520 | -37.9% (943) | -5.5% (1437) |
| Float16 vector field | 2607 | -47.0% (1382) | +6.4% (2774) |
| BFloat16 vector field | 2586 | -46.5% (1383) | +8.8% (2813) |
| Int8 vector field | 3101 | -47.3% (1633) | +41.9% (4400) |
---------
Signed-off-by: Shawn Wang <shawn.wang@zilliz.com>
2025-07-17 11:20:51 +08:00
..
2025-06-12 20:46:36 +08:00
2025-07-17 11:20:51 +08:00
2025-07-16 17:46:51 +08:00
2025-07-02 22:18:44 +08:00
2025-05-10 09:19:16 +08:00
2024-12-11 11:32:44 +08:00
2025-06-17 15:22:38 +08:00
2025-06-12 20:46:36 +08:00
2025-06-12 20:46:36 +08:00
2025-01-07 13:26:55 +08:00
2025-06-04 09:58:39 +08:00
2024-08-23 12:35:02 +08:00
2024-12-26 10:32:49 +08:00
2024-11-19 15:48:40 +08:00
2023-04-26 10:14:41 +08:00
2025-06-12 20:46:36 +08:00
2025-06-17 15:22:38 +08:00
2023-05-12 14:11:20 +08:00
2025-07-11 19:04:49 +08:00
2023-05-12 14:11:20 +08:00
2025-06-12 20:46:36 +08:00
2025-01-13 09:58:57 +08:00
2025-04-02 18:46:20 +08:00
2025-05-10 09:19:16 +08:00
2025-06-12 20:46:36 +08:00
2025-07-02 22:18:44 +08:00
2025-07-02 22:18:44 +08:00
2025-04-28 10:52:40 +08:00
2025-07-02 22:18:44 +08:00
2025-07-02 22:18:44 +08:00
2025-03-04 11:56:02 +08:00
2024-08-02 15:40:14 +08:00
2024-03-11 14:45:02 +08:00
2025-06-17 10:08:38 +08:00
2025-04-30 14:22:58 +08:00
2025-06-17 15:22:38 +08:00
2025-05-27 18:28:27 +08:00
2025-06-12 20:46:36 +08:00
2025-07-16 17:46:51 +08:00
2025-07-02 22:18:44 +08:00
2025-06-12 20:46:36 +08:00
2024-10-25 15:25:30 +08:00
2024-06-22 09:38:02 +08:00
2024-09-30 13:23:32 +08:00
2025-06-12 20:46:36 +08:00
2025-07-07 14:24:46 +08:00
2025-06-12 20:46:36 +08:00
2025-06-27 14:44:42 +08:00
2025-06-12 14:38:35 +08:00
2025-06-17 15:22:38 +08:00
2025-06-10 11:06:35 +08:00
2025-01-17 15:55:03 +08:00
2025-06-17 15:22:38 +08:00
2023-12-26 11:04:47 +08:00
2025-06-12 20:46:36 +08:00
2025-06-17 15:22:38 +08:00
2025-06-12 20:46:36 +08:00
2025-06-24 16:32:42 +08:00
2025-06-24 16:32:42 +08:00
2025-06-24 16:32:42 +08:00
2025-04-30 08:48:54 +08:00
2025-05-08 10:12:54 +08:00
2023-06-25 14:38:44 +08:00
2025-04-16 17:14:33 +08:00
2024-03-21 11:15:20 +08:00
2025-06-03 14:42:31 +08:00
2024-07-19 14:01:39 +08:00
2025-07-14 20:34:50 +08:00
2024-08-02 19:32:46 +08:00
2024-08-02 19:32:46 +08:00
2025-04-09 10:18:26 +08:00
2025-06-12 20:46:36 +08:00
2025-06-12 20:46:36 +08:00
2025-06-12 20:46:36 +08:00
2025-04-30 08:48:54 +08:00
2023-03-08 11:03:51 +08:00
2023-03-10 09:47:54 +08:00
2024-04-26 12:01:26 +08:00
2025-07-16 17:46:51 +08:00
2023-03-10 09:47:54 +08:00
2024-09-30 13:23:32 +08:00
2025-06-12 20:46:36 +08:00
2024-12-16 19:40:45 +08:00
2025-04-28 10:52:40 +08:00
2025-07-16 17:46:51 +08:00
2025-07-02 22:18:44 +08:00
2025-04-28 10:52:40 +08:00
2023-08-31 20:07:00 +08:00
2025-04-28 10:52:40 +08:00
2025-05-20 13:26:23 +08:00
2025-06-18 00:42:38 +08:00
2025-06-12 20:46:36 +08:00
2025-06-25 16:52:42 +08:00
2025-07-04 10:10:45 +08:00
2025-05-30 10:18:36 +08:00
2023-03-10 09:47:54 +08:00
2025-07-10 17:32:48 +08:00
2025-04-16 17:14:33 +08:00
2025-05-08 14:28:52 +08:00
2025-06-12 14:38:35 +08:00