Scale and performance metrics Crunchy - stolostron/search-v2-operator GitHub Wiki
Data obtained running Locust jobs on cluster. All response times are in milliseconds.
July 7
User queries (search-api)
- 5 users (5 per second)
- LIMIT of 1000 resources
With 20 small managed clusters on Crunchy DB (July 7)
| Environment - operation | # reqs | # fails | Avg | Min | Max | Median | req/s | failures/s |
|---|---|---|---|---|---|---|---|---|
| GP2 storage | ||||||||
| searchByCount | 109 | 0(0.00%) | 139 | 79 | 523 | 120 | 0.22 | 0.00 |
| searchByFilter | 99 | 0(0.00%) | 181 | 135 | 338 | 160 | 0.20 | 0.00 |
| searchByKeyword | 119 | 0(0.00%) | 171 | 119 | 570 | 150 | 0.24 | 0.00 |
| searchComplete | 128 | 0(0.00%) | 102 | 66 | 760 | 77 | 0.26 | 0.00 |
| searchRelatedCount | 115 | 0(0.00%) | 2120 | 1829 | 3253 | 2100 | 0.23 | 0.00 |
| searchRelatedItems | 102 | 0(0.00%) | 2132 | 1790 | 3392 | 2100 | 0.20 | 0.00 |
| Aggregated | 672 | 0(0.00%) | 785 | 66 | 3392 | 160 | 1.35 | 0.00 |
With 100 small managed clusters on Crunchy DB
max connections set to 100 (July 7)
| Environment - operation | # reqs | # fails | Avg | Min | Max | Median | req/s | failures/s |
|---|---|---|---|---|---|---|---|---|
| GP2 storage | ||||||||
| searchByCount | 67 | 0(0.00%) | 820 | 227 | 2851 | 640 | 0.13 | 0.00 |
| searchByFilter | 68 | 0(0.00%) | 200 | 77 | 636 | 180 | 0.14 | 0.00 |
| searchByKeyword | 75 | 0(0.00%) | 300 | 118 | 831 | 260 | 0.15 | 0.00 |
| searchComplete | 64 | 0(0.00%) | 564 | 274 | 1558 | 500 | 0.13 | 0.00 |
| searchRelatedCount | 71 | 0(0.00%) | 7603 | 3997 | 19633 | 7600 | 0.14 | 0.00 |
| searchRelatedItems | 71 | 0(0.00%) | 8192 | 3743 | 16931 | 8000 | 0.14 | 0.00 |
| Aggregated | 416 | 0(0.00%) | 3001 | 77 | 19633 | 580 | 0.83 | 0.00 |
without max connections set to 100 (July 8)
| Environment - operation | # reqs | # fails | Avg | Min | Max | Median | req/s | failures/s |
|---|---|---|---|---|---|---|---|---|
| GP2 storage | ||||||||
| searchByCount | 76 | 0(0.00%) | 649 | 247 | 2327 | 480 | 0.15 | 0.00 |
| searchByFilter | 68 | 0(0.00%) | 171 | 77 | 627 | 140 | 0.14 | 0.00 |
| searchByKeyword | 92 | 0(0.00%) | 298 | 126 | 769 | 240 | 0.18 | 0.00 |
| searchComplete | 82 | 0(0.00%) | 553 | 288 | 1526 | 480 | 0.16 | 0.00 |
| searchRelatedCount | 62 | 0(0.00%) | 7592 | 3581 | 19691 | 6900 | 0.12 | 0.00 |
| searchRelatedItems | 71 | 0(0.00%) | 7055 | 3572 | 17998 | 6600 | 0.14 | 0.00 |
| Aggregated | 451 | 0(0.00%) | 2451 | 77 | 19691 | 490 | 0.91 | 0.00 |
With indexes on edges table for sourceid and destid
| Environment - operation | # reqs | # fails | Avg | Min | Max | Median | req/s | failures/s |
|---|---|---|---|---|---|---|---|---|
| GP2 storage | ||||||||
| searchByCount 150 | 0(0.00%) | 344 | 218 | 1289 | 290 | 0.30 | 0.00 | |
| searchByFilter 114 | 0(0.00%) | 104 | 63 | 517 | 85 | 0.23 | 0.00 | |
| searchByKeyword 108 | 0(0.00%) | 179 | 113 | 497 | 160 | 0.22 | 0.00 | |
| searchComplete 130 | 0(0.00%) | 366 | 291 | 841 | 330 | 0.26 | 0.00 | |
| searchRelatedCount 116 | 0(0.00%) | 569 | 199 | 1184 | 600 | 0.23 | 0.00 | |
| searchRelatedItems 122 | 0(0.00%) | 600 | 202 | 1446 | 590 | 0.24 | 0.00 | |
| Aggregated 740 | 0(0.00%) | 364 | 63 | 1446 | 320 | 1.48 | 0.00 |
Replace all_edges view with Union between edges table and materialized view (MV) to store intercluster edges (with all 3 indexes)
composite unique index(on sourceid, destid, edgetype) and index on sourceid and destid
| Environment - operation | # reqs | # fails | Avg | Min | Max | Median | req/s | failures/s |
|---|---|---|---|---|---|---|---|---|
| GP2 storage | ||||||||
| searchByCount 123 | 0(0.00%) | 327 | 213 | 1054 | 290 | 0.25 | 0.00 | |
| searchByFilter 122 | 0(0.00%) | 102 | 59 | 677 | 82 | 0.24 | 0.00 | |
| searchByKeyword 119 | 0(0.00%) | 169 | 117 | 434 | 150 | 0.24 | 0.00 | |
| searchComplete 135 | 0(0.00%) | 377 | 293 | 806 | 330 | 0.27 | 0.00 | |
| searchRelatedCount 123 | 0(0.00%) | 571 | 225 | 1255 | 600 | 0.25 | 0.00 | |
| searchRelatedItems 114 | 0(0.00%) | 610 | 223 | 1329 | 620 | 0.23 | 0.00 | |
| Aggregated 736 | 0(0.00%) | 358 | 59 | 1329 | 310 | 1.47 | 0.00 |
with MV to replace all_edges view and composite unique index(on sourceid, destid, edgetype)
| Environment - operation | # reqs | # fails | Avg | Min | Max | Median | req/s | failures/s |
|---|---|---|---|---|---|---|---|---|
| GP2 storage | ||||||||
| searchByCount | 83 | 0(0.00%) | 743 | 220 | 2854 | 580 | 0.17 | 0.00 |
| searchByFilter | 88 | 0(0.00%) | 168 | 69 | 535 | 150 | 0.18 | 0.00 |
| searchByKeyword | 75 | 0(0.00%) | 313 | 125 | 1375 | 280 | 0.15 | 0.00 |
| searchComplete | 59 | 0(0.00%) | 662 | 295 | 2266 | 570 | 0.12 | 0.00 |
| searchRelatedCount | 74 | 0(0.00%) | 7011 | 3028 | 22454 | 6100 | 0.15 | 0.00 |
| searchRelatedItems | 70 | 0(0.00%) | 7729 | 3316 | 18058 | 6900 | 0.14 | 0.00 |
| Aggregated | 449 | 0(0.00%) | 2670 | 69 | 22454 | 560 | 0.90 | 0.00 |
With MV and composite unique index(on sourceid, destid, edgetype) and index on sourceid and destid
| Environment - operation | # reqs | # fails | Avg | Min | Max | Median | req/s | failures/s |
|---|---|---|---|---|---|---|---|---|
| GP2 storage | ||||||||
| searchByCount | 123 | 0(0.00%) | 341 | 214 | 1036 | 300 | 0.25 | 0.00 |
| searchByFilter | 140 | 0(0.00%) | 89 | 63 | 238 | 81 | 0.28 | 0.00 |
| searchByKeyword | 121 | 0(0.00%) | 167 | 110 | 551 | 150 | 0.24 | 0.00 |
| searchComplete | 138 | 0(0.00%) | 369 | 297 | 775 | 330 | 0.28 | 0.00 |
| searchRelatedCount | 120 | 0(0.00%) | 234 | 66 | 898 | 240 | 0.24 | 0.00 |
| searchRelatedItems | 128 | 0(0.00%) | 223 | 66 | 766 | 240 | 0.26 | 0.00 |
| Aggregated | 770 | 0(0.00%) | 237 | 63 | 1036 | 240 | 1.54 | 0.00 |
Best times seems to be with materialized view (MV) and 3 indexes - composite unique index(on sourceid, destid, edgetype) and index on sourceid and destid