3.1.5 Non-Functional Requirements

The Demo 3 quality requirements are grouped under Performance, Scalability, Security, Reliability and Availability, and Maintainability. Each requirement is quantified and mapped to an architectural tactic, an executable test, and a recorded result.

QR1: Performance

QR1.1 Supported-load performance

Quantified requirement	Architectural tactic	Executable test	Target	Actual result	Verdict
Non-recommendation read endpoints support a 10-user baseline and a 20-user peak for 5 minutes each.	Region co-location, connection pooling, stateless backend	k6 constant-VU performance test and comparison script	Both runs: p95 <= 2 s and error rate < 1%. Peak throughput >= 1.8x baseline.	Baseline: p95 368.382 ms, 0% errors, 6.965 requests/s. Peak: p95 522.876 ms, 0% errors, 13.103 requests/s. Throughput ratio: 1.881x.	Met

How the test works

The k6 test signs in once, starts a fixed number of virtual users, and rotates their authenticated requests through vaults, pantry, shopping lists, and recipes. Only these read requests are measured. The comparison script reads the sanitized 10-user and 20-user result files and checks both latency and error targets before calculating the throughput ratio.

Why these measures and targets were chosen

The 10-user run represents the normal baseline and the 20-user run represents the expected peak for the supported deployment. Five minutes measures sustained behavior instead of a short traffic burst.

p95 latency is the time within which 95% of measured requests finish. The 2-second target sets the longest acceptable wait for nearly all supported-load requests without allowing a few extreme requests to define the result. The recorded values mean that 95% of baseline requests finished within 368.382 ms and 95% of peak requests finished within 522.876 ms.

An error rate below 1% requires more than 99 of every 100 requests to succeed. Both runs recorded 0% errors. Throughput is the average number of completed requests per second. Doubling users should produce close to double the completed work, so the minimum ratio is 1.8x, allowing 10% scaling overhead. The measured ratio of 1.881x means throughput increased by 88.1%, from 6.965 to 13.103 requests per second.

QR2: Scalability

QR2.1 High-load resilience

Quantified requirement	Architectural tactic	Executable test	Target	Actual result	Verdict
Non-recommendation read endpoints remain available for 100 concurrent users for 5 minutes without a service crash or manual recovery.	Cloud Run autoscaling, stateless backend, managed restart	k6 constant-VU resilience test	At least 99% successful responses; no service crash; no manual recovery. Latency is observed, not thresholded.	99.934% successful responses; 19,603 passed checks and 13 failed checks. The run completed without a service crash or manual recovery. Observed p95 latency: 903.709 ms.	Met

Latency increases at high load because the deployment is capped at three Cloud Run instances for cost control.

How the test works

The resilience profile starts 100 constant virtual users for 5 minutes. Each user performs the same four authenticated read operations with one second of pacing. The test checks the successful-response rate and records latency without applying the 2-second performance threshold.

Why these measures and targets were chosen

One hundred users is five times the supported 20-user peak, so this test measures resilience beyond normal operating load. The 99% success target allows at most one failed response per 100 while still requiring the service to remain usable. The measured 99.934% success rate means only 0.066% of checks failed.

No latency threshold is applied because this test measures continued service, not supported-load speed. The observed p95 of 903.709 ms records the degradation for comparison while keeping it separate from the QR1 performance claim. Completing the full run without a crash or manual recovery shows that the service remained operational under the higher load.

Stress-test observation

The paced 1,000-user recipe storage and retrieval test completed with no failed operations. Observed p95 latency was 16.603 s for storage and 15.239 s for retrieval. This is a stress-test observation and is not used as evidence for the supported-load latency target.

Each virtual user creates one temporary private recipe, repeatedly retrieves that recipe, and removes the test data during cleanup. Startup is spread over 240 seconds to avoid sending all creation requests at once.

QR3: Security

ID	Quantified requirement	Architectural tactic	Executable test	Target	Actual result	Verdict
QR3.1	Protected endpoints reject missing and invalid tokens.	JWT filter and deny-by-default authorization rules	k6 authorization matrix	100% of authorization checks pass.	154 of 154 checks passed; 0 authorization failures.	Met
QR3.2	Authenticated users cannot access data owned by another user.	Ownership checks in the service layer	Backend service and integration tests	100% of ownership tests pass.	109 ownership tests passed.	Met
QR3.3	No secrets are committed to the repository.	CI secret scanning	Gitleaks	0 verified secrets	Latest recorded result: 0 findings.	Met
QR3.4	Backend and engine container images contain no known high or critical vulnerabilities.	Dependency and container-image scanning	Trivy	0 high or critical findings	Latest recorded result: 0 findings.	Met
QR3.5	The public and authenticated staging API contain no unresolved medium, high, or critical security findings.	Input validation, security headers, limited exposure	ZAP GET-only public and authenticated API scan	0 unresolved medium, high, or critical findings	0 unresolved medium, high, or critical findings.	Met

How the tests work

Test	Method
Authorization matrix	Sends requests without a token and with an invalid token to every protected route. Each request must return HTTP 401 or 403. A smaller valid-token control set must return HTTP 200.
Ownership tests	Use authenticated requests and service calls to attempt access to another user's data. The service must reject each attempt and integration tests must expose the rejection as HTTP 403.
Gitleaks	Scans the repository history during CI and fails when a verified credential or secret is detected.
Trivy	Scans the built backend and recommendation-engine container images during CI and fails on any high or critical vulnerability.
ZAP	Builds GET-only public and authenticated scan definitions from the staging OpenAPI document. The evaluator combines both reports, removes sensitive data, and fails if any unresolved medium-or-higher finding remains.

Why these measures and targets were chosen

Authorization and ownership use a 100% target because one incorrectly exposed protected route can disclose or change another user's data. The 154 of 154 authorization checks show that every tested missing-token, invalid-token, and valid-token case produced its expected result. The 109 ownership tests cover the rule that authenticated users may access only their own data.

Secret, container, and API security targets are zero because a single verified secret, high or critical container vulnerability, or unresolved medium-or-higher API finding creates a release risk. Gitleaks and Trivy recorded zero findings.

QR4: Reliability and Availability

ID	Quantified requirement	Architectural tactic	Executable test	Target	Actual result	Verdict
QR4.1	The production API remains available on demo day and after a reasonable period after demo day (keeping within free-tier cost contraints).	Health checks, managed restart, external monitoring	UptimeRobot keyword monitor	>= 99.9% availability	Latest recorded monitored availability: 99.97%.	Met
QR4.2	After one successful synchronization, recipes, vaults, pantry items, and shopping lists remain readable when the backend is unreachable or the token has expired.	Viewer-scoped SQLite cache, repository fallback, persisted viewer identity	Flutter repository and integration tests	4 of 4 supported collections remain readable; 0 failed cached reads	Passing cache tests cover all 4 supported collections.	Met
QR4.3	Offline operation does not permit writes or expose another user's cached data.	Read-only offline state, mutation interceptor, viewer-scoped composite cache keys	Flutter widget, repository, mutation-interceptor, and cache-isolation tests	0 enabled offline mutations; 0 cross-user reads	Passing tests reject offline mutations before HTTP and keep cache records isolated by viewer.	Met

How the tests work

Test	Method
Availability monitor	UptimeRobot calls the production health endpoint every 5 minutes. A check passes only when the response contains "status":"UP"; availability is calculated over the required 30-day period.
Offline cached reads	Repository tests first store successful online responses for recipes, vaults, pantry items, and shopping lists. They then simulate an unavailable backend and verify that each repository returns data from the active viewer's cache.
Offline safety	Mutation-interceptor tests verify that writes are rejected before reaching the HTTP adapter. Cache tests store matching record IDs for different viewers and verify that each viewer can read only their own records.

Why these measures and targets were chosen

The 99.9% availability target permits no more than about 43 minutes of downtime in a 30-day period. An exact 30-day window makes repeated results comparable. The recorded 99.57% is 0.33 percentage points below the target, so it is marked Not met.

The engine test uses 20 requests to show that fallback behavior is repeatable rather than a single successful case. The 3-second limit prevents an unavailable dependency from leaving the user waiting, and zero unhandled HTTP 500 responses requires every failure to be controlled.

The offline-read target is 4 of 4 because recipes, vaults, pantry items, and shopping lists are the four collections promised for offline use. Offline writes and cross-user reads both have a target of zero because either result could corrupt data or expose another user's personal information.

QR5: Maintainability

ID	Quantified requirement	Architectural tactic	Executable test	Target	Actual result	Verdict
QR5.1	Backend application logic maintains automated test coverage.	Controller-service-repository layering and dependency injection	JaCoCo in CI	>= 80% coverage	97% backend coverage	Met
QR5.2	Frontend application logic maintains automated test coverage.	Repository abstractions and provider overrides	Flutter test with lcov in CI	>= 80% line coverage	88% frontend coverage	Met

How the tests work

Test	Method
Backend coverage	Maven runs the backend tests and JaCoCo measures application instructions, branches, and methods. CI fails if any enforced coverage measure falls below 80%.
Frontend coverage	Flutter runs the automated test suite with lcov. Generated files are removed from the report before CI checks that line coverage is at least 80%.
Accessibility guidelines	Static-Landing page, Brand-guide, User Manual scaned with Google Lighthouse's accessibility tool.

Why these measures and targets were chosen

The 80% coverage threshold is the enforced project baseline for both backend and frontend application logic. It is high enough to require broad automated testing while allowing exclusions for generated files and classes with no application logic. The recorded 97% backend and 88% frontend values are the portions of measured code covered by the automated tests; they are not test pass rates.

NFR Test Evidence within the Repo under nfr-tests/

Evidence	Location
Supported-load test	nfr-tests/performance/non-recommendation-load.js
Supported-load comparison	nfr-tests/performance/compare-results.js
10-user baseline result	nfr-tests/results/performance-baseline-10vu.json
20-user peak result	nfr-tests/results/performance-peak-20vu.json
Performance comparison result	nfr-tests/results/performance-comparison.json
100-user resilience result	nfr-tests/results/resilience-100vu.json
Authorization test	nfr-tests/security/authorization-matrix.js
Authorization result	nfr-tests/results/authorization-matrix.json
Recipe concurrency test	nfr-tests/performance/recipe-concurrency-load.js
1,000-user paced result	nfr-tests/results/recipe-concurrency-1000vu-paced.json
ZAP preparation and evaluation tools	nfr-tests/security/zap/

Screenshot evidence can be found in Mealchemy/diagrams/NFR-Screenshots.pdf

---

Last updated: 30 September 2026