Palmes validation - davidlabee/Graph4Air GitHub Wiki

External Validation with Palmes Tubes

What are Palmes Tubes?

Palmes diffusion tubes are passive NO₂ samplers installed at static urban locations in the city of Amsterdam.

Purpose

We use the Palmes measurements to externally validate our GNN predictions of NO₂ concentration.

Method

  1. Spatial Join: For each Palmes location, we:

    • Buffer a radius (e.g., 50m)
    • Find all graph nodes within that buffer
    • Take the mean predicted NO₂ over those nodes

  2. Fallback: If no nearby nodes are found, we take the single nearest graph node’s prediction.

  3. Metrics:

    • RMSE
    • MAE
    • R²
    • Pearson correlation

Code Example

from sklearn.metrics import mean_squared_error, r2_score

actuals = palmes_gdf['mean_annual_palmes_no2']
preds = palmes_gdf['pred_mean']

rmse = np.sqrt(mean_squared_error(actuals, preds))
mae  = np.mean(np.abs(preds - actuals))
r2   = r2_score(actuals, preds)
pearson = np.corrcoef(actuals, preds)[0,1]

Results

  • GCN RMSE: ~7.8–8.5
  • GAT RMSE: ~7.7–8.1
  • Pearson: ~0.71–0.75

Visual Output

We provide scatter plots (measured vs predicted) and spatial plots of residuals. These confirm good agreement with external data.

External Validation with Palmes Tubes

What are Palmes Tubes?

Palmes diffusion tubes are passive NO₂ samplers installed at static urban locations. They absorb ambient nitrogen dioxide over multiple weeks and are later analyzed in lab to get accurate NO₂ concentrations. In Amsterdam, such tubes have been used by official monitoring agencies and are considered reliable ground-truth measurements.

Purpose

We use the Palmes measurements to externally validate our GNN predictions of NO₂ concentration, to ensure generalization to real-world field data.

Method

  1. Spatial Join: For each Palmes location, we:

    • Buffer a radius (e.g., 50m)
    • Find all graph nodes within that buffer
    • Take the mean predicted NO₂ over those nodes

  2. Fallback: If no nearby nodes are found, we take the single nearest graph node’s prediction.

  3. Metrics:

    • RMSE
    • MAE
    • R²
    • Pearson correlation

Code Example

from sklearn.metrics import mean_squared_error, r2_score

actuals = palmes_gdf['mean_annual_palmes_no2']
preds = palmes_gdf['pred_mean']

rmse = np.sqrt(mean_squared_error(actuals, preds))
mae  = np.mean(np.abs(preds - actuals))
r2   = r2_score(actuals, preds)
pearson = np.corrcoef(actuals, preds)[0,1]

Results

  • GCN RMSE: ~7.8–8.5
  • GAT RMSE: ~7.7–8.1
  • Pearson: ~0.71–0.75

Visual Output

We provide scatter plots (measured vs predicted) and spatial plots of residuals. Unknown