QGIS & PreEA tool runthrough - datasciencecampus/WorldPop-Malawi-fork GitHub Wiki

Notes on running through the PreEA tool based on the Malawi Workshop from November 2025

MP through:

Ex001 - simple QGIS settings set-up
Ex002 - merge divided roads - can't run as do not have access to ArcGIS plugin required to run 'merge divided roads'. May be a QGIS/coding option if this step is necessary?
Ex003 - data preparation for the boundary files ( 2018 EA and national border) and OSM data (roads, water, rail). Some of these pre-processing steps could be coded I would have thought, as just requires some simple filtering Total features for 2018 EAs = 18,779
Ex004 - further data preparation and pre-processing. A lot of tasks were around editing road, water and rail OSM data to create the 'uncrossable boundaries' features, e.g. editing water course to follow satellite image, editing road network to capture roads missed from OSM). This looks to be reliant on user knowledge and interaction (though could there be a way to load in another more accurate layer and automate the process, though suspect still requires user knowledge to check).
Ex005 - intro to preEA tool - seems to run fine. Exercise 5.3 (Salima) - different output to that given in the example, and log file different to the example shown also, despite using same parameters
- Log file does not run through iterations as shown in screenshot, and has some warning messages for feature ID 182
- Total number of features in PreEA is 8 (example has 80)
- Not to do with eliminating small areas
- Potential solution by changing the parameter for the 'maximum population for output tracts'. In the example it says to put 3000, but the screenshot has it set to 300. Changing to 300 produced a similar output to the one given in the example, though not exact same (70 PreEAs instead of 80).
- I don't think this is a big problem, as the guide says if you play around with parameter values then the output will change. It is just a bit confusing that inputting the same parameters and using the same data as the example is giving a different output...
Ex006 - Eliminate small areas
- Error message when trying to run 'eliminate small errors' from the PreEA plugin:
  - 2026-02-12T15:04:11 WARNING Traceback (most recent call last): File "C:\Users/plummm/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\preea_processing\preea_processing.py", line 261, in procResultElim self.runAlg(EliminateEAs,uiParams,None,'preEA Eliminate',outputs) File "C:\Users/plummm/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\preea_processing\preea_processing.py", line 345, in runAlg preEAAlg.processAlgorithm() File "C:\Users/ajph/AppData/Roaming/QGIS/QGIS3\profiles\preEA/python/plugins\preea_processing\preea_processing_eliminate.py", line 165, in processAlgorithm UnboundLocalError: cannot access local variable 'holeids' where it is not associated with a value

The error is, during the eliminate small areas code, a script is called call preea_processing_eliminate.py, in which there is the variable 'holeids'. According to copilot, this is established when the input file has 'holes' in the data. If there are no holes in the input data, then 'holeids' is not created and the code continues to try and use it but fails.

One solution to debug is to edit the code in 'preea_processing_eliminate.py', however this file doesn't actually exist in my PreEA installation, but a .pyc file does (which I can't edit). There is then a solution to 'decompile' the .pyc into a .py file but I can't get this to work currently.

One other peculiarity is that in the QGIS error, the 'preea_processing_eliminate.py' file it is calling is in a user profile which does not exist on my machine ("C:/Users/ajph/..."). Which copilot diagnoses as the the user profile of someone who has worked on a package on their machine. So this may be the cause of the problem?

Ex007 - Merge/split zones
- A shorter exercise where the tutorial demonstrates areas where you would merge areas where the Household population was too small, or split areas where the Household population was too high. But these were very manual steps rather than a particular preEA tool
Ex008 - preEA in rural areas
- Again a shorter exercise with a lot of manual pre-processing/preparation of a rural area, without running the preEA tool as such
Ex009 - apply preEA in rural area
- Run through of the preEA tool for a rural Malawi region, a similar procedure to the urban area run in exercises 5 and 6. Still getting the same error for the 'eliminate small areas' tool
Ex010 - Updating EAs
- Was just a runthrough of the preEA and eliminate small areas tool for the Salima and Kasungu districts again
Ex011 - enumeration effort tool
- A straightforward run-through of calculating the enumeration effort matrices for EAs within the Kasungu district

Full runthrough of urban and rural areas

Not sure if I have run this correctly because the outputs seem strange to me. Below are the breakdowns of total EAs produced from the preEA tool, run for Urban and Rural areas separately.

Two possible issues to explore:

folders contain filenames that are aligned to the Malawi boundary, which may raise the question if there is some misalignment of boundaries causing odd results
- The raster layer has a CRS different to the rest, but when you try to realign boundary to the Malawi CRS, it relocates the raster away from Malawi? So there may be something wrong here with the raster (all other input files are assigned to the recommended Malawi CRS)
the household raster file has data points on islands in Lake Malawi, but also in Tanzania. So there may be an issue with the population file being read into the preEA tool (see image below)

Filepaths and data used for initial PreEA tool runthrough:

Household raster file:

003_preEA_Application/005_UpdateEAs_al_Country/rural/HH_count_prediction_mean.tif

Rural

003_preEA_Application/005_UpdateEAs_al_Country/rural/001_Inputs/Malawi_OSM_Major_Roads.shp
003_preEA_Application/005_UpdateEAs_al_Country/rural/001_Inputs/Malawi_River.shp
003_preEA_Application/005_UpdateEAs_al_Country/rural/001_Inputs/Malawi_uncrossable.shp
003_preEA_Application/005_UpdateEAs_al_Country/rural/001_Inputs/Malawi_Dist_Rural.shp

Rural

003_preEA_Application/005_UpdateEAs_al_Country/urban/001_Inputs/Malawi_OSM_Major_Roads.shp
003_preEA_Application/005_UpdateEAs_al_Country/urban/001_Inputs/Malawi_River.shp
003_preEA_Application/005_UpdateEAs_al_Country/urban/001_Inputs/Malawi_uncrossable.shp
003_preEA_Application/005_UpdateEAs_al_Country/urban/001_Inputs/Malawi_Dist_Urban.shp

Update Ran the PreEA tool for Rural and Urban areas using the road and river datasets that indiciated they had been aligned to the relevant Coordinate Reference System for Malawi. These results suggest that many more EAs are produced, and there are many more EAs that are between 100-300 estimated households, but still a good number that are very small (< 1) or very large (> 300).

Urban EAs		Rural EAs
Districts	80	Districts	334

Total EAs produced	6911	Total EAs produced	26463
Total EAs with HH est below 1	358	Total EAs with HH est below 1	3544
Total EAs with HH est below 100	1313	Total EAs with HH est below 100	8946
Total EAs with HH est below 200	4438	Total EAs with HH est below 200	13971
Total EAs with HH est between 200-300	1879	Total EAs with HH est between 200-300	1481
Total EAs with HH est above 300	594	Total EAs with HH est above 300	1669

Original PreEA output

Urban EAs		Rural EAs
Districts	80	Districts	334

Total EAs produced	442	Total EAs produced	4438
Total EAs below 1	157	Total EAs below 1	2758
Total EAs below 100	213	Total EAs below 100	3120
Total EAs below 200	232	Total EAs below 200	3271
Total EAs between 200-300	16	Total EAs between 200-300	107
Total EAs above 300	194	Total EAs above 300	1059