Porting Project Data to QGIS: June/ July 2016

Vector Data

Start of port of mapping data from mapping application previously used, Mapmaker Gratis. Mapmaker Gratis is a free version of proprietary software which was used for several years, but now perceived to lack the functionality required to extend the mapping function to provide representation of other types of project data. The vector files were exported from Mapmaker Gratis via dxf files, from the proprietary dra files used by Mapmaker to the universally-used ESRI shapefiles recognised by QGIS. In many ways, dra files are more compact than shapefiles and can contain mixed point, line and polygon features whereas shapefiles are specific to a particular type of vector data, with each so-called shapefile being a group of three files which must be kept together. In general, the porting of the vector data was successful with some re-plotting carried out when a particular feature did not appear correctly.

Vector data includes the following layers:

Pits: position of the excavations (polygonal)

Structures: representation of any upstanding structures on site, such as buildings and walls, or underground such as drains (polygonal)

Surfaces: representation of hard surfaces encountered on site, such as paving, cobbles, concrete paths (polygonal)

Boundaries: representation of site boundaries, between land compartments or between land.water (line).

Raster/ Image Data

Following the import of vector data, image data, including the basemap and pit image data were carried across and re-georeferenced where necessary in QGIS. One problem which arose was the generation of very large raster files which proved inconvenient when archiving and sharing the project online. The files were shrunk to reasonable sizes without any apparent loss of quality in the image by using Raster>Conversion>Translate (convert format). The conversion input box requires selection of the Input Layer (via the Browse.. button), selection of the Output file (via the Browse.. button to select the directory and then either selecting or inputting the output file). To perform the translation to a smaller file size, check the 'Creation Options' box and from the dropdown options menu, choose 'High compression' - it was found that 'JPEG' compression produced undesirable artefacts in the resulting image.

Each pit image is a square or rectangular image taken above the pit in plan view. When georeferenced the image appears as rotated quadrilateral on a white or black background with the overall dimensions defined by the vertices of the original image. When displayed on top of the vector layer defining the pits, the excess white/black background obscures an area outside the pit to which the image applies. To deal with this, a selective transparency has been defined for the pit image, available through the 'Properties>Transparency' tab for the raster image layer. Within the tab, under 'Custom transparency options', click add key, '+', and under each of the pixel values enter '0' (zero) to indicate 'white'. Click 'add' again and on the next line enter '255' to indicate 'black'. This should trim off the border, leaving visible the rotated original image - if preferred the 'global transparency' can also be adjusted using the global transparency slider.

Experiments in Display of Pottery Distribution Data: August/September 2016

Experiments in the representation of the distribution of pottery finds as raster overlays to the Millhams map. Analysis of the pottery finds has resulted in the recognition of medieval pottery arising from the lowest layers of the trial pits which contain ancient timbers shown by dendrochronology to date to the 14th century and overlain and surrounded by water-borne gravels which yielded the pottery finds. A typology analysis was conducted in some detail on the pottery, and in the first instance overall summary results were supplied for mapping comprising number of sherds and total weight of the medieval pottery associated with each trial pit.

In order to produce a raster indicating the relative density of pottery finds, in the first instance the areas of each pit were calculated as an attribute of the 'Pits' layer using $area function. The resulting vector layer was then exported to a point vector layer using the Vector>Geometry>Centroids function to associate each pit with the coordinates of its centroid as a basis for plotting pottery distributions via new point layer named 'Pottery Distribution'.

Additional attributes of 'total_num' (total number of sherds of medieval pottery) and 'total_wt' (total weight of medieval pottery) were introduced and initially data were entered manually from a pottery analysis spreadsheet. Later, the spreadsheet was edited and the edited version imported via a CSV file containing the columns 'Pit no.', 'Total_sherds' and 'Total_wt', using 'Add text-delimited layer' to produce an attribute layer containing no geometry, entitled 'Pottery Summary'. (Note that in preparing the CSV file, it is also necessary to prepare a companion CSVT file in a text editor to define the formats for each of the columns - text, integer, real - otherwise the import defaults to text.) Then a 'join' was enabled in the 'Pottery Distribution' Properties by selecting 'Pit no.' from the 'Pottery Summary' and 'label' from the 'Pottery Distribution' as the join fields and selecting the fields 'Total_sherds' and 'Total_wt' as the attributes to be joined. (It was important to ensure that features are called by exactly the same descriptor name is used in both the 'label' and 'Pit no.' fields, otherwise the join will not work for features with mismatched names.)

In the 'Pottery Distribution' attribute table, a third attribute,'density' was was derived by dividing 'total_wt' by the 'area' attribute' of the corresponding pit. A raster indicative of the pottery density distribution was produced using Raster>Interpolation and selecting IDW (Inverse Distance Weighting' as the interpolation method, to produce the raster 'Density of Pottery Distribution'.

Links to External Data: October 2016

Each trial pit in the total excavation has a quantity of external data associated with it, primarily finds records and drawings of vertical pit sections, which are not susceptible to a plan view on a map. For these records it is desired to link geographic points on the Millhams map with further data contained in documents essentially separate from mapping project.

This was achieved via the 'Properties>Actions' menu of the 'Pits' layer after first setting up a text attribute field in the 'Pits' layer which defines the path to to file where the appropriate external document is stored. In the first instance, links were created to pdf files intended to show the N-, S-, E- and W-facing sections of the individual test pits. To maintain project portability, the external documents are stored within the 'AssociatedDocs' sub-folder of the main project folder, 'Millhams_QGIS'. An issue which became apparent was that QGIS creates absolute paths to files linked by the 'Open' action type, so that the link no longer worked when the project file was shared with other team members via Dropbox. The issue was solved by setting up the action type as 'Python' and inserting the following Python code into the action statement:

# Import the required libraries
import sys
from os.path import abspath, dirname, join

# Get an instance of the project object
proj = QgsProject.instance()

# Get the name of the project file
urfile = str(proj.fileName())

# Build the absolute path to the file to be displayed

path = join(abspath(dirname(urfile)), "[% "Section_W" %]")

# Determine the OS we're running on and open the file to be displayed
if sys.platform == "linux" or sys.platform == "linux2":
    import subprocess
     # Convert the backslashes to slashes
    path = path.replace('\\', '/')
    opener = "xdg-open"
    subprocess.call([opener, path])
elif sys.platform == "win32":
    from os import
startfile
    startfile(path)
else:
    # Can put OSX specific code here
    None

This script obtains the path to the folder containing the QGIS project and concatenates this with the remainder of the path defined to the document in the sub-folder. It has been tested to work with both Windows and Linux. "Section_W" in the script refers to the attribute storing the path to the document showing the west-facing section of the excavation; it should be replaced with other attribute names as appropriate.