We processed SVG 1.1 Second Edition Test Suite with Macaw, parsed and serialized into new SVG files, then converted these new SVG files and reference SVG into PNGs and compared them with ImageMagick and RMSE metric. To convert from SVG we used Google Chrome in both cases and defined it to be a reference/baseline renderer.

Test PASSES only if Diff value is less than 6%.

Macaw 0.9.0 Results

Data for Parse -> Render in Macaw -> Save Image -> Compare with reference cycle:

Name	Percentage (%)	Absolute Value
Total amount of tests	100%	521
Passed	11%	56
Failed	89%	465
Failed to parse	29%	151

Data for Parse -> Serialize -> Render SVG in Chrome -> Compare with reference cycle:

Name	Percentage (%)	Absolute Value
Total amount of tests	100%	521
Passed	16%	82
Failed	84%	439
Failed to parse	29%	150

How To Run SVG 1.1 Test Harness

First you will need to checkout task/test-svg-coverage branch from Macaw main repository.

Install prerequisites:

• ImageMagick

brew update && brew install imagemagick

• Python 2.7

brew install python

• Selenium and ChromeDriver

brew install chromedriver
pip install selenium

Running Test Harness

1. Go to Macaw/MacawTests/scripts and do sanity check to make sure that everything is OK:

./sanity-check.sh

Fix errors if there are any. Script will also try to fix possible problems, e.g. create missing folders.

There are two ways to calculate coverage:

1. For Macaw Rendering
2. For Macaw Serialization (Google Chrome Rendering)

Results data is always stored in data.csv file with the following format:

color-prof-01-f-manual,18213.2,0.277915
color-prop-01-b-manual,14368.2,0.219245

To Run Test Suite with Macaw Rendering

1. Open Macaw project in Xcode and run execute MacawSVGTestSuiteWithViews.
2. Run diff-all.sh script in Macaw/MacawTests/scripts folder:

cd Macaw/MacawTests/scripts
./diff-all.sh

To Run Test Suite with Chrome Rendering

Open Macaw project in Xcode and run execute MacawSVGTestSuite. TODO: Finish script to automate this step.