Programming for Everybody: Assignment 13.1 Web Services and XML - edorlando07/datasciencecoursera GitHub Wiki

Extracting Data from XML

In this assignment you will write a Python program somewhat similar to http://www.pythonlearn.com/code/geoxml.py. The program will prompt for a URL, read the XML data from that URL using urllib and then parse and extract the comment counts from the XML data, compute the sum of the numbers in the file.

We provide two files for this assignment. One is a sample file where we give you the sum for your testing and the other is the actual data you need to process for the assignment.

Sample data: http://python-data.dr-chuck.net/comments_42.xml (Sum=2553)
Actual data: http://python-data.dr-chuck.net/comments_340269.xml (Sum ends with 42)

You do not need to save these files to your folder since your program will read the data directly from the URL. Note: Each student will have a distinct data url for the assignment - so only use your own data url for analysis.

Data Format and Approach The data consists of a number of names and comment counts in XML as follows:

<comment>
  <name>Matthias</name>
  <count>97</count>
</comment>

You are to look through all the tags and find the values sum the numbers. The closest sample code that shows how to parse XML is geoxml.py. But since the nesting of the elements in our data is different than the data we are parsing in that sample code you will have to make real changes to the code.

To make the code a little simpler, you can use an XPath selector string to look through the entire tree of XML for any tag named 'count' with the following line of code:

counts = tree.findall('.//count')

Take a look at the Python ElementTree documentation and look for the supported XPath syntax for details. You could also work from the top of the XML down to the comments node and then loop through the child nodes of the comments node.

###Sample Execution

$ python solution.py 
Enter location: http://python-data.dr-chuck.net/comments_42.xml
Retrieving http://python-data.dr-chuck.net/comments_42.xml
Retrieved 4204 characters
Count: 50
Sum: 2...

###Actual Code

import urllib                       #imports urllib library
import xml.etree.ElementTree as ET  #imports xml ElementTree library 
finaltotal = list()                 #sets up a list that will be used
                                    #to append all the numbers found
                                    #in the xml file

url = 'http://python-data.dr-chuck.net/comments_340269.xml'
                                    #sets up a variable for the
                                    #url to be passed through and
                                    #opened up later in the code

uh = urllib.urlopen(url)
data = uh.read()                    #reads the xml file from the
                                    #url above
print data

Some of the print data output is listed below:

<?xml version="1.0" encoding="UTF-8"?>
<commentinfo>
  <note>This file contains the actual data for your assignment - good luck!</note>

  <comments>
    <comment>
       <name>Cormak</name>
       <count>99</count>
    </comment>
    <comment>
       <name>Sylvain</name>
       <count>95</count>
    </comment>
    <comment>

The rest of the code continues below:

tree = ET.fromstring(data)
listA = tree.findall('comments/comment')
print "Count: " + str(len(listA))

The print statement above produces the following output: This count is the number of times a comment appears in the xml file.

Count: 50

The rest of the code continues below:

for item in listA:
    total = int(item.find('count').text)   #sets up loop to find the key/value
                                           #pair that is count:number
    finaltotal.append(total)               #appends the integers found in the
                                           #key value pair and creates a list
                                           #that can be aggregated later.

print "Sum: " + str(sum(finaltotal))       #sums up the integers found in the
                                           #key/value pair <count>95</count>

###The final output produces the following result.

Sum: 2542
⚠️ **GitHub.com Fallback** ⚠️