Old Functional Programming - RooyyDoe/functional-programming GitHub Wiki

These are the first steps I made into this course and this all changed when I got further into the coding and saw what I actually needed.

Create a JS file and put every query into a variable

I created my main.js file and assigned my queries and endpoint to a variable so that I can use them later in my code.

// Const variable with the API endpoint in it.
const url = 'https://api.data.netwerkdigitaalerfgoed.nl/datasets/ivo/NMVW/services/NMVW-33/sparql';

// Const variable with a query from sparQL in it.
const query = `
		PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
		PREFIX dc: <http://purl.org/dc/elements/1.1/>
		PREFIX dct: <http://purl.org/dc/terms/>
		PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
		PREFIX edm: <http://www.europeana.eu/schemas/edm/>
		PREFIX foaf: <http://xmlns.com/foaf/0.1/>

		SELECT ?superCategory  WHERE {
				<https://hdl.handle.net/20.500.11840/termmaster2802> skos:narrower ?superCategory .
		}`;

Re-build my old fetch function to an async function

I took my code fetch code from frontend-applications and tried to get it to work in this project. I found out that whenever I tried to recieve the data I was getting a pending response and I did not know how to work around this.

function runQuery(url, query){
	return fetch(url+'?query='+ encodeURIComponent(query) +'&format=json')
		.then(res => res.json()) 
		.then(json => {
			return json.results.bindings;
		});
	}

Making use of IIFE for testing

I needed to work around this and started converting my query into a async await function. In this way my code was more readable and Thijs Spijker explained that I could use IIFE to Immediately Invoked Function Expression. (Runs as soon as it is defined).

(async () => {
	// makes variables with rawData results of continent and categories
	let rawCategoryResults = await runQuery(url,query);
	let rawContinentResults = await runQuery(url,query2);
})();

// This async function will get all the data out of the database.
async function runQuery(url, query){
	// This will fetch the URL/Query and get the json data and put it in a response
	let response = await fetch(url+'?query='+ encodeURIComponent(query) +'&format=json');
	// when the response happened it will get turned to json and put in another variable
	let json = await response.json();
	// Here you return the rawData so it can be used out of the async function.
	return json.results.bindings;
}

It isn't possible to invoke an async function on top level code and await can't be used inside a normal function. This is mainly to test if my functions are working. When I have written down all my functions and I have cleaned my data the exact way I want it, I want to rewrite this and make it a normal Async function instead of an IIFE.

Results:

Creating a clean up function that cleans my rawData

Now I needed to clean up my data so I would get an array full with every URI of the continents and the main categories. So I needed to make a clean up function for this. At first I started to make a foreach that went through the rawData. The foreach did not work as I wanted it to work.

	let cleanResults = []
	// rawResults.forEach(function (rawResult) {
	// 	for(let key in rawResult) {
	// 		cleanResults.push(rawResult[key].value);
	// 	}	
	// 	// console.log(rawResults[0].categoryName.value, index);
	// });

	// return cleanResults;

Adding if statement to turn string into integer when datatype is integer.

So when this did not work how I wanted it too. I found out Reduce did exactly what I needed. This method executes a reducer function on every element of the array and resulting in a single output value. As initial value I added an empty array that will get filled with all the cleaned values out of the rawData.

// This function will clean up the rawData results.
function cleanData(rawResults) {
	// Using the reduces function to go through every result of the rawData. and make a new 
	// array of it.
	return rawResults.reduce((cleanResults, rawResult) => {
		// looping through key of the results
		for(let key in rawResult) {
			// if the datatype is an integer it will use a parseInt to make the value a real
			// Integer. and after that pushes it to cleanResults.
			if (rawResult[key].datatype === "http://www.w3.org/2001/XMLSchema#integer") {
				let parsed = parseInt(rawResult[key].value, 10);
				cleanResults.push(parsed);
			// Pushes the results into cleanResults
			} else cleanResults.push(rawResult[key].value);
		}
		// return results so it can be used outside the function
		return cleanResults;	
	// initial value (array with all the new data)
	},[]);

Creating a function that counts up the objects to one integer.

This function takes every key in the object and of this key it will put the value into a new array (cleanResults). This function can be re-used on every query that I have, but I also needed to build-in an if statement in the for loop. I will get back on this later in the progress when I am starting to count up all the category objects.

I wanted to look if it was possible to count up all the objects to a total value. For this I made another Reducer function and used the example on the reducer MDN page where they use a reducer function to count up different values in an array. Because I already made this it went pretty easy and I had no problems except it only made an array and not a integer.

// Function that counts up all the sub category objects and gives back one total number
function countCategoryResults(results) {
	// Reduce with an accumulator and the current The returned value will 
	// be in current and it will get count up by the totalObject, like a foreach
	return results.reduce((totalObjects, currentObjects) => {
		// returns totalObject add up by currentObjects
		return totalObjects + currentObjects;
	});
}

Trying to get this working to get all my results and link them to each other(Foreach)

Also added the last query that I made where I count every object in different categories.

SELECT (COUNT(?category) AS ?categoryAmount) WHERE {
  
       <https://hdl.handle.net/20.500.11840/termmaster3> skos:narrower* ?continent .
  	   ?obj dct:spatial ?continent .
  
  	   <https://hdl.handle.net/20.500.11840/termmaster2803> skos:narrower* ?category .
       ?obj edm:isRelatedTo ?category .
  	   ?category skos:prefLabel ?categoryName .
  	   
} GROUP BY ?categoryName

I needed to turn the string into an integer with parseInt and this is where I used the if statement in cleaningData code. It checks if there is a key.value with datatype and then if it sees that there is an integer in this datatype it changes it into an integer instead of a string and then adds it into a new array. When this is done the countUp function will run over this and add up every integer in this array and make a final value and show this.

After doing this I want to loop through a query and link all the values to each other. I was trying to do this with template literals Thijs Spijker showed me how this worked and how I could use this. At first I needed to make a foreach that looped trough the continents and inside this another foreach that looped through every category.

// foreach that loops over all the cleaned continent results
	cleanedContinentResults.forEach(async continentUri => {
		// foreach that loops over all the cleaned category results
		cleanedCategories.forEach(async categoryUri => {
			// Using template literals to put the results of the foreach into the query
			let catQuery = `
				PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
				PREFIX dc: <http://purl.org/dc/elements/1.1/>
				PREFIX dct: <http://purl.org/dc/terms/>
				PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
				PREFIX edm: <http://www.europeana.eu/schemas/edm/>
				PREFIX foaf: <http://xmlns.com/foaf/0.1/>
	
				SELECT (COUNT(?category) AS ?categoryAmount) WHERE {
					
					<${continentUri}> skos:narrower* ?continent .
						?obj dct:spatial ?continent .
	
					<${categoryUri}> skos:narrower* ?category .
						?obj edm:isRelatedTo ?category .
						?category skos:prefLabel ?categoryName .
					
			} GROUP BY ?categoryName`;
			// Variable with the results of the query that I made with the template literals
			let catResults = await runQuery(url, catQuery );
			// Cleaning the catQuery with my cleaning function
			let cleanResult = cleanData(catResults);
                        console.log(cleanResult);
		});
	});

QueryResult

When I am running this code it returns everything in the console, even empty arrays. I want to clean this up with an if statement so that I get a better overview of the data that I am getting in return.

I wanted to filter all the empty arrays out of this clean result. I did this with an if statement that only shows the arrays with more than one value in it. It is still not perfect but it is a big step. After that made a string with template literals in it to give a overview for what I have right now.

if (cleanResult.length > 0) {
	let finalCatResult = countCategoryResults(cleanResult);
	// Presentation of what I have right now. with template literals.
	console.log(`Continent ${continentUri} has ${finalCatResult} in categorie ${categoryUri}`);
}

BetterQuery

This function is getting a count of the objects that are in the main category and the sub/sub-sub categories. And again I am using a .then chain in this function to clean the data and look if it is from the datatype integer and if that is true it will use parseInt to turn it into an real integer.

When this function is invoked it will return different arrays with only integers in it. I want to count up all of this to make only one value. And then I want to add this to the category uri.

In this function I need to loop trough every continent and in every continent I need to foreach through every category. So I can get all the objects from every category as result. I am doing this with this piece of code:

I am also making a promise.all that is going to wait till the for..in and the foreach is done running and after that it will put the results into a new array promises. Over this data I am using the count up function that I made and will explain a bit further in the process.

The bottom five arrays with the numbers are the total count of every category in every continent. I am doing this through a map and reduce function.

This is the function that I am using to count up all the objects in the different categories and put them in one value.

This is going to be the last steps that I need to take. After that I also need to look into d3.js if they have a format that calculates percentage other ways I need to make a function for that as well.