Transforming - HappyPantss/frontend-data GitHub Wiki

Go to:

• Survey data
• RDW data

Survey data

The first tries were difficult for me. I had to load the JSON file in my JS/HTML.

With help from Jordy I got it to work using Node.js. The code is used to make it work looked like this:

let survey = require('./data.json');
var oogKleur = survey.map(function (el) {return el.oogKleur});

console.log(oogKleur);

This code ensured I got the eye color data from the json file in my text editor's terminal. I was happy it worked, but I wanted it to show in the console of the browser. So I waited till Robert would explain how to do this, without using Node.js. I followed the lecture, but Robert couldn't figure it out. He was coding live so he tried some things, but it didn't work unfortunately. The next day Laurens had a lecture where he explained what he knew about running a local server without using Node.js. He used some code that set up a python local server, and this worked! It was a bummer that the command he used, didn't work for Windows 10, only for Mac OS. So I searched up how to do so on Windows 10. Happily I found this Stackoverflow post about it, and they used the command py -m http.server, which worked for me! So I decided to share this in the Microsoft Teams team of the course (Tech Track - JavaScript Support).

Last

Because I worked with the local HTTP python server, let survey = require('./data.json'); did not work anymore. So I had to use let surveyData = data. data is the name of the array with all the data in it, you can find in the file data.js (which has sensitive information). The data looks somewhat like this:

let data =
    [
      {
        "geboortedatum": "[dd-mm-yy]",
        "geboorteplaats": "[latitude, longitude]",
        "hoeveelheidBroers": "[?]",
      }
    ]

Functional programming in my code

.map

I used the following code to map and replace the data to valid hex codes. The data contained some dirty values like hex codes without a hashtag, some random white spaces, RGB codes and strings like 'blue'.

// Arrow function eye colors
const replaceAll = surveyAnswers.map(answer => answer[columnName]
    .replace("#", "")
    .replace(" ", "")
    .toUpperCase()
    .replace("BRUIN", hexBrown)
    // Replace blue and lightblue for 1 hex
    .replace("BLAUW", hexBlue)
    .replace("LICHT", "")
    .replace("GROEN", hexGreen)
    // Replace RGB for hex
    .replace("RGB", "")
    .replace(".", ",")
    .replace(rgbCode, (rgbToHex(139,69,19))) // Give the numbers to rgbToHex.. 139 = r, 69 = g, 19 = b
    .replace("", "#")
);

.filter

I filtered the data to see which colors were most common.

Code:

let showBrown = replaceAll.filter(answer => answer === "#" + hexBrown);
let showBlue = replaceAll.filter(answer => answer === "#" + hexBlue);

RDW data

Changing the data

To make perfectly use of the data, I have to change it. If I console.log the whole dataset it looks like this:

After filtering them to only see the startdataarea data:

As you can see, the values give you the year, month and day. But if I want to convert that into my chart, it won't work. Because I only want the years. So I used the following code, to clean the data to something usefull.

// Returns all values for a certain key in an array of data
let filterData = (dataArray, key) => {
  return dataArray.map((item) => item[key].slice(0, 4));
};

This code gets the filtered data, so all the years, and slices everything but the first 4 letters/ numbers. Outcome: Here you can also see the unique years. That's nice to know, because that are the years that will show in my pie chart!

Making a new array

As shown in Fetching I used a function to loop through the array of years. If there is a object with an empty value, we change it to 0. If there is a value, we push it into a new array, and add them together to see how many there are.

// Get data from dataSource 2 (GEO PenR) and count all the years in the array
getData(dataSource2).then((RDWData) => {
  
  // Thanks to @Razpudding / Laurens
  let aantalPerJaarArray = [];
  locationArray.forEach((jaar) => {
    // if the array includes a empty value, change it to 0
    // else make a new array with all the years + number of values
    if (aantalPerJaarArray.find((item) => item.year == jaar) == undefined) {
      aantalPerJaarArray.push({
        year: jaar,
        value: 0,
      });
    }
    // Count for every found item 1 to the value
    aantalPerJaarArray.find((item) => item.year == jaar).value += 1;
  });
});

Outcome: