Sarah Larimer's Work Report - spatial-data-discovery/sdd-2021 GitHub Wiki

Week 10 (Nov 1)

Summary: NetCDF File Format

What I did this week:

  1. Learned to create NetCDF files
  2. Quiz 9
  3. Discussion 9

What I will do next week

  1. Quiz 10
  2. Discussion 10

Question: What dimensions? Setting dimensions in a NetCDF file is nice because it helps to load and store data in a more optimized way. However it is not great if you don't know the size of your data ahead of time or if you predict it to change in more than one dimension.

Week 9 (Oct 25)

Summary: HDF5 File Format

What I did this week:

  1. Learned to create HDF5 files
  2. Quiz 7 and 8
  3. Discussion 8

What I will do next week

  1. Quiz 9
  2. Discussion 9

Question: What attributes? Attributes are an easy way to get information about a file without having to look at all the data.

Week 8 (Oct 18)

Summary: Fall Break/Geospatial Data Sciencee

What I did this week:

  1. Discussion on Geospatial Data Science. Looked into specifically the problem of variability in geospatial datatypes.
  2. Quiz 6

What I will do next week

  1. Quiz 7 and 8

Question: What's the challenge? In discussion 8, my group was tasked with looking at variability as one of the challenges in geospatial data science. One of the problems that we identified from the reading was that there are many data formats and datasets that are often difficult or impossible to integrate with each other. One of the potential solutions to this is to make a standard datatype that everybody uses. However, we realized this was a bad idea because each data type serves a specific purpose and is good for certain types of data. We wouldn't want to compromise on the data just to standardize things. Because of this, our solution was instead to make a good API that can support conversion between datatypes more easily. That way, data can be stored in the type that it is best supported, but easily integrated with other types after converted.

Week 7 (Oct 11)

Summary: Sparce Data Challenge

What I did this week:

  1. Sparce Data Challenge
  2. Discussion 7
  3. I was sick this week with a bad cold so I was unable to go to class

What I will do next week

  1. Quiz 6, I was unable to do it this week :(
  2. Quiz 7

Question: What's the process? For the sparce data challenge I first went the route that many did where I wanted to do some sort of nearest neighbors approach. But then I switched gears and instead of looking for the empty cells first and then finding their nearest neighbor, I found the cells with data that were next to empty cells and then populated the empty cells with the neighbor. I did this because I was having a hard time getting a nearest neighbors implementation to work. I realized that it would be slightly flawed in the sense that if there were a long stretch of empty cells then all of them would have the value of the leftmost cell, even when they were closer to the rightmost cell. But I think this would have also been a problem with a traditional nearest neighbors approach since filling in empty cells mean those become the nearest neighbor. I wasn unable to get my code fully functional but I think I had a pretty good outline of what I wanted to do .

Week 6 (Oct 4)

Summary: QGIS and ASCII data

What I did this week:

  1. Downloaded GQIS
  2. Figured out how to display ACSII data in QGIS
  3. Did discussion 6

What I will do next week

  1. Quiz 6
  2. Sparce Data Challenge

Question: How does it look? When visualizing my ASCII raster data, because we were using a layer map of the whole world I didn't want my cell size to be too big because I wanted it to at least sort of resemble real data, and if the cell size were too big it wouldn't seem like actual data because what would I be visualizing that has the large an area. I also played with the color of the base map and the colors of the cells to make a visualization where you could easily distinguish the map from the ascii data.

Week 5 (Sept. 27)

Summary: ASCII raster data

What I did this week:

  1. Discussion 5, modeling a day in the life of a typical college student
  2. ASCII raster demo
  3. Quiz 5
  4. Discussion 5

What I will do next week

  1. Download QGIS
  2. Explore visualizations in QGIS
  3. Quiz and discussion 6

Question: Space vs. Place Space to me is a large area that contains places. Space is the area in between those places. It is also relative and depending on the scale that you are looking at. For example, looking in my house places might be my room or the kitchen where space is the hallway in between. But looking at a country, places might be large cities whereas space is the area in between those.

Week 4 (Sept. 20)

Summary: Introduction/considerations to real-world spatial data and formats

What I did this week:

  1. Did the Google Earth activity to find spatial information on Paris
  2. Discussion 4
  3. Quiz 3 and 4
  4. Started sandbox 2

What I will do next week

  1. Quiz, journal, and discussion #5
  2. Finish sandbox 2
  3. Sandbox 3
  4. Explore ASCII

Question: What is spatial data

Spatial data is any data that contains information about where something is located in space. This can be geographical or where something is in relationship to something else. Spatial data takes many forms but the one that we explored this week is longitude and latitude as a method of associating a piece of data with a location.

Week 3 (Sept. 13)

Summary: Explored utility scripts

What I did this week:

  1. Uploaded my utility script to the repository
  2. Learned how to create my own branch and merge it back to master
  3. Recorded a podcast of me explaining my script and added it to the Google Drive
  4. Wrote a post for Discussion 3

What I will do next week

  1. Quiz 3
  2. Discussion 4
  3. Continue to familiarize myself with GitHub

Question: What is Utility?

After looking through my classmates scripts, I realized that utility has a broad definition. For example something could have utility if it helps solve a problem, or does some sort of calculation or task. But utility isn't always just about work. For some people, having utility could be just adding something fun into their life.

Week 2 (Sept. 6)

Summary: Became more familiar with the git workflow and how to add my files to the repository

What I did this week:

  1. Practiced pushing and pulling on git
  2. Became familiar with using github on browser instead of command line
  3. Completed sandbox 1 challenge
  4. Did quiz 1 and discussions 1 & 2

What I will do next week:

  1. Podcast/utility script
  2. More reading into spatial data

Question: What is spatial data?

Spatial data is data that has some spatial component to it. I've come to realize that this doesn't just have to be maps and geographical information. Spatial data is any data that contains information about where things are in space or where they are relative to others.

Week 1 (Sept. 1)

Summary: I made sure that I had everything in place to be successful in the class

What I did this week:

  1. Checked to see that I could use/access git on my computer
  2. Checked my version of python was up to date

What I will do next week:

  1. Do the class readings
  2. Work on sandbox 1
  3. Become more familiar with the git workflow

Question: Getting started

I feel prepared to take on this class! I have never used git or markdown before so I am a little nervous about being successful. But I really enjoy how these skills are things we will need after we graduate