Server Side Data Flow - GlimmerLabs/MathOverImages GitHub Wiki

When a user goes to a page on the MIST website, a number of things happen on the web server to generate the needed page content. These page requests are handled by Node.js, which uses the file app/routes.js to determines what to do based off the url of the requested page.

This page walks through how to read the page request process as a new MIST developer, who may not be familiar with Node.js or website backend work.

Let's say we want to know what happens when a user looks at a random gallery page, accessed from the url http://glimmer.grinnell.edu/gallery/random. We start at the top of app/routes.js. The first chunk of code is a bunch of require lines. One of them is this:

var gallery = require("../functions/gallery.js");

So whenever gallery.someFunction is used in routes.js, we're actually using a function from the file /functions/gallery.js. That will probably be useful soon.

And it is! We search for “gallery/random” in routes.js, and find this:

app.get('/gallery/random', function(req, res) {
  gallery.buildRandomPage(req, res, database);
});

Which is a little weird. We don't quite know what app.get is, but let's say that it basically means “if your rout is '/gallery/random', then call this function and do something with the return” That's my understanding at least. When a user goes to the route '/gallery/random', we call the function buildRandomPage from /functions/gallery.js.

Now we jump over to /functions/gallery.js and look at what buildRandomPage does:

module.exports.buildRandomPage = (function(req, res, database) {
  filedatabase=database;
  module.exports.getRandomImages (9, function(images, error){
    if(error) {
      res.redirect("/404");
    }
    else {
      // userid is either logged in user or nil
      // add user like info to each image, then render page
      var userid = (req.session.user) ? req.session.user.userid : null;
      setLikes(images, userid, function(imageArray, errorArray){
        res.render('gallery',{
            user: req.session.user,
            images: imageArray,
            currentPage: req.params.pageNumber,
            type: "random"
          }
        );
      });
    }
  });
});

The first thing to notice is that buildRandomPage uses the callback design technique common throughout the MIST codebase. We call the function getRandomImages, and pass an anonymous function, or a callback, that details everything else to do. So we can guess that getRandomImages gets a set of random images, and then the callback calls setLikes using the resulting images. The function setLikes is yet another function that takes a callback. We can guess that setLikes modifies the data of the images from getRandomImages to add something that indicates if the current user has liked them or not, and then the callback renders the resulting page using res.render.

The function buildRandomPage outlines the entire process, but each step is executed as a series of callbacks within the nested functions. Now that we know our general trajectory, so we can look at each called function in tern.

buildRandomPage calls getRandomImages:

module.exports.getRandomImages= (function(count, callback) {
  filedatabase.query("SELECT images.*, users.username FROM images NATURAL JOIN users ORDER BY RAND() LIMIT " + count + ";", function(rows, error){
    callback(rows, error);
  });
});

getRandomImages gets 9 random images from the database, then executes the callback. The callback calls setLikes:

var setLikes = function(imageArray, userID, callback) {
  if(imageArray.length == 0)
    callback([]);
  else {
    var imageArrayClone = imageArray.slice(0, imageArray.length);
    var errorsArray = [];
    var likes = {counter: 0, likes: []};
    // for each image, check database to see if user has liked image
    // save true or false for each image in likes array
    // when all images checked, copy likes to the liked filed of each image
    // execute callback with modified imageArrayClone
    for(var image = 0; image < imageArrayClone.length; image++) {
      // use the image counter as var currentIndex
      (function(currentIndex){
        filedatabase.hasLiked(userID, imageArrayClone[currentIndex].imageid, function(liked, error) {
          likes.likes[currentIndex] = liked;
          likes.counter++;
          if(likes.counter == imageArray.length) {
            for(var i=0;i<likes.likes.length;i++) {
              imageArrayClone[i].liked = likes.likes[i];
            }
            callback(imageArrayClone, errorsArray);
          }
          errorsArray.push(error);
        });
      })(image);
    }
  }
}

setLikes is a bit of a mess, but you can probably convince yourself after squinting a bit that we make a copy of the images from getRandomImages, and change the .liked field of each image to true/false depending on if the current user has liked the image. We then execute a callback with the modified images. The callback calls res.render with a number of arguments, including our modified image array. What does res.render do?

The codebase is not helpful at this point. We know that res is the result, and that it's passed in when we first call gallery.buildRandomPage from routes.js. So render is a method of the object res. I made a guess that this was some kind of Node.js thing and googled it. It seems that res.render is a Node.js feature, and it a complies an embedded javascript file (a .ejs file) to HTML, using the provided information. The first call to render is “gallery”, so we can guess that there's a file named gallery.ejs to compile and send to the user. If we search for gallery.ejs we find it in the /views directory, with all the other *.ejs files. In our res.render call we pass in our modified image array as images, so it's good to see that in gallery.ejs, the contents of images is used to render images on the page.

We now know that res.render creates an HTML page with our content. Then what happens?
res.render returns the HTML to setLikes
setLikes returns the HTML to getRandomImages
getRandomImages returns the HTML to buildRandomPage
buildRandomPage returns the HTML to app.get
app.get renders the page for the user. The end.