Case Study: A Download Manager - deanrad/rx-helper GitHub Wiki

Case Study: A Download Manager

Let's say that in a front-end app, a user can download any of several files that they see in a list, placing them into the browser's Local Storage.

However, you want to program it so that they download only one at a time in a queue. A simple enough scenario, but like anything, tricky if you don't have the right tools. Here's a screen capture, showing how the network requests only occur once the previous download is done, and how Local Storage saves items as they arrive, after a forced delay for demo purposes:

Download Manager GIF

We'll get going on this using RxJS, RxHelper, and the power to name events of our app whatever we want.

Events Need Names

First a quick recap on Observables: An Observable is just a data structure of a set of values over time. Just like a salary is an agreement for money over time.

But if your app has Observables of values, what's in those values? Rx Helper suggests they are events. Not browser or server events - just events of significance for your application. Our downloader has a start, and a complete event, as we leave off errors for the sake of demonstration. And just like in Redux, these event objects (aka Actions) contain a field in which you can place a well-chosen name for what the event contains.

Let's use the name download/start for these events which - you guessed it - signify the start of the download of a file! We namespace it with slashes (/) for organizational purposes, knowing there are multiple events of a download. This scheme will work for our tutorial, though Rx-Helper can work with any event-naming scheme you can throw at it.

Now, let's roll up our sleeves!

Fake It Till You Make It

Before you even write any HTML, you can define a simulated Observable of download requests for testing purposes:

const months = {0: 'January', 1: 'February', 2: 'March', 3: 'April', 4: 'May', 5: 'June', 6: 'July',
                7: 'August', 8: 'September', 9: 'October', 10: 'November', 11: 'December'}

// A simulated series of download start requests
let download$ = interval(1000)      // Every second
   .map(i => months[i])             // becomes a month and we
   .map(month => ({                 // package it in an object
        type: 'download/start',
        payload: { file: month + '.csv' }
    })
    .take(3)                        // For only 3 months

Let's break it down:

This Observable contains the data for 3 download requests, packaged in Flux Standard Actions, 1 second apart, for files January.csv, February.csv, and March.csv.

But, why simulate? The ability to simulate is a powerful thing! Not only does this help with testing, it lets the rest of your system not care exactly how those events came to be. Abstraction, for the win! But I suppose you're now saying "I don't need a simulation, I need the real thing."

Getting Real Time

Okay, okay.. So, given some markup like the following:

<a class="downloadable" data-month="0">Download January</a>
<a class="downloadable" data-month="1">Download February</a>

Here's how you can populate an Observable from real button clicks:

// A series of user-initiated download start requests
let download$ = fromEvent(document, 'click', '.downloadable')
   .map(e => months[e.target.attribues['data-month']])
   .map(month => ({
        type: 'download/start',
        payload: { file: month + '.csv' }
    })

Real enough for ya? But you just proved a point about the power of Observables. Whether simulated or real, a single variable download$ now contains a series of requests to download files over time. Once we have it, where the events came from becomes unimportant! If it's not in the payload, it wasn't deemed to matter. But what's more significant is what's not there...

Notice how we have no async/await logic. We simply have a variable that represents 'Download requests over time'. This variable will soon be "handled" in a way that creates AJAX requests so that we can get their results, and configure any queueing or other timing variations. But let's not get too far ahead of ourselves..

Every Action Has Consequences

We don't know yet how we will get the bytes of the file, but with Rx Helper, that shouldn't stop us from naming the event that carries the bytes (and original filename) in its payload. How about: download/complete?

Now, let's imagine that upon a download/complete event we want to put a file in local storage and log to the console.

Here's a way the Rx-Helper Agent will allow:

let download$ = (simulated or real download requests)
let result$ = (simulated or real results of downloads)
// TODO create result$ from download$ 

agent.on('download/complete', ({ action }) => {
  const { filename, bytes } = action.payload
  localStorage.set(filename, bytes)
  console.log('Saved ' + filename)
})

agent.subscribe(result$)

It reads:

Tell the Agent that when a download/complete event occurs, to pluck the fields we need from it and write to localStorage and the console. Then tell the agent to use result$ as a source of events.

Now that is simple! The on handler takes us back to the simplicity of JQuery. Our event names (the type field) are our own, not a framework's. And this code will work in browsers or NodeJS, if we use a server localStorage implementation, or simply change that line to do something more server-like.

Handle It!

The function in the middle is called a Handler. It's where you'll write the code to be run upon a new occurrence of the event. Rx-Helper will have you place anything with consequences in a Handler function attached via on, or filter.

Comparing Fujis To Honeycrisps

The raw RxJS way is not much longer, just a it makes use of a tap - which Functional Programming fans and Rubyists use to signify doing something "off to the side", like a tap coming off a keg. You don't need braces around action in this version:

result$.tap(action => {
  const { filename, bytes } = action.payload
  localStorage.set(filename, bytes)
  console.log('Saved ' + filename)
}).subscribe()

After calling tap to specify the "side-effect" (which Rx Helper calls simply a consequence), we have a new Observable that will save upon every event. Finally, the call to subscribe() begins pulling items through the saving-enhanced Observable. Observables are like boulders sitting at the top of a hill - one must subscribe() to them to get them rolling. Each tap or map ties a new tin can on a string behind the boulder, and a single subscribe makes it all noisily roll down!

Map To The Point!

Now let's get real about another part: how to get download/complete events populated with the file's bytes via AJAX.

In Functional Programming, changing one object into another is called mapping (as opposed to tapping from before). So when our tech-lead asks what we're doing we can sound smart by saying:

I'm just mapping download/start events to download/complete events by making AJAX calls.

That'll impress them enough to get them off our backs for a second. Now we can actually figure it out. After a Pomodoro or two, we'll have:

agent.on('download/start', ({ action }) => {
   const { filename } = action.payload
   return ajax({ url: 'download.php?file=' + filename })
      .map(result => ({
         type: 'download/complete',
         payload: {
            bytes: result.response,
            filename
         } 
      })
}, { concurrency: 'serial', processResults: true })

agent.subscribe(download$)

Let's unpack this, shall we? We're telling our friendly helper Agent to create an Ajax request each time we see a download/start event; the consequence will be putting the AJAX request out on the wire.

The filename we're downloading is in the event payload, and we'll use it in a url like download.php?file=January.csv. The ajax method comes from RxJS, and we call it with our url, giving us a return value, that is an object with a response property. We map this object to an event object containing the filename and the bytes that came back, and tell the agent we want it to process these results, serially (explained shortly).

Finally our agent is made aware of the download$ requests, in the call to agent.subscribe. Internally, the agent calls subscribe on its argument to get the ball rolling, which is what allows the handlers to fire. Cool, eh? It may be strange stuff the first time you see it, but it's ultimately just a 3 step process.

It's All In The Timing

Lastly, the operational behavior (aka: timing) is controlled simply by the string 'serial'. Notice there are no local variables to keep track of in-progress downloads. In fact, not a single data structure of ours is needed, it is handled internally by RxJS! Brilliant. If we change our mind later, we simply change that string and our app runs very differently, though most of its code remains the same. That's the Real Ultimate Power of Rx Helper!

But Does It ConcatMap?

The way raw RxJS lets you specify timing is a bit different and in my opinion, a bit harder to read. This was the main motivation for Rx Helper to be created. See for yourself:

let result$ = download$.concatMap(action => {
   const { filename } = action.payload
   return ajax( /*  same AJAX stuff as before */ )
})

result$.tap(action => {
   // save to localStorage
}).subscribe()

Here we are creating an Observable chain as we call concatMap on download$, followed by tap on result$. It all gets rolling when you call subscribe just once.

concatMap is how you specify serial in raw RxJS, and it's there on line 1. The act of returning the Observable is the same in the Rx Helper and raw Rx version, but while the goal in raw RxJS is to create a single chain and call subscribe once, the goal in RxHelper is to create isolated renderers, and call subscribe multiple times implicitly, usually via processResults: true in the config argument.

Congratulations, you have made it! Let's see what this looks like in the end:

Definition of Done

This code needs a fork in it - it's now done. Here's our code fully simplified by Rx Helper:

let download$ = fromEvent(document, 'click', '.downloadable')
   .map(e => months[e.target.attribues['data-month']])
   .map(month => ({
        type: 'download/start',
        payload: { file: month + '.csv' }
    })

agent.on('download/start', ({ action }) => {
   const { filename } = action.payload
   return ajax({ url: 'download.php?file=' + filename })
      .map(result => ({
        filename,
        bytes: result.response
      }))
}, { concurrency: 'serial', type: 'download/complete' })

agent.on('download/complete', ({ action }) => {
  const { filename, bytes } = action.payload
  localStorage.set(filename, bytes)
  console.log('Saved ' + filename)
})

agent.subscribe(download$)

We can see clearly for each renderer both a) what it is triggered by, and b) what it may trigger. You'll never have such grep-pable codebases as when using Rx Helper, and we all know what a good thing that is!

Summary

Now that you see the power of Observables, combined with a good Helper Agent for using them, you may want to know if they're safe to use. The good news is that Observables, like Promises just a few years ago, are on their way to being standardized in the JavaScript lanaguage in this TC39 proposal. And RxJS is a mature, performant implementation of Observables, in use already by some of the biggest companies in tech, such as Google, Microsoft, and Netflix.

The Rx Helper library is based on a 2 year old library called the Antares Protocol, which has over 500 commits, a full test suite and several embedded client and server examples. It is the fastest way to get up to speed with RxJS.

These Open Source tools are yours for the using today! So waste no time checking them out. Thank you for your time, and if you've found this interesting, please share, clap, like, star, and let's get a discussion going. May your future coding endeavors be ever in your favor!

Concurrency Mode Reference

For ease of reference, here's a cheat sheet on the Rx Helper concurrency modes, and their correspondence to their raw RxJS concurrency modes:

Non-canceling modes

Behavior	RxJS	Behavior
parallel	`mergeMap/flatMap`	ASAP
serial	`concatMap`	queued

Canceling modes

Behavior	RxJS	Cancels/Prevents
cutoff previous	`switchMap`	oldest
mute new	`exhaustMap`	newest

And as a parting gift, some memorable use cases to help you recall when to use each mode:

parallel - Independent. Multiple "Like" Buttons.
serial - Queued. 1-at-a-time Download Manager (this article!)
cutoff - Renewable countdown. Session Timeout.
mute - Ignore repeats. An Elevator Call Button.