HTML5 Client Code Technical Details - SpiderOak/so_client_html5 GitHub Wiki

Table of Contents Architectural Elements ContentNode Data Model Content Addressing Organization of Content Traversal Application Entry/Init Overview Facilities for Persisting Values Dynamic Page Traversal Case Study: Implementing a Special Node Type Internal Transmission of Special Content and Operational Modes Content Node Navigation Modes Storage and Share Rooms Operational and Incidental Development Details Development, release, and building the platform-specific versions Extend html5 with PhoneGap Plugins Incorporating Custom Animations Test and package iOS builds using build_platforms and Xcode Adjusting the jQuery Mobile styling, including the color schemes Special Browser Requirements Managing the UI Theme Maintaining our theme through the theme roller Interspersing theme swatches on pages Generating our basic theme, in the first place Special caution on reused content node pages Consequences of default caching of typical $.ajax() Complicated popup menu provisions

The ContentNode object hierarchy provides a data model around which obtaining, assimilating, and presenting the SpiderOak-managed user content data is organized - storage devices versus share rooms, and their contained folders and file "nodes".

• Node - Abstract basis for all nodes.
  • ContentNode - Abstract basis representing SpiderOak-managed content
    • RootContentNode - Consolidated storage and share roots container
    • FolderContentNode - General behaviors used by storage and share folders
    • FileContentNode - General behaviors used by storage and share files
    • StorageNode - Backups content
      • RootStorageNode
      • DeviceStorageNode
      • DirectoryStorageNode
      • FileStorageNode
    • ShareNode - Share Rooms and their contents
      • OriginalRootShareNode - List of share rooms published by an account
      • PublicRootShareNode - List of share rooms visited, in Anyone's tab
      • RoomShareNode - a Share Room, contains ShareNodes
      • FileShareNode - a file contained in a Share Room
    • RecentContentNode - recently visited items list, most recent first
  • PanelNode - abstract basis for console/UI panels
    • SettingsPanelNode - settings collection
      • RootSettingsPanelNode - root of the settings panel hierarchy
    • AccountPanelNode - details of the remote account

Content model elements are addressed by the URLs of their JSON content items:

• as the '#' fragment identifiers of the app's URL, for jQuery Mobile changePage navigation, and
• as the key for retrieval (and allocation) within the app's internal node database, mediated by the node_manager object.

The templates for these items, as well as the page structures for the root content nodes and settings panel pages, have addresses prefixed with "https://", and followed by:

• the storage root address for the logged-in account, for storage nodes
• the id of the div defining the template, for more static locations, like settings panels and the root of the non-original share rooms.

The respective content-specific roots are encompassed by a consolidated root, the RootContentNode.

• The actual content items are indexed in the node_manager by their JSON access URLs, and addressed in app URLs as the fragment identifiers.
  • The RootStorageNode contains the storage items, with a URL consisting of the storage server host (which can vary) and a trimmed, b32 encoding of the account name. Access to the contained items depends on login, at which point the storage host server is determined.
• The list of account-originated share rooms is collected within the OriginalRootShareNode. It has a special JSON address on the server within each account's storage root. The original share rooms are situated with all the public share rooms, in...
• The PublicRootShareNode has a URL that is the common stem of all visited SpiderOak share rooms.
• A encompassing root node, the RootContentNode, contains all the other roots, and presents their contents in a consolidated view. The app uses an arbitrary, internal address for it, sufficiently URL-like to be recognized as such by the jQm URL traversal machinery in changePage().

The node_manager is the broker by which everything obtains nodes, given their URL handles. It also is the node allocator, fabricating new ones when first referenced. For the latter purpose, it requires identification of the parent node in any situations that might be first encounters, in an entire session or since a logout.

There are a few kinds of structural connections by which users navigate the content hierarchies.

• Navigation into contained items: Downwards navigation is by clicking on the items presented as the contents of some kind of folder, including share room and storage devices, their contained folders, and the various root containers.
• The items have URLs consisting of the application's address followed by a fragment identifier consisting of the content's address. The app's [#DynamicPageTraversal] (below) retrieves the JSON data for content nodes and populates and presents an application node for it.
• Besides content items and the special provisions listed below, fragment addresses are simple locations of jQuery mobile page divs ([data-role="page"]), conveyed to the regular jQm pageChange if none of the specific conditions obtain.
• Some of the encompassing root nodes, like the RootContentNode and PublicRootShareNode, use static DOM templates, with corresponding static document addresses. Intra-document (fragment) links to those static template addresses are mapped to the addresses of corresponding root content nodes using a handle_content_visit() helper, internalize_url().
• The special consolidated root node, RootContentNode, arranges to update the contents of the respective actual content roots - RootStorageNode, OriginalRootShareNode, and PublicRootShareNode - and presents the content associated with the current account - the RootStorageNode, OriginalRootShareNode.
• The app keeps track of the most recent content tab within which content is visited, so that tab can be resumed when (1) returning to content from the Recents tab, and (2) ascending from top-level content to the tab. This is necessary because Share Room content can reside simultaneously in the My Stuff and Anyone's tabs.
• Some addresses are of application-managed, non-content-specific facilities, like 'logout'. These are listed in the document_addrs object, which is used by handle_content_visit() to dispatch the listed functions.
• Navigation from more contained nodes to less contained ones depends on registration of the parent node URLs in the offspring when the offspring nodes are allocated.
• RootStorageNode: (eventually) adjusting backup coverage
• OriginalRootShareNode: (eventually) managing the share rooms published by the account
• PublicRootShareNode: adding to and omitting from the collection of share rooms being visited.

• so_init_manager (in custom-scripting.js) has a roster of frameworks that need to complete their initialization before the app can be launched. Calls to so_init_manager.ready() are included in initialization-completion routines for the various frameworks, and when the last one completes, spideroak.init() (below) is invoked.
  • jQuery Mobile: an anonymous function that is bound (in (custom-scripting.js) to the jQm mobileinit event calls so_init_manager.ready('jQm')
  • PhoneGap: onDeviceReady() (also in custom-scripting.js) is either registered (in index.html) on the PhoneGap deviceready event, if PhoneGap is observed present. If we are running outside of PhoneGap (useful for a few purposes), then onDeviceReady() is invoked directly. onDeviceReady() calls so_init_manager.ready('PhoneGap')
  • Document loaded/DOM: The index.html body element has an onload trigger which calls so_init_manager.ready('DOM').
  • Application code: The last statement in SpiderOak.js is so_init_manager.ready('app').

1. This arrangement is insensitive to the order in which the frameworks happen to initialize, or interdependencies between them. When the initialization of each has been accounted for, the main app inits.

• spideroak.init():
  • establishes the traversal handler, handle_content_visit() on jQuery Mobile's pagebeforechange traversal event.
  • instantiates the RootContentNode, RecentContentNode, and PublicRootShareNode.
  • makes index.html DOM adjustments for branding
  • prepares the various credentials forms submit callback and visibility controls
  • arranges the combo-root (RootContentNode) intial fade-in does initial fetch of persistent settings
  • does a traversal to the combo-root RootContentNode, to be dispatched by our handle_content_visit() jQm pagebeforechange handler.
• the RootContentNode and other content node .visit() methods, which handle traversal to them are described [#DyanamicPageTraversal].

Persistent settings are managed via two facilities:

• the persistence_manager: provides a simple API for managing persistently stored values javascript, preserving structures by using JSON transparently:
  • .set(name, value)
  • .get(name)
  • .remove(name)
  • .keys()
  • .length()
• the remember_manager, a layer on top of the persistence_manager for managing non-sensitive user account info, and whether or not retention of user account info is elected ("remembering").
  • The maintained fields:
    • username
    • storage_host
    • storage_web_url
  • .unset(): True if no persistent remember_manager fields are available.
  • .active(): Establish UI "Remember Me" if passed a truthy value, return whether or not it the mode is active if passed no value, else deactivate if passed a non-truthy (false or related) value.
  • .fetch(): Return an object with the currently remembered field values
  • .store(values_obj): Persist the set of field values, passed in an object. The object must have settings for all the maintained fields.
  • remove_storage_host(): Remove the storage_host persisted value. This is the way to inhibit auto-login, without losing the convenience of a remembered username (in the absence of a way to remove the authentication cookies).

Most content nodes ([#ContentNodeDataModel]) are presented as jQuery Mobile "pages" - <div data-role="page>. Most are cloned copies of static index.html template nodes, upon in-app traversal of content URLs (see [#ContentAddressing], above).

• The spideroak object's handle_content_visit() function is assigned to handle the jQuery Mobile pagebeforechange event, which is triggered by changes to the browser's location.hash setting and $.mobile.changePage() invocations.

1. Our handle_content_visit() routine only discontinues default jQm traversal machinery for handling of string addresses. That way, the standard jQm traversal facilities handles traversal to the jQm objects that our machinery fabricates, by passing them to $.mobile.changePage() as jQm objects.

• A few different classes of addresses are used in the application, as described in the [#OrganizationofContentTraversal] section, above.
• For navigation URLs that go to application content, including the user's SpiderOak-managed content, we:
  • Fetch a suitable ContentNode-based object for the node using node_manager.get(), which returns an already existing node, if any, or else allocates a new one with suitable initial settings.
  • The obtained ContentNode-derived object uses its .visit() method to handle the visit - cloning or directly using a (jQm <div data-role="page"/>) page object and recursively using the traversal machinery to visit it.
• In most cases, ContentNode.visit():
  • Fetches the node contents from the server if necessary (intially, always necessary)
  • Gets a jQuery object for the node's data-role="page", cloning the index.html id="content-page-template" <div> if it hasn't already gotten a copy this way, and adjusting the basic structure.
  • Fills in the page header and footer according to the node's navigation context. Different types of nodes have different actions and relative ascending navigation routes associated with them.
  • Populates the page listview with entries for the node's contents, or special activities associated with the node.
• The RootContentNode's .visit() method is different. It dispatches a visit to the PublicRootShareNode, RootStorageNode and OriginalRootShareNode. All of those visits are delegated such that:
  1. The delegated presentations do not take browser focus (using the 'passive' mode option), leaving focus on the RootContentNode, and
  2. the RootContentNode is notified of success, along with an indicator of the transaction that succeeded, so it can redisplay its consolidated content view, as it notifications are received.
  3. The visit is contingent to receiving success status notification from the RootStorageNode visit, since both depend on successful authentication, while access to contents of the PublicRootShareNode does not.
     • If the RootStorageNode visit succeeds, the OriginalRootShareNode visit is dispatched.
     • If the RootStorageNode visit fails, the RootContentNode presents the login form, with the username filled in and Remember Me is activated, as appropriate.

See [#ContentNodenavigationmodes] for details about the navigation modes options.

Adding an exceptional node type, to maintain the roster of recently visited content items, involves unusually many of the various traversal arrangements.

Here are the changes to plug the new object in to the surrounding traversal infrastructure:

• In the spideroak object generic settings object, add a contrived recents_url, for a node_manager address to the object.
• Add an is_recents_url() predicate routine,
  • and include that among those checked in is_content_root_url(). (The recents collection is the root of a single-node hierarchy.)
• Provide a root-type case in node_manager.get()
  • and, to accommodate exceptionally frequent access, a dedicated .get_recents() method, like .get_combo_root().
• Also in generic settings, add 'recents_page_id'
  • and, in internalize_url(), provide for translating the recents_page_id to the recents_url, so traversals to the page_id are handled as node traversal.

Here are the steps to fabricate the object, itself:

• Add RecentContentsNode, basing its prototype on ContentNode
• Give it a distinct .visit() prototype method, skipping any provisioning (unlike normal items, its contents aren't constituted from remote data).
• Add to it a unique .add_visited_url(), for use by handle_content_visit() to register each traversal to a content item.

There were some other, small changes to incorporate the functionality, eg adding a no_dividers mode_opts parameter, with respect for it in ContentNode.layout_content(), or adding a generic recents_max_size setting, but that is essentially it.

The traversal provisions are clearly more elaborate than the actual object implementation, and may bear some scrutiny for simplification and consolidation. The above details should be good fodder for such scrutiny.

Special navigation behaviors and user settings are communicated internally using specific operational parameters, using parameters conventionally called page_opts and mode_opts .

• page_opts is used within the app to convey jQuery Mobile's $.mobile.changePage data.options between the apps navigation and presentation functions.
• mode_opts is used to convey internal operating modes across method calls. mode_opts settings are passed through, eg, the .visit(), .provision*(), and .show() / .layout*() node methods, some of which may have behaviors conditioned by the options.
• mode_opts values are transposed to and from page traversal query strings by the apps navigation machinery, so mode_opts settings are conveyed across page traversals. The mode_opts key/value pairs are conveyed in URL addresses as URL '?' query string x=y parameters. handle_content_visit() translates from URL to mode_opts using query_params(url). (Some modes are not conveyed as URL query parameters.)
• User settings are conveyed in DOM traversal as URL var_name=name and var_value=value query parameters, and internally as corresponding mode_opts {var_name: name, var_value: value} attributes.

Currently implemented mode_opts modes include:

• refresh: refetch the data for the node.
• logout: return to the dashboard/combo root and logout from storage
• passive: when showing a node, just adjust the DOM $page, don't transfer browser focus by doing a $.mobile.changePage(). Useful for page updates in the background, or for the RootContentNode composite page, which is informed by combined subnode updates.
• notify_callback: report visit success or failure to this callback function. The callback requires: the visit success (true or false), the accompanying value of notify_token (see below, and on failure, the XMLHttpResponse (xhr) object.
• notify_token: object to be passed back to notify_callback. A means for the initiator to identify the transaction and convey useful state.
• actions_menu_link_creator: Pass in a function to be used by .layout_item$() to get a link to a context-specific actions-menu, for inclusion on the item's list entry as, eg, a split-button.
• no_dividers: tell ContentNode.layout_content() to not inhibit creation of dividers even when content is plentiful.
• icon: Name a particular icon, for use in .layout_item$()
• transition: Name a specific transition effect, for use in .layout_item$()
• alt_page_selector: A means for a content node to make adjustments to pages in addition to its own. Currently only supported by .layout_header(), it is a selector passed in to identify the pages to adjust. (As of writing, used by RootContentNode to set the header of some static info pages.)

Content urls are recognized by virtue of beginning with one of the registered content roots.

The storage root is registered when the user logs in.

The client keeps track of two share rooms roots:

• the collection of share rooms that belongs to the logged-in account, known here as "personally owned share rooms", or "personal share rooms",
• the collection of share rooms that the client has been used to visit, or familiar "public share rooms".

The account login procedure includes a provision for redirecting the storage access from the default server to secondary ones, for those accounts that have their storage service provided by secondary servers. This redirection step is the reason for the elaborate storage_login function, which recurses to the indicated server when required.

The html5 app, as cloned from the git repository, is arranged so that development is done in the files organized around the clone's root. A few scripts in the ./tools subdirectory compose the releases, and build the platform-specific application packages, using the development copies and resources collected in specific subdirs.

• The cloned repo's top level files, including index.html, SpiderOak.js, and the other files included in the index.html header, are what the developer edits.

1. Some of the top level files are actually symlinks into specific versions of resources that vary for different renderings of the app.

   This top level is organized so you can run by pointing your browser at the index.html (though you need to specifically enable your browser to allow cross-origin operation for file-based pages), for immediate feedback, or you can view release versions produced by some scripts, described next.

• The tools/prep_release script produces various, complete-unto-themselves "release" collections of the html/javascript/css files, in the releases subdirectory. These releases vary according to branding, color scheme, and platform theme, and are named accordingly.

1. The releases created by prep_release are not platform-specific executables. (As of this writing, only the iOS platform theme is implemented, Android will be added soon).

   This prep_release script is configured with lists of the variations within a set of categories, currently brand, color scheme, and platform style. The script takes selection of no variants within a category to mean doing all the variants for that category. You can see the available variants (without any work done) by invoking the script with --help.

   The script assembles the releases from a combination of the development copies and resources residing in the release_artifacts subdirectory.

• The tools/build_platforms script uses the results of (and machinery from) the tools/prep_release script to assemble and compile platform-specific executable application packages in the releases/PhoneGap subdirectory. It takes the same set of variant specifiers as the prep_release script.

1. Note: build_platforms uses PhoneGap pluginstall, which depends on npm, the node.js package manager, to be installed on your development host machine.

   Unlike prep_release, build_platforms does nothing if no variants at all are selected. You can explicitly get all variants built, without enumerating all the variant selectors, by passing the flag --all.

   build_platforms currently uses the respective platform SDK to build the platform-specific packages. (It may be worth periodically reevaluating the PhoneGap cloud build services, as an alternative.)

   build_platforms uses PhoneGap pluginstall, which requires npm, the node.js package manager.

The app depends on PhoneGap plugins for platform capabilities that are not yet currently directly available via html5.

• The plugins we're using are located in release_artifacts/PhoneGap/plugins directory. Our build machinery only includes those plugins in platform builds for which the plugins have implementations.
• For details about proper structuring of a PhoneGap plugin, see the Cordova Plugin Specification.
• build_platforms uses PhoneGap pluginstall to programmatically hookup the plugins in the package builds. Pluginstall requires that npm, the node.js package manager, is installed on your development host machine.

Current plugins:

• iOS keychain plugin - we're aiming to extend this for use in Android.

We have reason to believe that we can improve the behavior of jQuery Mobile transition animations by incorporating custom animation code, getting the advantages of native facilities, like Core Animation on iOS, over the standard webkit browser animations.

Details for extending the jQuery Mobile transitions animations are on The jQuery Mobile Transitions page. The Creating custom JavaScript-based transitions section, part way down, has details for integrating custom javascript functions. We would expose our custom animations via PhoneGap, as javascript functions that fit the signature described in the Transition Handlers subsection. (The A model for Custom transition handler development section points to an example to use as a coding reference.)

The jQm busy/loading spinner is a distinct animation, with its own, distinct customization points. See the jquery.mobile.loading() box on the jQuery Mobile Methods and Utilities page, and the jquery mobile pageLoading widget for another way to invoke it. (The "Custom HTML" example on that page may be useful.)

To create the hybrid application package, use tools/build_platforms specifying the <brand>, <color>, and <platform>. This will compose and build the project in releases/PhoneGap/<platform>/<brand>-<color>.

You can also use the tools/build_platforms script to run the application in the iOS emulator (that comes with Xcode) via the command line, by passing --run before any of the parameters.

Alternatively, you can run the project, built by the tools/build_platforms script, under Xcode. Open the <brand>-<platform>.xcodeproj file within the above mentioned directory, using Xcode. Make sure to select the '<brand>-client' product on the left hand side of the Xcode scheme selector, and the device/emulator you want to use on the right. Then you can run, debug, and package the project as you normally would with Xcode.

NOTE that currently the *Acme* brand will not build successfully, because it is missing several of the required launch and icon images. This also means that tools/build_platforms --all is currently blocked, since the *Acme* brand is at the front of the alphabetic roster.

The tools/prep_release script produces various application renditions, varying branding, color scheme, and platform look and feel. The script is also useful as guidance, indicating the locations of rendition-specific elements.

The script has external configuration variables towards to top, for easy incorporation of new elements.

Currently, the development code is only viewable by running from local files, or as a fully built and side-loaded application. (We are working on organizing a facility for making it possible to view the latest development code, via a web visit from a regular browser session. In the meanwhile, going through these hoops is necessary to try it out.)

To run the client from local files you must have to run your browser with flags that reduce browser security, including allowing local file visits to code (javascript) and also disregarding restrictions on Cross-Origin Resource Sharing (CORS). Beware that this means substantially reduced browser security.

I use a colleague's suggestions for doing so with Google Chrome. I haven't found a way to do so with other browsers (recipes welcome).

Here are the command options you need to pass to Chrome:

• --allow-file-access-from-files --allow-http-access-from-files --disable-web-security --enable-file-cookies

To avoid using an insecure browser as my primary browsing session, I installed a copy of Chrome Canary, so I can run a dedicated insecure Chrome session concurrent with my regular session one. I haven't hit any problems specific to running Canary, but I may have been lucky in the particular Canary build for MacOS that I'm using. If you hit odd problems trying to get Canary going, try restarting regular Chrome with the arguments, just to establish that things work.

  Note: I've added a tiny app to the same share room with the canary dmg.
  Also useful only in Mac os, "Chrome Permissive.app" launches canary
  (if you install canary in the default location, as
  <code>/Applications/Google Chrome Canary.app</code>). Put a copy of this whole
  directory hierarchy in your <code>/Applications</code> folder and launch it to
  run Canary enabled to run local files with CORS restrictions eased.

The default set of jQuery Mobile theme swatches - collections of theme features assigned to letter identifier) - get us near enough to Mike's designs to be worth building on.

1. We use the jQm theme roller to adjust the swatches and add new ones. You need to use the procedure (next subsection) to preserve the existing adjustments.
2. We customize the limited theme settings with two css code files, icons.css and css/tweaks.css . That's where most of the ongoing action should happen.

When changing a theme or adding a new swatch, it's crucial to feed our tailored theme's css back into the tool, in order to preserve our existing adjustments. (Even if we don't make further theme customizations via the theme roller, we may need to use it to upgrade to subsequent jQm versions.)

To do so:

• Visit the tool at http://jquerymobile.com/themeroller/
• Open the "Import (or Upgrade)" activity
• Enter our adjusted css/themes/iphone.css in the text box and hit Import
• Make adjustments
• Use the "Download (theme zip file)" activity to get the zip file
• Put in place copies of the desired artifacts - .css, .min.css, and incidentals

Because button and list item styles are used across theme swatch page elements (header, content, footer), we sometimes have to use different swatches for different parts of the same page.

In case we need to re-derive our theme, it's based closely on the default jQm theme, using the following procedure:

1. Visit http://jquerymobile.com/themeroller/
2. Use "Import or Upgrade" tab and "Import default theme"
3. Turn swatch A to stark white:
4. - Copy the white patch to the header
5. - Set header/footer background gradient to be #ffffff to #ffffff
6. - Copy the white patch to the content body
7. - Set content background gradient to be #ffffff to #ffffff
8. Download (with name "spideroak")
9. Unzip in css subdir, or elsewhere and copy out the specific artifacts.

I've added a bunch of theme swatchs, including notable the "f" swatch for SpiderOak orange buttons and other elements.

jQuery Mobile DOM "enhancements" don't just involve adding class (and other) attributes to tags - structures are injected around things, as well. This means that elements location in the DOM may be different, before and after enhancement. For example, the header button labels and icons get enveloped in structures, so that you have to look in the new spot before just changing the location where they were originally found.

As of 2012-05-30, ContentNode.prototype.layout_header_fields() shows examples of this elaboration.

Default caching of typical $.ajax() requests - including json ones - means that normal traversal of the content hierarchy won't update after first pass without explicitly asking for a refresh. We specifically override the cache on explicit refreshes, but not on return by traversal to already visited nodes. This seems like a good tradeoff, reducing server load by requiring that the user explicitly ask for an update, but we should warn the user about it.

The (new in jQuery Mobile 1.2) popup machinery requires special provisions to work properly with our handle_content_visit() pagebeforechange handler.

It appears that the URL that launched the popup is fed back through changePage when the popup is dismissed, and the default machinery (evidently) needs to see that traversal to account for closing of the popup.

In order to recognize and properly pass through the second traversal, we treat URLs that are going to trigger popups with transit_manager.distinguish_url(). That adds a query parameter for handle_content_visit() to recognize, using transit_manager.is_repeat_url(), and pass it through to the standard jQm traversal mechanism.

Despite all this, the popup menu on the public share rooms split-button (in the public share rooms visit root) misbehaves if it is dismissed twice without any intervening selections or page travesals. In that case, the system will traverse back to the prior page visited. This needs to be fixed, but it is somewhat obscure, and darned stubborn.