2. API - Automattic/stories-android GitHub Wiki

Glossary

Here are a few terms we use:

  • A Story is a collection of slides, organized in a certain order.
  • A Slide (or more commonly seen in code: StoryFrame) is a flattened media item, which is saved into some known media format such as jpeg (for static images) or mp4 (for videos).
  • “Flattened” in this context refers to a background image / video plus any other views that were dropped on top (typically: text and emoji), all composed together in a new produced image or video file. The added views (emoji, text) in this newly produced media cannot be separated from it given they are now one unique image / video composition.
  • Added Views: these are all the views that were added to an original background source (again typically: emoji and text).
  • Background Source: the original image / video being used as a background for working on a Slide’s composition.

Entry points

The Stories library main entry point is the abstract ComposeLoopFrameActivity. By implementing an Activity that extends from it and setting the mandatory providers and listeners, you'll be able to integrate it in your project in minutes. An example of this can be found in the demo app here.

Given that the ComposeLoopFrameActivity is central to the architecture, most of the actions and functionality can be triggered by means of the Android Intent system and a few modifiers as seen in the following section.

Intent Extras

  • EXTRA_LAUNCH_WPSTORIES_CAMERA_REQUESTED
  • KEY_STORY_SAVE_RESULT
  • KEY_STORY_EDIT_MODE
  • KEY_STORY_INDEX (see getStoryIndexFromIntentOrBundle())
  • requestCodes.EXTRA_MEDIA_URIS

The Live Preview / Capture mode (implemented with CameraX) can be triggered by passing a boolean extra EXTRA_LAUNCH_WPSTORIES_CAMERA_REQUESTED to the ComposeLoopFrameActivity launching intent.

The Error handling mode will be automatically presented when a KEY_STORY_SAVE_RESULT is passed along with a serialized StorySaveResult object, and any of the frames in the Story is errored. The StorySaveResult object will contain all the information needed to populate the Story slides along with which slides were errored, so that the user can retry each one individually, or directly discard each slide individually. Error handling flows were designed as a core part of the library from the beginning.

Finally, the Story Editing mode can be launched in two ways:

  • when passing a KEY_STORY_SAVE_RESULT serializable extra along with the intent, and all the slides in the Story have a successful saved result. The StorySaveResult will contain all the information to populate the Story slides.
  • when passing a Story index (which should be already loaded in the StoryRepository) combined with the KEY_STORY_EDIT_MODE extra value set to true.

Providers and listeners

The library is designed to be extended via different java interfaces we generically called Providers, because the functionality will be provided externally via these interfaces. If these are set, these will be used in different ways and moments as explained below.

  • MediaPickerProvider (mandatory): this is an interface that serves the purpose of passing control to a media picker interface that is provided by the user of the library, in which the application user can select media that is finally going to be passed back to the Stories library, to be used as background for Story slides. The media picker on the other hand will be passing back a set of picked media URIs using the EXTRA_MEDIA_URIS extra in onActivityResult(). This extra can be passed in a plain intent and will be loaded in onLoadFromIntent() if startActivityForResult() is not used.
  • SnackbarProvider (optional): this interface will be used when the Stories library needs to express something to the user (for example, that a saving process has started or finished, did it finish with error or successfully, etc.). The Stories library falls back to using Toasts when a SnackbarProvider is not present.
  • AuthenticationHeadersProvider (optional): use this interface if you need special HTTP headers to be sent along with image / video requests for background images being fed to the library to be used as Story frame backgrounds.
  • MetadataProvider (optional): this is a joker provider; information passed through this provider will be sent along with the result events so the application can determine which results correspond to which application triggered action (or use it for whatever you need as well).
  • NotificationIntentLoader (optional). When saving a Story, the library will be calling this in the case of errors.
  • StoryDiscardListener (optional): implement and set this listener to be informed of when the user discards either a single Frame or the entire current Story.
  • StoriesAnalyticsListener (optional): called each time the text on a Story slide is changed.
  • NotificationTrackerProvider (optional): called to track events for showing internal notifications (differentiates on shown, tapped, dismissed).
  • PrepublishingEventProvider (optional): this is getting called each time the user taps on the check to save a Story. If you need to request metadata (such as a title for the Story) before saving, this is the hook to use.
  • PermissionDialogProvider (optional): only used in the case the permissions needed by the library are permanently denied. Fill in your own dialog to be presented when called. Defaults to using a Toast if not.
  • GenericAnnouncementDialogProvider: to be called in onStart() once. You can choose to make a generic announcement here.
  • setUseTempCaptureFile(boolean): just indicates whether CameraX captures will be saved using a temporary file or a file that can be broadcasted as available to use through the system (note these are not the final produced Story slides files).

The best way to see these in action is to check the demo app.

Output

The Stories library will end up producing media files that can be consumed as you wish. Given it produces one jpeg/mp4 file per Story slide, you can use it in whatever form you'd like, without any further need to process or understand other non-standard formats.

The information about how the Story was originally arranged by the user, along with the error/success state of each saving process is delivered via EventBus using a simple abstraction which we called StorySaveResult.

9:16 exports

The library optimizes output for viewing (consumption). The majority of devices out there are 9:16 or a taller variant of that and as such, we decided the simplest way is to standardize at 9:16.

At the same time, we're trying to offer a WYSIWYG kind of experience so, whenever possible, we try to accommodate things on the screen the same way things will be looking like when exported. This piece of code reflects the decision tree we're using to decide where to anchor background images and whether or not to show a black area at the bottom to cover whatever is getting cut after exporting to 9:16 compatible formats.

  • if the image being loaded matches the aspect ratio of the device screen, use center_crop. No parts would actually be cropped, given the matching aspect ratio it should fit except the opaque bar given we're normalizing to 9:16 on export
  • if the device is taller than 9:16, and the image is portrait just crop the bottom (showing the opaque bar)
  • else, load with fit-center (black bars on the side that doesn't fit)

EventBus events

The FrameSaveService process will emit save progress signals for each of the Story’s slides through EventBus events. (If you’d like to see more, take a look at StorySaveProcessor in FrameSaveService in the Stories for Android library).

We have these events for each frame:

  • FrameSaveStart. Triggered each time a frame is about to start its saving process.
  • FrameSaveProgress. Informed progressively as the save operation is performed.
  • FrameSaveCompleted. This is broadcasted after a successful frame save.
  • FrameSaveCanceled. This is currently unused, given we don’t offer user-triggered cancelation as of today.
  • FrameSaveFailed. This is an error in attempting to save a Story slide/frame.
  • StorySaveResult. Once all frames have finished being processed, this event is issued with the overall result and also an array of results for each contained frame.

There is also a subtlety: the static image slides will get a start event and then a complete or failed event, but no intermediate progress reports given we don’t really have much progress to inform about; the process is snappy with these. Progress events are mostly there for lengthier operations such as video composing and processing.

You should be able to register to listen to any of these events and do what you need in your app.

Next: architecture