APIs - strohne/Facepager GitHub Wiki

One option to extract data from the web is by means of Application Programming Interfaces (APIs). Most large platforms and social media websites have an API. APIs provide operations to interact with the service– e.g. for requesting data from a platform. Which operations and queries are possible depends on the specific API. Therefore, it is necessary to read the documentation of the API. What most APIs have in common is the data format of the results. Facepager can be used to get data from all kinds of APIs that deliver JSON or XML data.

Note: The basic principle underlying the data collection via APIs is assembling URLs with the help of placeholders. With the assembled URLs, the required data is requested. To understand the structure of URLs and how Facepager works with placeholders, learn more about URLs, Placeholders, Nodes and Keys.

API Modules

To work with the APIs from Facebook, YouTube, Twitter and Amazon, Facepager has respective modules in the Query Setup. To fetch data from other APIs, the Generic Module can be used. For example, using the Generic Module you can interact with Google+ or Wikipedia. To get familiar with the different APIs, see the Presets and the Getting Started tutorials.

API Viewer

Even if the Query Setup of the modules look quite similar, the functionality of each API is different. So, it is necessary to know the endpoints and parameters of the different APIs. An endpoint is the URL of a specific resource you can access with the API, for example the URL to access YouTube comments.

In Facepager, some basic endpoints of the Facebook, Twitter and YouTube APIs are already implemented. They can be accessed and applied via the API Viewer. You can open The API Viewer in different ways. One option is to click on the APIs button in the Menu Bar. Another option is to click on the ? next to the Resource dropdown list in the Query Setup. This will show you the information of the applied endpoint. Also, if documentation is available, you can select a key in the Data View and click on the ?, which will show you the documentation of the selected item.

The API Viewer gives an overview about endpoints of the Facebook, Twitter and YouTube APIs. This comprises a selection of the possible resources that can be requested via the particular API. For more options, you should lookup the documentation of the API provider. You will find links in the API viewer.

Every endpoint is documented with:

• A description of the endpoint.
• A link to the official documentation of the endpoint.
• The path of the endpoint (assembled from the base path and the resource path).
• The parameters you can use with the endpoint. Sometimes, parameters are mandatory. For example, if you want to get the list of Twitter friends from a Twitter user, you have to specify the user name in a parameter.
• The description of the response, that is returned after requesting the resource. The response data is visible in the Data View after fetching the data.

Once you apply an endpoint, the settings will be transferred to the Query Setup of the respective module (see the colored boxes in the images).

Before clicking Fetch data to fetch the data from the applied endpoint, you have to add a seed node. A seed node contains the data you start with. If you, for example, want to get the list of Twitter friends from “TheAcademy”, you have to add "TheAcademy" as a seed node in the Nodes View and apply the Twitter API resource "friends/ list". Also note, that authentication data is never included in the presets or endpoint definitions. Thus, you need to login to the API provider before accessing APIs to obtain an access token and whatever is necessary. For more information about authorization have a look at the page Login and Access Tokens.

In the example on the picture, if you add "TheAcademy" as a seed node the final URL would be https://api.twitter.com/1.1/friends/list.json?screen_name=TheAcademy. With this URL, the list of friends from the Academy is requested and returned in JSON format. Information about what data is contained in the response is documented in the API Viewer. The fetched data appears in the Data View.

The buttons in the API Viewer

Buttons	Function
Reload	The documentation of the APIs will be reloaded. This happens automatically, whenever you open Facepager. If you change the API definitions while running Facepager, you can reload it without having to close and reopen Facepager.
Close	The window of the API Viewer will be closed, without applying any of the selected endpoints.
Apply	The selected endpoint will be applied. This means, that the respective settings are transferred to the query setup.

Additional APIs

You can use additional API definitions with Facepager. In the lower left corner of the API Viewer, the folder containing API definitions is linked. You can add your own OpenAPI-files here. The default definitions that come with Facepager are marked with an asterisk "*" in the API-Viewer.

Note: The APIs are changing very often. By the time you use Facepager, the implemented documentation may be outdated. In this case, you can look directly at the documentation of the respective providers.