Design - add-ons/service.iptv.manager GitHub Wiki

This design document describes the different requirements and implementation decisions, and explains why specific choices were made over other choices.

NOTE: If you are looking for information on how to integrate your add-on with IPTV Manager, look at the Integration wiki page.

Requirements

The IPTV Manager design is based on the following requirements:

Mandatory:

The add-on should not require a service to subscribe to IPTV Manager (i.e. it works with simple add-ons)
The interface that add-ons must use should be simple and clean
Add-ons may subscribe to VOD services and do not necessarily use XMLTV for EPG internally
We need to support add-ons using path-based (/iptv/channels) as well as query strings (?iptv=channels) routing
Do not write information to SD cards to not wear them out

Nice to have:

Add-ons with static streaming URLs could drop channels in a file and be done with it (poor man's integration)

Subscription

The first task for the IPTV Manager is to determine which add-on has support for IPTV Manager. One option would be for the add-on to register with the IPTV Manager, but that is cumbersome (when does an add-on do this?). Another option is to advertise this information to IPTV Manager:

Require service.iptv.manager in addon.xml

Not withheld.

This is the cleanest option. The add-on advertises that it (optionally) requires service.iptv.manager and IPTV Manager simply queries all add-ons if it is listed as a requirement.

NOTE: However at the moment there is no Kodi interface to check for add-on requirements, which means we have to read addon.xml files from add-ons which is in violation of kodidev rules for inclusion in the official Kodi repository.

Advertise from settings.xml

This is now implemented.

Another option is to advertise support through the settings.xml as hidden settings. A standard could state:

iptv.enabled -- When this exists, IPTV Manager is supported by the add-on
iptv.channels_uri -- The add-on advertises the URL to request for channels
iptv.epg_uri -- The add-on advertises the URL to request for EPG data

IPTV Manager would check add-on settings for iptv.enabled and act on it. Possibly caching this information so it doesn't have to scan very often.

Advertise using iptv.json in add-on directory

Not withheld.

We could also scan add-on directories for a standardized iptv.json file in the add-on directory. This file could advertise other information:

{
    "channels": "plugin://plugin.video.example/iptv/channels?output=$FILE",
    "epg": "plugin://plugin.video.example/iptv/epg?output=$FILE",
    "version": 1
}

NOTE: Since this means we also have to scan add-on directories, this would be in violation of kodidev rules for inclusion in the official Kodi repository.

The upside here is that for simple add-ons the add-on could expose other information as well from the channels pointing to a local JSON STREAMS-format-formatted file, which would not require any change to the add-on.

{
    "channels": "channels.json",
    "version": 1
}

Inter-addon communication

One problem in Kodi currently is communication between a service and an add-on that may not have a running service.

RunScript

Not withheld.

One option is to call an add-on by an exposed script.

Unfortunately this interface does not allow for returning information to the requester. So this is only useful to initiate something, but is no message-passing mechanism.

Arguments can be provided with this call but are limited to sys.argv data limits and may require serialization for sending python data structures.

Usable for service → add-on communication

RunPlugin

This is now implemented.

Typically add-ons are being called through the RunPlugin interface which accepts a plugin://-type URL.

Unfortunately this interface does not allow for returning information to the requester. So this is only useful to initiate something, but is no message-passing mechanism.

Arguments can be provided with this call but are limited to sys.argv data limits and may require serialization for sending python data structures, so we cannot use this for the add-on to return information by connection back to the service.

Usable for service → add-on communication

JSON-RPC

Not withheld.

Unlike RunScript and RunAddon, JSON-RPC is a mechanism that supports returning information to the requester. Unfortunately, JSON-RPC requires the receiver to have a service running, and therefore is not a suitable candidate for calling an add-on as it would require add-ons to have a service running. We do not want to impose this complexity on add-on developers.

JSON-RPC could be a mechanism for an add-on to return information to the IPTV Manager service.

NOTE: We identified a problem for using JSON-RPC for transferring large amounts of data. The amount of memory claimed does not seem to be released, we also found that there is no maximum limit, but too large payloads crashes Kodi.

~~Usable for add-on → service communication~~

Named pipes

Not withheld.

If we have a mechanism to tell the add-on to subscribe to a named pipe, we could have the add-on send information over a named pipe back to the IPTV Service.

The advantages of named pipes are that they do not require disk-based access and have no imposed data limit, which are important features.

The downside is that they only work on Unix, so this is not acceptable as a solution.

File-based communication

Not withheld.

Another option we have is to write data to a tmpfs based file-system. We need to avoid doing writes to an SD card, and most (if not all) systems with SD cards make use of tmpfs for /tmp which makes the a possible candidate.

There is no data limit and service can communicate the location via another mechanism.

Usable for add-on → service communication

Socket-based communication

This is now implemented.

One way to avoid writing to disk and keep information in memory is by using sockets for communication. The service calls the add-on to request information and requests to get this information delivered to a specific (random) port. The add-on connects to the socket and dumps the requested information (channel listings and EPG).

Doing socket communication in add-ons is not very complicated, but definitely something different than what add-on developers are probably used to.

Usable for add-on → service communication

HTTP-based communication

This is what we would like to support in the future.

Another option is that the IPTV Manager starts an HTTP-webserver in order for add-ons to communicate data back.

There is no data limit and the service can communicate the location via another mechanism. The only concern we have that it complicates the service with another mechanism that should not be necessary for inter-addon communication.

For add-ons this offers very simple integration (basically an HTTP POST request to send the requested data over).

Usable for add-on → service communication

Possible solutions for inter-addon communication

Communication from service to add-on

RunPlugin → implemented
RunScript

Communication back from add-on to service

File-based
Socket-based → implemented
HTTP-based → wanted

Data formats

Channels

This information that needs to be exchanged is pretty limited.

JSON-STREAMS (new standard)

This is now implemented.

Convenient for add-ons to integrate in Python
Extensible

NOTE: More information on the new JSON-STREAMS standard is available from JSON-STREAMS format.

M3U

Not withheld.

Known standard
Used by iptvsimple natively
Requires parsing in IPTV Manager

EPG

JSON-EPG (new standard)

This is now implemented.

Convenient for add-ons to integrate in Python
Extensible

NOTE: More information on the new JSON-EPG standard is available from JSON-EPG format.

XMLTV

Not withheld.

Known standard
Used by iptvsimple natively
Requires parsing in IPTV Manager
Some add-ons may have this information already in XMLTV format

JSON (XMLTV-based using Badgerfish or Abdera)

Not withheld.

Maps to XMLTV, which is used by iptvsimple natively
Known standard
Cumbersome for integration

{
  "tv": {
    "@source-info": "http://www.schedulesdirect.org/",
    "@source-info-name": "Schedules Direct",
    "$": [
      {
        "channel": {
          "@id": "vrt.nu.een",
          "display-name": "Eén",
          "icon": {
            "src": "https://foo.bar/logos/een.png"
          },
        },
      }, {
        "channel": {
          "@id": "vrt.nu.canvas",
          "display-name": "Canvas",
          "icon": {
            "src": "https://foo.bar/logos/canvas.png"
          }
        }
      }, {
        "programme": {
          "@start": "20080715003000 -0600",
          "@stop": "20080715010000 -0600",
          "@channel": "vrt.nu.een",
          "$": [
            {
              "title": {
                "@lang": "en",
                "$": "Now on PBS"
              },
              "desc": {
                "@lang": "en",
                "$": "Jordan's Queen Rania has made job creation a priority to help curb the staggering unemployment rates among youths in the Middle East.",
              },
              "date": "20080711"
            }
          ]
        }
      }
    ]
  }
}