Another new server in urlparser: typical strategies - oe-mirrors/e2iplayer GitHub Wiki

Hoster pages rarely show the stream url in plain text. These are the tricks you meet most often, and the tools E2iPlayer has against them. They work the same in a resolver (libs/urlparser.py) and in a host.

Always start with the debug log: printDBG(data) shows what the box really received - and the browser's developer tools (Network tab) show which request finally returns the .m3u8/.mp4. Then work backwards from that request.

1. Two kinds of link

Many hosters have a download page and a player page for the same video, e.g. https://hoster.com/f/1f13jq and https://hoster.com/e/1f13jq. The player page (/e/, /embed/, /v/) is the one with the stream - turn the other one into it first:

url = baseUrl.replace('/f/', '/e/')

2. Escaped strings

Code like this hides text in \xNN and \uNNNN escapes:

y["_\x70op"]=["s\x69\x74\u0065Id",3365713],["h\u0074\u0074ps:\u002f/u\x74h..."](/oe-mirrors/e2iplayer/wiki/"s\x69\x74\u0065Id",3365713],["h\u0074\u0074ps:\u002f/u\x74h...")

This helper decodes both kinds and works on Python 2 and 3:

from Plugins.Extensions.IPTVPlayer.libs.e2ijson import loads as json_loads
from Plugins.Extensions.IPTVPlayer.p2p3.manipulateStrings import ensure_str


def jsUnescape(text):
    # \xNN and \uNNNN escapes of a JavaScript string -> characters
    return re.sub(r'\\x([0-9a-fA-F]{2})|\\u([0-9a-fA-F]{4})',
                  lambda m: ensure_str(json_loads('"\\u%s"' % (m.group(2) or '00' + m.group(1)))), text)

jsUnescape(code) turns the line above into y["_pop"]=["siteId",3365713],["https://uth..."](/oe-mirrors/e2iplayer/wiki/"siteId",3365713],["https://uth...").

[!NOTE] unicode_escape() in libs/urlparserhelper.py decodes only \uNNNN and returns bytes on Python 3 - wrap it in ensure_str() if you use it. The old text.decode('unicode-escape') works only on Python 2.

Other encodings you meet: base64 (atob("...") in the JavaScript) → b64Decode(text) from tools/iptvtools.py (adds missing padding, returns a string; binary=True for bytes), URL encoding → urllib_unquote from p2p3.UrlLib, HTML entities → self.cleanHtmlStr() or ph.clean_html().

3. Packed JavaScript

Very common on JWPlayer hosters:

eval(function(p,a,c,k,e,d){ ... }('2.8="//3-4.5.6/i/7.f";2.o="//3-4.5.6/a/7.b?3=c&e=d";...',25,25,'||MDCore|s|delivery4|...'.split('|'),0,{}))

This is Dean Edwards' packer: the real code is in the first string, the words in the list at the end. It does not need a JavaScript engine - libs/jsunpack.py unpacks it in Python:

from Plugins.Extensions.IPTVPlayer.libs.jsunpack import get_packed_data

if 'function(p,a,c,k,e' in data:
    data = get_packed_data(data)   # all packed blocks of the page, unpacked

The unpacked code then contains sources:[{file:"..."}], MDCore.wurl="..." or similar, which a regular expression finds. The generic parserJWPLAYER in urlparser.py does exactly this - read it as an example.

4. When code has to run

Some players compute the url with their own JavaScript (string shuffling, small ciphers). If it can't be rewritten in Python with reasonable effort, E2iPlayer can run it:

from Plugins.Extensions.IPTVPlayer.tools.e2ijs import js_execute

jscode = 'var key = %s;\n%s\nprint(decode(key));' % (keyFromPage, decodeFunctionFromPage)
ret = js_execute(jscode, {'timeout_sec': 20})
if ret['sts'] and ret['code'] == 0:
    streamUrl = ret['data'].strip()
  • js_execute() uses QuickJS (/usr/bin/qjs) when it is installed, otherwise duktape (/usr/bin/duk), which the E2iPlayer installer offers. It returns {'sts': ..., 'code': exit code, 'data': output}.
  • The script has no browser around it: no window, document, atob, no network. Define what it needs (var window = {};) and give it the data as variables. Its result is what it writes with print(...).
  • It costs time on a box (a process per call) - use it for the small decode step, not for whole pages.

5. Tokens, cookies and headers

A stream url that works in the browser and gives 403 on the box usually misses a header. Check the request in the Network tab and give the stream exactly what the browser sent:

streamUrl = urlparser.decorateUrl(streamUrl, {'User-Agent': HTTP_HEADER['User-Agent'], 'Referer': 'https://hoster.com/', 'Origin': 'https://hoster.com'})
  • signed urls (?token=...&expires=...) are bound to the IP and often to the User-Agent of the request that got them - resolve them with the same UA the player will use, and resolve them only when the user starts the video (need_resolve: 1), never when the list is built
  • a cookie the hoster set on the player page goes along with 'Cookie': ... in the meta
  • some hosters check the order of requests (player page → API → playlist) - repeat the browser's sequence

6. What not to do

Based on the original text by Maxbambi, thank you!