release310to319 - get-iplayer/get_iplayer GitHub Wiki

get_iplayer 3.19 - 2019-02-24

get_iplayer 3.18 - 2018-12-28

get_iplayer 3.17 - 2018-08-17

get_iplayer 3.16 - 2018-07-05

get_iplayer 3.15 - 2018-07-04

get_iplayer 3.14 - 2018-05-07

get_iplayer 3.13 - 2018-03-24

get_iplayer 3.12 - 2018-01-14

get_iplayer 3.11 - 2018-01-14

get_iplayer 3.10 - 2018-01-07

• Fixed a bug that caused one-off PVR searches created with --pvr-queue to be deleted when run with --test (thanks @wiehe).

• Replaced BBC Two Scotland with BBC Scotland in programme indexing. If a cache update extends back before the week of 2019-02-18, it will produce errors because a BBC Scotland schedule does not exist for earlier dates. The error can be ignored. EDIT: If you find that BBC Scotland programmes of interest are missing from search results, force all available BBC Scotland programmes to be indexed and cached with (ignore any 404 errors):

    get_iplayer --refresh --refresh-include="BBC Scotland" --refresh-limit=30
  

• Adapted --pid-recursive and --pid-recursive-list functionality for the change in layout of programme episode lists on the iPlayer site. The switch to series-by-series listings for many programmes meant that get_iplayer would only display episodes from the earliest series listed. This almost exclusively affected CBBC and CBeebies programmes, but could affect any programme.

• The episode ordering with --pid-recursive and --pid-recursive-list is now always oldest to newest (as determined by the order of the listings on the iPlayer site). This ensures a consistent order regardless of the source of the episode lists, and it is generally consistent with how search results are ordered by default. Use --sort-reverse to list episodes from newest to oldest.

• Added <senumx> substitution parameter to format series and episode numbers as ##x## instead of s##e## as with <senum>. As with <senum>, <senumx> may be absent.

• Added --expires-after option for use with --expires-before to bracket searches based on programme expiration date.

• Added --pid-index option. The programme index cache is no longer used by default with --pid, whether in a PVR search or from the command line. For an indexed programme, the cache entry provides a backup source of basic metadata, but it is not required. If you notice incomplete metadata or output file names with missing components, use --pid-index to ensure the cache is updated and used with --pid. This would only be of help with indexed programmes from the past 30 days.

• Added --no-merge-versions option. get_iplayer now attempts to merge media streams from programme versions with the same name and duration (e.g., editorial and editorial2). There are some extremely rare cases where you may not want to do so because broken streams may be included in the result due to problems with BBC metadata. In that case, use --no-merge-versions to prevent merging and pick out the specific version you want with --versions.

• Performed final removal of --hls-hq-audio, --no-dash-remux and --stream-http options.

See: https://github.com/get-iplayer/get_iplayer/wiki/installation

See: https://github.com/get-iplayer/get_iplayer/wiki

• Fixed a bug that caused unnecessary cache updates when --refresh-future was used.

• Fixed a bug that prevented a stale cache from being updated with --pvr when first PVR search was PID-based. EDIT: This change will refresh both TV and radio programme index caches when PID-based PVR searches are used, even if --type=radio is not defined in preferences or the PVR searches. This is to ensure availability of at least minimal metadata for indexed programmes found with PID-based PVR searches. If you want to avoid ever indexing any radio programmes, delete the radio.cache file in your profile directory and use --refresh-exclude-groups-radio=local,regional,national to exclude all radio stations from indexing. You will have a radio.cache file, but it will contain no entries.

• Added --hls-lq-audio option to download lower-quality audio with HLS video streams. Use this if 320k audio is too rich for your diet. You will get the same audio as DASH streams (128k or 96k, depending on the stream).

• Added --available-before option for use with --available-since to bracket searches based on programme availability date.

• The --no-subtitles option on the command line will now override all --subs-{embed,mono,raw,required} options saved in preferences. Those options no longer imply --subtitles (Web PVR: "Download subtitles" in "Recording" tab), which is now required with any of them.

• get_iplayer no longer looks for a local options file under the current working directory (in ./.get_iplayer/options). That could cause problems when using --profile-dir while working in user's home directory if a default profile directory had been created. Use a preset instead. EDIT: This functionality was never documented and thus you are extremely unlikely to be affected by its removal.

• The --test option will only show lists of matching episodes with PVR searches and --pid-recursive. Download and parsing of media stream data will only occur with --test if --get or standalone --pid is specified.

• Changes to substitution parameters:

  • Added <sebcast>, <sebcastdate>, <sebcasttime>, <sesort>. See definitions in Substitution Parameters. These parameters provide additional options for constructing sortable file names with --file-prefix.

  • A value for <episodenum> must now be found in programme metadata in order for <senum> to be populated. A value of "1" is no longer synthesised for some TV programmes. A value of "1" is still synthesised for <seriesnum> if <episodenum> is populated and no series number is found in programme metadata.

  • The episode number prepended to <episode> is now zero-padded to 2 digits. This is reflected in the default output file prefix. EDIT: If you use the default file prefix and you mistakenly use --force for a programme you do not wish to re-download, or you mistakenly attempt to re-download a programme you already have that has been removed from your download history, it is likely that you will end up with a duplicate because the zero-padding generates a different file name (assuming an episode number is present). To revert to the previous default file prefix without zero-padding, use:

      get_iplayer --prefs-add --file-prefix="<name> - <episodenum. ><episodeshort> <pid> <version>"
    

    If you use a custom file prefix that contains <episode>, replace it with <episodenum. ><episodeshort> to achieve the same effect.

  • Conditional separators can now be used as suffixes. See example in Filenames and Directories.

• The functionality of the options below has been removed. The options are now ignored and will be completely removed in the next release. A warning message will be printed if any of these options is found in your preferences.

  • --hls-hq-audio: Now essentially useless and could lead to audio sync problems.
  • --no-dash-remux: If you prefer DASH media streams for some reason you can no longer produce potentially unusable output files, at the cost of some extra time in post-processing downloads.
  • --stream-http: get_iplayer will now automatically look for HTTP media playlist URLs if no HTTPS URLs are found.

  You can remove these obsolete options from your preferences (whether or not they are actually defined) with:

    get_iplayer --prefs-del --hls-hq-audio --no-dash-remux --stream-http
  

• A get_iplayer profile directory (containing programme index caches, pvr searches, preset definitions, download history) can now be shared between Windows and Linux/BSD/macOS. get_iplayer on all platforms now processes any CRLF line endings (default for text files on Windows) when reading files from the profile directory. Files will still be written with native line endings (CRLF on Windows, LF on other platforms), so different files may have different line endings if you share them between platforms. This also means that the "download_history" file will have mixed line endings if written on different platforms. The different line endings shouldn't affect get_iplayer, but be aware of them if editing those files directly.

  • If you only use get_iplayer on Windows, or only on Linux/BSD/macOS, you should notice no difference.
  • Warning for Linux/BSD/macOS users: This only works with get_iplayer 3.18 or higher. Once you have CRLF line endings in your files, you cannot use get_iplayer versions before 3.18 unless you first change all line endings in all files to LF and then never again write to those files in Windows.

See: https://github.com/get-iplayer/get_iplayer/wiki/installation

See: https://github.com/get-iplayer/get_iplayer/wiki

• The --force and --overwrite options are no longer allowed in the user options file, with a warning printed if found. They may still be added to presets and PVR searches.
• The --sortreverse option is now recognised with --pid-recursive so that series may be downloaded from earliest episode to latest.
• When subtitles are requested, programme versions containing subtitles are prioritised even if it changes the order of versions used for download attempts. This works around a problem with some box sets where subtitles are not associated with the same programme versions for different episodes.
• Added additional substitution parameters: <firstbcastyear, <firstbcastmonth>, <firstbcastday>.
• Revived HLS streams that are the only supported format available for some ancient radio programmes (mode prefix = hla, enabled by default).
• When sanitising file/directory paths, ellipsis is now replaced with space rather than underscore, unless it appears at the beginning of programme name or episode title, in which case underscore is still used. This means that in many cases ellipsis is effectively removed since white space is compressed during sanitisation.
• Removed BBC Radio Highlands and Islands schedule URLs (no longer available) from programme indexing. This will prevent unnecessary error messages during refresh.
• Fixed a regression in v3.13 that prevented --tag-only from working with --history.
• Removed unnecessary error message when attempting to download non-existent credits or track lists.

See: https://github.com/get-iplayer/get_iplayer/wiki/installation

See: https://github.com/get-iplayer/get_iplayer/wiki

• Fixed a regression in v3.15 that broke cache refresh on Ubuntu 16.04 and other Linux/BSD distros with obsolete versions of Mojolicious (< 7.13). On Ubuntu 16.04, the problem produced a series of error messages during cache refresh similar to:

    Mojo::Reactor::EV: I/O watcher failed: Can't locate object method "is_success" via package "Mojo::Message::Response"
  

• See v3.15 release notes for other recent changes.

See: https://github.com/get-iplayer/get_iplayer/wiki/installation

See: https://github.com/get-iplayer/get_iplayer/wiki

• Restored --pid-recursive functionality broken by changes to BBC web site.
• Added --pid-recursive-list option. Use instead of --pid-recursive to show list of available episodes without attempting download.
• Removed unused cache fields: versions, categories, guidance. Those fields will be removed the first time you refresh your cache with v3.15.
• The <web> substitution parameter now always expands to the URL of the episode information page on the BBC web site instead of the series/brand home page. This is to bring it back into alignment with the equivalent web cache field. The series/brand home page is always accessible from the episode info page.
• Restored thumbnail images to Web PVR search results. Images will be added incrementally as you update your cache for 30 days after installing v3.15, or you can rebuild your full cache. BBC logo is displayed where thumbnail image is not available. As in previous releases, clicking on thumbnail image displays episode information page on the BBC web site.
• If a schedule page produces no programme listings during cache update (e.g., if page is truncated in transit), it is now flagged as an error. However, this is only relevant if you are using --refresh-abortonerror. Otherwise, cache update will continue.
• Adapted for SSL certificate verification changes in Mojolicious >= 7.80
• Windows: Fixed a regression in v3.14 that prevented get_iplayer from launching in a console configured with a code page unsupported by Perl.
• macOS: An installer package is now available for macOS users. This provides a "double-click installation" of all applications and dependencies on a par with the Windows installer. See the macOS installation instructions via the link below. The installer package is now the only macOS installation method supported by the developer. All macOS users are encouraged to upgrade with the installer.
• macOS: The default output directory is now the "iPlayer Recordings" folder on your desktop. This is the same as the default location on Windows. If you have already configured a custom output directory, it will still be used.

See: https://github.com/get-iplayer/get_iplayer/wiki/installation

See: https://github.com/get-iplayer/get_iplayer/wiki

In April 2018 the BBC removed some media streams, including the 1280x720@25fps streams referred to by the recording mode "hlshd" in get_iplayer. The "hlshd" streams still exist for some repeats and box sets, but in general they are not available for new programmes. Consequently, some changes have been made to get_iplayer recording modes:

• 50fps streams will now be tried first for 1280x720, 960x540, and 704x396 video, with 25fps streams as fallback for 960x540 and 704x396. This means that those TV downloads will roughly double in size and take twice as long compared to previous releases (when using default settings). 50fps HD video requires 2GB+ storage per hour of video.
• Removed --fps50 option as it is now the default
• Added --fps25 option to force use of 25fps streams only (HD video not available). An equivalent "Use only 25fps streams" option was added to the "Recording" tab of the Web PVR.
• Removed all "hls"-prefixed TV and radio modes, except for "hlshd" (1280x720@25fps). Old values are automatically mapped to alternative modes, with warning emitted.
• Although "hlshd" appears to be on the way out, some older programmes, repeats and box sets may still be available with those streams. However, "hlshd" is no longer included in any mode shortcuts or defaults. It must be specified separately. See Recording quality for examples. Check for the presence of "hlshd" streams with --info.
• Removed "vgood" mode shortcuts. Now mapped to "better".
• Removed "worse" mode shortcuts. Now mapped to "good".
• Removed "vhigh" TV quality level (832x468@25fps). Now mapped to "xsd" (960x540@25fps).
• Removed "std" TV quality level (640x360@25fps). Now mapped to "xhigh" (704x396@25fps).
• Added hd,sd,high,low shortcuts for --tvmode and high,std,med,low shortcuts for --radiomode. Prefix with "tv" or "radio", respectively, for use with --modes or Web PVR. These shortcuts provide a means to download only streams of the corresponding resolution or bit rate, without fallback to lower resolution or bit rate as is the case for shortcuts such as "best" or "better". These single-quality shortcuts may be combined. Notes:
  • Don't use --tvmode=hd with --fps25 since HD streams are not available.
  • Don't use --tvmode=hd with audiodescribed programmes since HD streams generally are not available.
  • Don't use --tvmode=med or --radiomode=hd,sd since such streams are not available.
  • Be careful if using --radiomode=high,std for World Service programmes since such streams may only be available for podcast versions.
• HLS ("haf" modes) is now default stream format for radio downloads, with DASH ("daf" modes) as fallback

If you plan to re-encode 50fps HD downloads to smaller 25fps files, you may wish to consider adding the --raw option to get_iplayer downloads and using the resulting .ts or .m4a/.m4v files as re-encoding inputs. That would save a bit of time by not copying the file twice more during remuxing to MP4 and metadata tagging.

• get_iplayer now tops up its cache to 30 days no matter how long it has been since the last update. If you update weekly, you shouldn't notice any difference. If you update after a long gap, the first update will take noticeably longer. The initial update after a new installation will also take noticeably longer since you are building an entire 30-day cache. You can use the --refresh-limit option to restrict the time period covered by a cache update.

• get_iplayer now uses HTTPS everywhere. All internally-generated URLs are HTTPS, and only HTTPS media URLs are used by default. There may be rare cases where streams for a programme are missing because no HTTPS media URLs are available. In that event, use --stream-http to see if any non-HTTPS URLs are available. Removed --include-https option as it is now the default.

• DASH downloads are now put through an extra remux to ensure that output files are compatible with Windows Media Player, iTunes, and other similarly finicky media players. This should affect few users since HLS is the default stream format. If you don't need or don't want the extra post-processing, disable it with --no-dash-remux.

• Use the new --credits and --credits-only options to download programme credits. Credits are available for many TV programmes, and a few radio programmes such as Radio 4 dramas. The credits are downloaded to a plain text .credits.txt file in your output directory. Not configurable in Web PVR.

• Use the new --tag-credits option to append the programme credits decribed above to the output file's "lyrics" metadata tag. You do not need to specify --credits unless you wish to retain the .credits.txt file as well. Not configurable in Web PVR.

• The file extension for track lists has changed from .trk to .tracks.txt.

• Added --available-before option to bracket searches with --available-since. Not configurable in Web PVR.

• Added --thumbnail-square option to retrieve square version of thumbnail image. Not configurable in Web PVR.

• Removed unused --no-hq-audio option

• All non-core Perl modules used by get_iplayer (except those from Mojolicious) are now loaded when the get_iplayer script is compiled before execution. If you are missing some dependencies, you will see error messages similar to:

    Can't locate XML/LibXML.pm in @INC (you may need to install the XML::LibXML module)
  

  EDIT: LWP::Protocol::https is also not explicitly loaded before execution, but it will added in the next release.

See: https://github.com/get-iplayer/get_iplayer/wiki/installation

See: https://github.com/get-iplayer/get_iplayer/wiki

• Implemented the --pvr-series option, which provides a shortcut to create simple PVR searches to record specific series. This is analogous to an "Add Series" link in the Web PVR. For examples, see the PVR usage section of the documentation.
• Implemented the --mark-downloaded option to prevent downloading programmes found in search results that you already have or do not want. This is done by inserting records for one or more programmes into the download history. These entries can still be overridden by --force. Not configurable in Web PVR. For examples, see the Recording programmes section of the documentation.
• Implemented the --subs-embed option, which embeds soft subtitles in the MP4 output file. Embedded subtitles can be used by iTunes, Apple TV, and some other media players that do not support external .srt files. The normal .srt files are still created.
  • Embedding is done by ffmpeg when converting the raw download to MP4, so there is a possibility that problems with the subtitles may prevent the MP4 conversion from completing.
  • Colours likely will be ignored in embedded subtitles, so you may wish to use --subs-mono.
• Use the new --tracklist and --tracklist-only options to download music track lists available for some programmes. Track lists are generally available for radio music programmes, but may exist for some TV programmes, such as Top of the Pops. The track list is downloaded to a plain text .trk file in your output directory. Not configurable in Web PVR. Thanks to @StevenMaude for the original idea and the Python implementation.
  • Where available, track timings are included. Track timing data is usually only available for music programmes on national radio stations. The track timing data also may be incorrect at source.
  • When track timing data is available and --start and --stop are used, all tracks are included in the list, but only tracks between --start and --stop will have timings.
• Use the new --tag-tracklist option to append the music track list decribed above to the output file's "lyrics" metadata tag. You do not need to specify --tracklist unless you wish to retain the .trk file as well. Not configurable in Web PVR.
• The output when using --pid or --url has changed slightly:
  • When using multple --pid or --url options (or --pid-recursive) all associated episodes are displayed in a single list before any downloads commence, rather than looping through those related to each --pid or --url option in turn.
  • If the value of --pid is a series or brand PID, get_iplayer will no longer retrieve and display an episode list unless --pid-recursive is also specified. A warning is shown instead.
  • The --hide option is now applied to episode lists generated with --pid and --url and --pid-recursive
• The raw subtitles file produced with --subs-raw now has a file extension of .ttml rather than .ttxt. This allows the file to be recognised as a subtitles file by VLC 3.0+.
• Fixed a bug that prevented subtitles from being trimmed with --start and --stop.
• Restored --pid-recursive functionality for CBBC and CBeebies, which was broken by changes to BBC web sites.
• Restored cache updating, which was broken by changes to BBC web sites. If you find search results missing programmes from the week of 19 Feb, rebuild your cache with --rebuild-cache to fill any holes.

See: https://github.com/get-iplayer/get_iplayer/wiki/installation

See: https://github.com/get-iplayer/get_iplayer/wiki

• Fixed a bug introduced in v3.11 that prevented programme version names from being shortened to one word, as in previous releases. With default file naming in force, this could cause duplicate downloads with --force even if previous download was in the same directory since the original file would have a slightly different name. For example, a file name may contain "originalversion" instead of just "original".
  • If you rename any files downloaded with v3.11, you also may wish to change the names in your download history (see FAQ list for location). However, this is only useful if you delete or play files via the web PVR Recordings tab. The file path in the history is not used for the normal overwrite check during downloading.

See get_iplayer 3.09 release notes for other recent changes.

See: https://github.com/get-iplayer/get_iplayer/wiki/installation

See: https://github.com/get-iplayer/get_iplayer/wiki

• Fixed a bug that generated incorrect schedule URLs (used for cache refresh) for the first calendar week of 2018 (and some future years). (@welwood08)
  • This would be a potential problem only if you plan to use --cache-rebuild or --refresh-limit. For example, if you did not update your cache at all between 1 Jan and 14 Jan and then attempt to backfill it with --cache-rebuild during the period 15 Jan to 31 Jan, you will need this fix to ensure that programmes from the week of 1 Jan are not omitted from the update.
  • If you have been updating your cache in the normal manner, on a weekly (or more frequent) basis, it should be intact. There would be no need to rebuild it, so this change will not affect you.
  • This fix means that --cache-rebuild and --refresh-limit will no longer work by default with Perl 5.8 because they now rely on the Time::Piece module, which is a core module only in Perl 5.10+. If you use Perl 5.8 - which is no longer formally supported - you should be able to restore --cache-rebuild or --refresh-limit functionality by installing the Time::Piece module separately.
• Fixed a bug that caused resuming partial downloads to fail if the programme version contained a non-alphanumeric character, e.g., the hyphen in "pre-watershed". (@notnac)
• Fixed a spurious (but harmless) "ERROR: Search term(s) required." message when using --cache-rebuild if you also have any search options (e.g., --since, --channel) saved in your preferences.

See get_iplayer 3.09 release notes for other recent changes.

See: https://github.com/get-iplayer/get_iplayer/wiki/installation

See: https://github.com/get-iplayer/get_iplayer/wiki

get_iplayer v3.10 contains only one minor change:

• Fixed a spurious (but harmless) "ERROR: Search term(s) required." message when using --refresh if you also have any search options (e.g., --since, --channel) saved in your preferences. (@welwood08)

See get_iplayer 3.09 release notes for other recent changes.

See: https://github.com/get-iplayer/get_iplayer/wiki/installation

See: https://github.com/get-iplayer/get_iplayer/wiki