• INI File Location
  • Command Line Interface(CLI)
  • Calibre Plugin
  • Calibre Plugin CLI
• INI File Sections
  • Section Precedence
    • defaults(#defaults)
    • site(#site)
    • format(#format)
    • site:format(#site__format)
    • overrides(#overrides)
  • Section Complications
    • defaults.ini vs personal.ini
    • [www.domain] Sections
    • Duplicated Sections
    • Base Adapter Sections
    • Individual Story URL Sections
• Modifying Keyword Settings
  • extra_keywords
  • extrametadata
  • add_to_keyword
  • keyword_filelist
• CLI Command Line Options

INI File Location

Command Line Interface(CLI)

The CLI loads personal.ini files from the present working directory (./personal.ini), ~/.fanficfare/personal.ini, $XDG_CONFIG_HOME/fanficfare/personal.ini and ~/.fanficdownloader/personal.ini (for backward compatibility), in that order of precedence. IE, if you have both ~/.fanficfare/personal.ini and ./personal.ini both are read and used, but the contents of ./personal.ini take precedence.

Calibre Plugin

The personal.ini 'file' is stored in each Calibre library's database and can accessed from the FanFicFare plugin configuration dialog:

Note that each Calibre library has separate FanFicFare settings. The settings from the previously opened library will be copied to a new library when you first change FanFicFare settings in the new library (assuming a different library was open in the same session).

Calibre Plugin CLI

When invoked by CLI, the plugin version will use the copy of CLI defaults.ini packaged in the plugin ignoring any other defaults.ini. personal.ini will come from ./personal.ini, then ~/.fanficfare/personal.ini and finally your personal.ini settings from the default (or selected) Calibre library in that order of precedence.

INI File Sections

The personal.ini settings control many FanFicFare features and give you control over many of the details of how FanFicFare works. But the different keyword settings will or will not apply depending on what section they appear in and when.

This document is intended to explain how the precedence of different sections and keywords work. As of this writing (April 2016), the defaults.ini file is still the best reference of all FanFicFare INI settings (CLI and Calibre Plugin have slightly different versions).

Section Precedence

The basic precedence order for sections (using an epub format download from the non-existent site test1.com as an example) from least to most is:

1. [defaults]
2. [test1.com]
3. [epub]
4. [test1.com:epub]
5. [overrides]

A setting keyword in [defaults] is the most general version and will be 'overridden' by the same keyword in any higher precedence (IE, more specific) section. A keyword in [overrides], the highest precedence section, overrides the settings in any other section.

[defaults]

[defaults] is intended to hold keywords you generally want applied to all of your downloads, but might want to be set differently for certain formats, sites, or specific site:formats. is_adult:true is probably the most common; others include: mark_new_chapters, include_subject_tags (changes which metadata categories will be included in Calibre tags), strip_chapter_numbers/add_chapter_numbers, etc.

[site]

Which site section ([test1.com] in the example list) is applied depends on which site the story is being downloaded from. The most common use for site sections is to save your username and password for each site. Some sites have special keywords that only apply to those specific sites; for example [www.fanfiction.net]'s check_next_chapter keyword.

[format]

[epub] is only applied when downloading to an epub file, likewise for [html], [mobi], and [txt]. The expectation is that settings in these sections will be specific to output formatting. output_css, for example.

It can be argued that the site section should have higher precedence than the format section. However, in practice, there's generally little overlap between the keywords you want to set by site and by format. For those cases there is:

[site:format]

The site:format section ([test1.com:epub] in the example list) section is applied only when both the site and format match.

[overrides]

[overrides] is intended to hold keywords you want applied always, without exception. Generally, it should be possible (and better) to use [defaults].

Section Complications

Like so many things in life, right after the list of rules comes the list of exceptions and complications.

defaults.ini vs personal.ini

The interactions between defaults.ini and personal.ini usually work out pretty intuitively, but they can occasionally trip you up. FanFicFare reads defaults.ini first, and then personal.ini which 'overwrites' any of the same keywords in the same section. Effectively the basic precedence list is:

1. defaults.ini - [defaults]
2. personal.ini - [defaults]
3. defaults.ini - [test1.com]
4. personal.ini - [test1.com]
5. defaults.ini - [epub]
6. personal.ini - [epub]
7. defaults.ini - [test1.com:epub]
8. personal.ini - [test1.com:epub]
9. defaults.ini - [overrides]
10. personal.ini - [overrides]

Which does mean that [epub] from defaults.ini is higher precedence than [test1.com] from personal.ini. As mentioned above, site and format sections will rarely use the same keywords. But they can, which is why there are site:format sections.

[www.domain] Sections

Some sites use domain names beginning with www, some do not. Most, but not all, will work with either. So unfortunately, FanFicFare can't just declare that the canonical domains for all sites will (or won't) have www.

After a few times when users' settings didn't work because they had/had not included www, FanFicFare was modified to accept [site] sections both with and without www for all sites; with www is lower precedence than without, ie a setting under [domain] will override the same setting under [www.domain].

This also means [domain] in defaults.ini will take precedence over [www.domain] in personal.ini. You could use [domain] sections rather than [www.domain] in your personal.ini as a general rule and be fine in most cases, but you could see unexpected effects with add_to_X if defaults.ini uses [www.domain].

In general, it will work best to match the [section] name used by that site in defaults.ini.

1. [defaults]
2. [www.test1.com]
3. [test1.com]
4. [epub]
5. [www.test1.com:epub]
6. [test1.com:epub]
7. [overrides]

Duplicated Sections

And here's what happens when you do add the same section more than once. (It's actually not hard to do if you're copying snippets into your personal.ini without stopping to look for a previously existing section.)

Python's INI parser is actually perfectly happy to have the same section duplicated. It will even work most of the time. The problem comes when the same keyword is in both duplicate sections; then the last section overrides the earlier ones.

[www.fanfiction.net]
# Works fine because not included in later section
check_next_chapter:true

# FAILS because replaced by replace_metadata in later section.  You can spend hours
# trying different things and they'll never work here.
replace_metadata:
 author=>Bobmin356=>Bobmin

<other sections>

[www.fanfiction.net]
# Overrides replace_metadata from earlier section.
replace_metadata:
 title=>WatLP: Chamber of Secrets=>Chamber of Secrets

Base Adapter Sections

There are a couple classes of sites that share code and, for convenience, have another level of INI section. These sites are those that use base_xenforoforum or base_efiction adapters. So, for example, when downloading to epub from forums.spacebattles.com the precedence order is:

1. [defaults]
2. [base_xenforoforum]
3. [forums.spacebattles.com]
4. [epub]
5. [forums.spacebattles.com:epub]
6. [overrides]

There are a number of keyword settings specific to base_xenforoforum adapters in defaults.ini that would otherwise have to be duplicated between the different base_xenforoforum adapter site sections. It's also a convenient place in personal.ini to apply customization to all base_xenforoforum adapter sites at once.

Individual Story URL Sections

A recent addition to FanFicFare in early 2016 was individual story URL INI sections.

1. [defaults]
2. [base_xenforoforum]
3. [forums.spacebattles.com]
4. [epub]
5. [forums.spacebattles.com:epub]
6. [https://forums.spacebattles.com/threads/384594/]
7. [overrides]

This lets you apply keyword settings to specific stories. Here are a couple examples:

[https://forums.spacebattles.com/threads/346640/]
always_include_first_post:true
first_post_title:Prologue and Chapter 1
extratags: Original, Sci-Fi, AI

[https://forums.spacebattles.com/threads/384594/]
replace_br_with_p:true

Modifying Keyword Settings

Not terribly long after defining the sections and precedence, it became clear that it would also be useful to be able to add to certain keyword lists without completely replacing them.

extra_keywords

First, a small handful of extra_ keywords were created, but they are not general, nor even consistent. Luckily, the list is short: extra_titlepage_entries, extra_logpage_entries and extra_subject_tags (which adds to include_subject_tags).

(extra_valid_entries is not considered an extra_ keyword for this discussion. There is not a corresponding valid_entries keyword--the initial valid entries list is hard coded. I did mention it's inconsistent...)

Take titlepage_entries as an example. When titlepage_entries is used, both titlepage_entries and extra_titlepage_entries are fetched and concatenated (as a list). Both follow the ordinary section precedence rules. So you can set extra_titlepage_entries in [test1.com] without overwriting titlepage_entries from [defaults]. But only once. If you set extra_titlepage_entries in [epub], it overwrites the version from [test1.com].

It's not recommended to use the extra_ keywords (extra_valid_entries excepted, see above) when adding new settings to your personal.ini, but they are not (yet) officially deprecated.

extrametadata

While on the subject, there are also several similar, but unrelated extrametadata keywords (extratags, extracategories, extragenres, extracharacters, extraships and extrawarnings). These are used to explicitly add values to the categories, genres, etc, metadata lists. This is used mostly in defaults.ini for fandom/character/ship specific sites.

For example, www.dracoandginny.com is specifically for HP fics featuring Draco Malfoy/Ginny Weasley, so in defaults.ini there is:

[www.dracoandginny.com]
...
## Site dedicated to these categories/characters/ships
extracategories:Harry Potter
extracharacters:Draco Malfoy,Ginny Weasley
extraships:Draco Malfoy/Ginny Weasley

In defaults.ini [defaults] the setting extratags:FanFiction adds the tag 'FanFiction' by default to all downloaded stories. FanFicFare does support a few sites that are original fiction instead and for those sites, defaults.ini has extratags: (keyword with no list--clearing the setting from [defaults]) under [finestories.com], [www.fictionpress.com], etc.

add_to_keyword

As new keyword settings were added over time, and users' configurations became more and complex (especially replace_metadata), it became clear that a better mechanism for handling additive keywords between sections was needed than the ad-hoc extra_ keywords had been.

The add_to_ mechanism is designed to be generalized for all INI keywords and add more flexibility to allow more specific (higher precedence) sections to 'add to' the keyword settings from more general sections without having to repeat everything.

When the value for a keyword setting (eg, replace_metadata) is retrieved, FanFicFare first gets the keyword settings from the highest precedent section it can be found in as normal. Then it looks for the same keyword with add_to_ prepended (eg, add_to_replace_metadata) in every section in the precedence list it can be found in and concatenates it (as a string) with the previous value. The interesting part is that each section's add_to_s are applied from most general (lowest precedence) to most specific (highest precedence). More specific add_to_s appear after more general ones.

Here's an example:

[defaults]
add_to_replace_metadata:
 genre=>Comedy=>Humor

[base_xenforoforum]
add_to_replace_metadata:
 category=>StarTrek=>Star Trek
 category=>Warhammer 40,000=>Warhammer 40k

[forums.spacebattles.com]
add_to_replace_metadata:
 title=>Wail of the Banshee 3: Boxhead Blues and the Zombocalypse=>Wail of the Banshee

When downloading a story from forums.spacebattles.com, the effective replace_metadata becomes:

replace_metadata:
 genre=>Comedy=>Humor
 category=>StarTrek=>Star Trek
 category=>Warhammer 40,000=>Warhammer 40k
 title=>Wail of the Banshee 3: Boxhead Blues and the Zombocalypse=>Wail of the Banshee

(Because there's no replace_metadata without add_to_, its value was empty when it started adding.)

The 'gotcha' about add_to_ is that it's a string concatenation. Which means you need to include a leading comma(,) when the keyword is a list like titlepage_entries instead of a string like replace_metadata:

[epub]
titlepage_entries: seriesHTML,category,genre,...,description

[forums.spacebattles.com]
add_to_titlepage_entries:,tagsfromtitle,forumtags
# must have leading comma(,) because add_to_ is string concat.  Otherwise 
# ...,descriptiontagsfromtitle,forumtags

There's another non-intuitive complication in the interaction between defaults.ini/personal.ini and add_to_: Because of the way the INI parser is used, each keyword in each section of personal.ini overwrites the same keyword in the same section of defaults.ini. For example, if this occurs:

defaults.ini

[base_xenforoforum]
add_to_extra_valid_entries:,tagsfromtitle,forumtags

personal.ini

[base_xenforoforum]
add_to_extra_valid_entries:,othertagsthing

...the add_to_extra_valid_entries:,tagsfromtitle,forumtags from defaults.ini is discarded in favor of add_to_extra_valid_entries:,othertagsthing.

keyword_filelist

This was requested by davidfor for output_css, but I've generalized.

1. Instead of using output_css in personal.ini, you can put your CSS in a file and use output_css_filelist to pull it in automatically. But it will work for other settings as well.

2. keyword_filelist is a comma separated list of URLs. This includes http, https and file URLs. The contents of all URLs will be concatenated and used as one blob of text.

3. If you use keyword_filelist, keyword values will be ignored; even if they appear in a more specific [section]. If you choose to use _filelist settings, you'll have to be a bit more careful than usual.

4. add_to_keyword_filelist works--you can add additional, more specific files to the filelist.

5. While keyword is ignored in favor of keyword_filelist, add_to_keyword is still applied, letting you still add text directly in personal.ini.

6. When moving output_css from personal.in to a file or URL, be aware that percent(%) substitutions done by the INI parser are not done to content loaded via _filelist.

   background_color: ffffff
   
   output_css:
    body { background-color: #%(background_color)s; }
   

   Needs to instead be:

   body { background-color: #ffffff; }
   

7. Similarly, where percentages in INI need to be escaped with %%, they should not be in _filelist files:

    .full     {width: 100%%; }
   -- vs --
    .full     {width: 100%; }
   

8. The plugin performs some syntax and safety checks on replace_metadata, custom_columns_settings and generate_cover_settings. These checks are not performed on _filelist loaded settings.

9. The INI parser used on defaults.ini and personal.ini tends to remove leading and trailing whitespaces. Settings loaded using _filelist are not subject to that. I don't believe it will cause problems, but it's possible regexps may be effected.

Here's a short example using output_css:

[epub]
output_css_filelist:https://www.somesite.com/epub.css
# - or - 
output_css_filelist:file:///C:/Users/user/Desktop/FanFicFare/epub.css

add_to_output_css:
 .ADDED_epub { border: 1px solid black; padding: 2px; }

[defaults]
add_to_output_css:
 .ADDED_default { border: 1px solid black; padding: 2px; }

[test1.com]
add_to_output_css_filelist:,https://www.somesite.com/epub2.css

add_to_output_css:
 .ADDED_test1 { border: 1px solid black; padding: 2px; }

The final resulting CSS is:

#WEB1 { display: none; }
...
#WEB2 { display: none; }
...
.ADDED_default { border: 1px solid black; padding: 2px; }
.ADDED_test1 { border: 1px solid black; padding: 2px; }
.ADDED_epub { border: 1px solid black; padding: 2px; }

CLI Command Line Options

The Command Line Interface(CLI) version allows keywords to be set at run time from the command line with -o NAME=VALUE, --option=NAME=VALUE.

Keyword settings given on command line are automatically included as if they were placed in personal.ini [overrides] section, replacing the same settings, if they exist in the actual personal.ini [overrides] section.