MetaParameters_v2 - Strategic-Futures-Lab/Topic_Mapping_Pipeline GitHub Wiki

Topic Mapping Pipeline


Meta-Parameters

The meta-parameters allow you to complete or overwrite module specifications at a project-wide scope.

Directories

{...
  "metaParameters": {
    "projectDir": "files/",
    "sourceDir": "projects/test/",
    "dataDir": "output/tmp/",
    "outputDir": "output/",
    ...
  },
... }

The directories meta-parameters allow you to set the directory of data file for the whole project.

Name Description
projectDir The top-level directory for the project.
sourceDir The directory of any of the input sources.
dataDir The directory for all the temporary data files generated.
outputDir The directory for all the final data files generated.

Examples of input sources:

  • input corpus files/directories for Input Modules;
  • previous serialised models for the Inference Module as well as previous distribution files or map files to compare and/or overwrite after having inferred documents.

Examples of temporary data files:

Examples of final data files:

All of these parameters are optional, and will only come to complete the filenames used at module level with the structure:

projectDir + [ sourceDir | dataDir | outputDir ] + filename

By default, they will be set to "", hence not affect the filenames set by modules.

Model Type

{...
  "metaParameters": {
    ...
    "modelType": "hierarchical",
    ...
  },
... }

Because modules after Model will behave differently depending on the nature of the model made, you can use the meta-parameter modelType to quickly set those behaviours. It takes two possible values: hierarchical and simple. It is optional, and will overwrite the module-level specification if set.

Document Fields

{...
  "metaParameters": {
    ...
    "docFields": ["title", "university", "money", "authors", "date"]
  },
... }

Lemmatise, InferDocuments and ExportModel are modules which will set or export a list of document data fields. Because it is often the case that those lists are identical, you can use the meta-parameter docFields to set these module specifications in one place. It is optional, and will overwrite the module-level specification if set.

⚠️ **GitHub.com Fallback** ⚠️