using_the_cli - Sidies/MasterThesis-HubLink GitHub Wiki

The SQA system provides a command-line interface (CLI) to interact with the system. In this wiki page, we will cover the basic usage of the CLI.

To start using the CLI, you need to open a terminal and navigate to the directory where the CLI is located.

cd [path_to_sqa_system]
cd sqa_system/app/

Inside of the sqa-system/sqa_system/app/ directory, is the file cli_controller.py. Running this file will start the CLI. You can run the following command:

python cli_controller.py

This will start the CLI. You console should look like this:

> python cli_controller.py 

Rotating log file
2025-05-30 15:41:53,682 - New session started
Welcome to the Question Answering System!
? What would you like to do? 
❯ Run a pipeline
  Run an experiment
  Manage secrets
  Manage configurations
  Exit

You now have multiple options to choose from. You can run a pipeline, run an experiment, manage secrets, manage configurations, or exit the CLI.

Secrets are sensitive information that should not be exposed in the code such as API keys, emails and passwords. The CLI uses an encrypted file to store secrets on your local machine. You can view, add, or remove secrets using the CLI over the "Manage secrets" option.

Once you select the "Manage secrets" option, you will be presented with the following options:

? What would you like to do? Manage secrets
Preparing secret manager...
? Manage Secrets 
❯ Add a secret
  Delete a secret
  List secrets
  Back to main menu

You can add a secret, delete a secret, list all secrets, or go back to the main menu.

The configuration system of the SQA system allows control the behavior of pipelines and experiments. Because each component of the system has its own configuration, which can be complicated to create by hand, the CLI assists you in the creation of these configurations. This ensures that the configurations are valid.

Once you select the "Manage configurations" option, the system will load all configurations that are currently present in the sqa-system/data/configs/ directory. Furthermore, any configuration that you create will be saved in this directory. Therefore, if you want to use your created configurations elsewhere, you can simply copy them from this directory.

You will be presented with the following options:

? Select the configuration to manage 
❯ Manage LLM configurations
  Manage dataset configurations
  Manage chunking configurations
  Manage embedding configurations
  Manage vector store configurations
  Manage knowledge graph configurations
  Manage pipe configurations
  Manage pipeline configurations
  Manage experiment configurations
  Manage evaluator configurations
  Back to main menu

These allow you to manage the configurations for different components of the system. You can select any of these options to view, add, or remove configurations for that specific component. For example, if you select the knowledge graph configurations, you will be presented with the following options:

? Knowledge Graph Database Management: 
❯ List Knowledge Graph Databases
  Add a Knowledge Graph Database
  Remove a Knowledge Graph Database
  Back to main menu

You can list the configurations, add a new configuration, remove an existing configuration, or go back to the main menu. The same applies to all other components of the system. Once you select to add a new configuration, a guided process will start to help you create the configuration. This will ensure that the configuration is valid and can be used in the system.

Another option in the CLI is to run a pipeline. This is an interactive mode that allows to select a configuration which is then used to create a pipeline that can then be queried by asking questions.

Once you select the "Run a pipeline" option, you will be presented with the following options:

? Select which pipeline to run 
❯ ORKG GPT-4o-mini on the ORKG
  Back

We have already prepared a pipeline by default that uses the HubLink approach on the ORKG using the GPT-4o-mini model. You can use this configuration to try out HubLink on the ORKG, however, note that this requires the indexing process to take place which costs some money (should be less than 5€ beause we are using GPT-4o-mini). Also be aware that the performance might not be optimal as we are using a small model here. If you want to use a different model, you can create your own pipeline configuration using the "Manage configurations" option or directly manipulate the pipeline configuration file in the sqa-system/data/configs/pipeline_configs/ directory.

Once you select a pipeline, the SQA system will automatically prepare the pipeline and knowledge graph. You will then be able to ask questions and receive answers.

You can also run an experiment using the CLI, though that is not the way we recommend to run experiments. It is better to use a dedicated experiment folder with a script that runs the experiment for reproducibility purposes. This is how we conducted the experiments as you can see in the sqa-system/experiments/ directory.

Once you select the "Run an experiment" option, you will be presented with the following options:

? Select which experiment to run 
❯ HubLink with GPT-4o-mini on ORKG
  Back

As you can see, we already prepared an experiment configuration that uses the HubLink approach on the ORKG using the GPT-4o-mini model with the KGQA dataset that we used in our experiments. You can use this configuration to try out the experiment, however, note that this requires the indexing process to take place which costs some money (should be less than 5€ because we are using GPT-4o-mini). Also be aware that the performance might not be optimal as we are using a small model here. If you want to use a different model, you can create your own experiment configuration using the "Manage configurations" option or directly manipulate the experiment configuration file in the sqa-system/data/configs/experiment_configs/ directory.