How to Sync Data from FIO? - ebitkov/prun-community-report GitHub Wiki

Instead of accessing the FIO API directly with every request, this project includes a sync tool that transfers and caches the data in a local database. This not only reduces loading times but also makes it possible to structure and aggregate the data.

To see all available sync commands, run php bin/console fio --help in the root directory of the project.

Sync All of FIO

You can sync all required data from FIO by running php bin/console fio:sync. This will queue up all required sync commands into the Symfony Messenger. Please note, that executing this command isn't actually importing any data. To run the imports you'll have to start a message consumer:

php bin/console messenger:consume command -l 1000

[!Note] Using the -l option limits the consumer to 1000 commands before it shuts itself down automatically. This is required to prevent exceeding the memory. You'll need to restart the consumer manually a few times, if you're planning on importing all of FIO.

[!Note] The runner won't tell you its progress, nor will it shut down by itself when it's ready. You can check the messenger_messages table in the database to check the queue. Processed messages will be removed after completion, so an empty table means everything is processed.

Sync a Region

Importing all data from FIO takes a lot of time (multiple hours). You can filter the dataset by setting the --region option:

php bin/console fio:sync --region antares

This will only include planets in the Antares region, reducing the overall import time massively.