GT7 Sync - jegr78/ctc-manager GitHub Wiki

GT7 Sync

GT7 Sync scrapes the official Gran Turismo 7 website to import car and track master data into the application.

Overview

Component Class Purpose
Scraper Gt7ScraperService Fetches and parses data from gran-turismo.com
Sync Gt7SyncService Merges scraped data with the local database
Controller Gt7SyncController Admin UI for triggering sync operations
Preview Gt7SyncPreview Preview model for review before import

Car Scraping

  • Source: https://www.gran-turismo.com/gb/gt7/carlist/
  • Parses JavaScript bundles embedded in the page to extract:
    • Car names
    • Manufacturer mapping
    • GT7 internal IDs
    • Car images
  • Handles the site's dynamic rendering by parsing the JS data structures directly

Track Scraping

  • Source: https://www.gran-turismo.com/gb/gt7/tracklist/
  • Two scraping modes:
Mode Speed Data
Fast Quick Track names and IDs only
Full Slower Names, IDs, and track images

Image Download

  • Images are downloaded in parallel using CompletableFuture for performance
  • Stored locally in the application's upload directory
  • Referenced by the Car and Track entities

Sync Workflow

  1. Trigger sync from the admin UI (GT7 Sync page)
  2. Preview mode: Review scraped data before committing
    • Shows new cars/tracks to be added
    • Shows existing entries that will be updated
  3. Duplicate detection: Matches by GT7 ID to avoid duplicates
  4. Execute import: Confirmed changes are persisted to the database

GT7 ID Tracking

Each car and track stores its GT7 ID, enabling:

  • Reliable duplicate detection across syncs
  • Stable references even if names change on the source website
  • Incremental updates (only new/changed entries are imported)

Demo Profile

When running with the dev,demo profile combination:

./mvnw spring-boot:run -Dspring-boot.run.profiles=dev,demo

The application automatically imports all GT7 cars and tracks (with images) at startup. This provides a fully populated database for manual testing without requiring a manual sync trigger.