GT7 Sync - jegr78/ctc-manager GitHub Wiki
GT7 Sync
GT7 Sync scrapes the official Gran Turismo 7 website to import car and track master data into the application.
Overview
| Component | Class | Purpose |
|---|---|---|
| Scraper | Gt7ScraperService |
Fetches and parses data from gran-turismo.com |
| Sync | Gt7SyncService |
Merges scraped data with the local database |
| Controller | Gt7SyncController |
Admin UI for triggering sync operations |
| Preview | Gt7SyncPreview |
Preview model for review before import |
Car Scraping
- Source:
https://www.gran-turismo.com/gb/gt7/carlist/ - Parses JavaScript bundles embedded in the page to extract:
- Car names
- Manufacturer mapping
- GT7 internal IDs
- Car images
- Handles the site's dynamic rendering by parsing the JS data structures directly
Track Scraping
- Source:
https://www.gran-turismo.com/gb/gt7/tracklist/ - Two scraping modes:
| Mode | Speed | Data |
|---|---|---|
| Fast | Quick | Track names and IDs only |
| Full | Slower | Names, IDs, and track images |
Image Download
- Images are downloaded in parallel using
CompletableFuturefor performance - Stored locally in the application's upload directory
- Referenced by the Car and Track entities
Sync Workflow
- Trigger sync from the admin UI (GT7 Sync page)
- Preview mode: Review scraped data before committing
- Shows new cars/tracks to be added
- Shows existing entries that will be updated
- Duplicate detection: Matches by GT7 ID to avoid duplicates
- Execute import: Confirmed changes are persisted to the database
GT7 ID Tracking
Each car and track stores its GT7 ID, enabling:
- Reliable duplicate detection across syncs
- Stable references even if names change on the source website
- Incremental updates (only new/changed entries are imported)
Demo Profile
When running with the dev,demo profile combination:
./mvnw spring-boot:run -Dspring-boot.run.profiles=dev,demo
The application automatically imports all GT7 cars and tracks (with images) at startup. This provides a fully populated database for manual testing without requiring a manual sync trigger.