Flyway - Yash-777/MyWorld GitHub Wiki

Flyway is an open-source database-migration tool

There are 2 approaches when it comes to a database build:

  • Full build from scratch [flyway:clean, flyway:migrate] – spin up a new, empty database or clean a database dedicated for builds and then apply all the migration scripts to take it from 0 to the latest version

  • Incremental builds [flyway:migrate] – have an existing database dedicated for builds. Do not clean it. Just run flyway migrate to apply the new migration scripts

All These Migrations file history is maintained in flyway_schema_history table. It will only execute the latest migration file and add its entry in flyway_schema_history table.

select * from flyway_schema_history;
Command & Description Image
Clean: Drops all objects in the configured schemas. If Flyway automatically created them, then the schemas themselves will be dropped when cleaning. image
Migrate:Migrates the schema to the latest version. Flyway will create the schema history table automatically if it doesn't exist. image

Flyway Naming Patterns: The default naming patterns within Flyway are documented clearly. You have a prefix that determines the type of file, whether a versioned migration (V), an undo migration (U), or a repeatable migration (R).

Flyway adheres to the following naming convention for migration scripts: <Prefix><Version>__<Description>.sql. Example: V1_1__My_description.sql

Naming Patterns Description
<Prefix> The default prefix is V, which we can change in the above configuration file using the flyway.sqlMigrationPrefix property.
<Version> Migration version number. Major and minor versions may be separated by an underscore.
The migration version should always start with 1.
<Description> Textual description of the migration. A double underscore separates the description from the version numbers.

Specifying the naming conventions

flyway.sqlMigrationPrefix File name prefix for versioned SQL migrations (default: V), For example, V1_1__My_description.sql if the default naming is used.
flyway.sqlMigrationSeparator File name separator for SQL migrations. The default is a double underscore (__) which would look like V1_1__My_description.sql
flyway.sqlMigrationSuffixes This is a comma-separated list of file name suffixes for SQL migrations. (default: .sql). Using the defaults, this looks like V1_1__My_description.sql. This allows several different suffixes (like .sql, .pkg, .pkb) for easier compatibility with other tools that require specific file types for use.

Configuration Order Settings are loaded in the following order (higher items in the list take precedence over lower ones): Flyway provides several ways to configure the necessary properties such as flyway.url, flyway.user, and flyway.password for database migrations. These configurations can be provided through

Flyway read the necessary properties in several ways
  1. Command-line arguments

    You can specify Flyway configuration properties directly in the command-line using -D flags. This is useful when you want to override configuration files or set properties for a single migration run.

    Default Configuration Behavior: Flyway will look for the following default properties:

    mvn --non-recursive 
    	-Dflyway.url=jdbc:mysql://localhost:3306/ \
    	-Dflyway.user=databaseUser \
    	-Dflyway.password=databasePassword \
    	-Dflyway.schemas=schemaName \
    	flyway:clean flyway:migrate
    • flyway.url: The JDBC connection URL to the database.
    • flyway.user: The username for database authentication.
    • flyway.password: The password for database authentication.
    • flyway.schemas: The default schemas Flyway will use
  2. Environment variables

    You can configure Flyway using environment variables, which is useful for keeping sensitive data (like passwords) out of the command line or configuration files. Flyway automatically detects certain environment variables for configuration.

    export FLYWAY_URL=jdbc:mysql://localhost:3306/mydb
    export FLYWAY_USER=root
    export FLYWAY_PASSWORD=root
    export FLYWAY_SCHEMAS=mydb

    After setting these variables, you can run Flyway commands without specifying them explicitly:

    flyway migrate
    

    Alternatively, if you're running the migrations with Maven, you can set environment variables before the Maven command to pass them to Flyway:

    export FLYWAY_URL=jdbc:mysql://localhost:3306/mydb
    mvn flyway:migrate
  3. Custom config files Flyway allows you to specify custom configuration files for more advanced and persistent configuration management. A custom config file lets you keep your Flyway settings in a separate file that Flyway will read at runtime.

    Example of flyway.conf File: You can create a custom configuration file (e.g., flyway.conf or any other name you specify) and set the necessary properties inside that file:

    flyway.url=jdbc:mysql://localhost:3306/
    flyway.user=root
    flyway.password=root
    flyway.schemas=mydb
    flyway.locations=filesystem:db/migrations

    You can tell Flyway to use this file by specifying it in the command:

    mvn flyway:migrate -Dflyway.configFiles=resources/config/flyway.conf
  4. Flyway command-line defaults

    Flyway has built-in defaults for many properties. If no value is provided for a specific property, Flyway will fall back on default values.

This means that if flyway.url is both present in a config file and passed as -url= from the command-line, the command-line argument will take precedence and be used.


Plugin configuration : Pom.xml

<plugin>
    <groupId>org.flywaydb</groupId>
    <artifactId>flyway-maven-plugin</artifactId>
    <version>8.5.13</version>
    <dependencies>
        <!-- Flyway -->
        <dependency>
            <groupId>org.flywaydb</groupId>
            <artifactId>flyway-core</artifactId>
            <version>8.0.5</version>
        </dependency>
        <dependency>
            <groupId>com.mysql</groupId>
            <artifactId>mysql-connector-j</artifactId>
            <version>8.0.33</version>
        </dependency>
    </dependencies>
</plugin>

Configuration Files See configuration for a full list of supported configuration parameters.

The default configuration file name is flyway.conf and by default it loads configuration files from:

  • installDir/conf/flyway.conf
  • userhome/flyway.conf
  • workingDir/flyway.conf

If we use any other name (e.g customConfig.conf/myFlywayConfig.conf) as the configuration file, then we have to specify this explicitly when invoking the Maven command: {workingDirectory}/resources/config/local/ flyway.conf / myFlywayConfig.conf

# Enable schema creation (if the schema doesn't already exist)
flyway.createSchemas=true
# Database connection settings
flyway.url=jdbc:mysql://localhost:3306/
flyway.user=root
flyway.password=root
# Define the schema(s) Flyway should work with (flyway.schemas=schema1,otherschema)
flyway.schemas=schema1
# Optional: Default schema to use for migrations
flyway.defaultSchema=schema1
# Disable placeholder replacement (if not using placeholders in scripts)
flyway.placeholderReplacement=false

Maven command to run:

mvn --non-recursive 
	-Dflyway.configFiles=resources/config/local/flyway.conf 
	-Dflyway.locations=filesystem:resources/mySqlDB/migration/common,filesystem:resources/mySqlDB/migration/env 
	flyway:clean 
	flyway:migrate

OutPut:

[INFO] --- flyway-maven-plugin:8.5.13:migrate (default-cli) @ MyApp ---
[INFO] Flyway Community Edition 8.0.5 by Redgate
[INFO] Database: jdbc:mysql://localhost:3306/ (MySQL 8.0)
[INFO] Creating schema `schema1` ...
[INFO] Creating Schema History table `schema1`.`flyway_schema_history` ...
[INFO] Current version of schema `schema1`: null

On upgrading flyway core module faced following issue: Add JDBC driver MySQL specific dependencies

Failed to execute goal org.flywaydb:flyway-maven-plugin:8.5.13:clean (default-cli) on project MyApp: org.flywaydb.core.api.FlywayException: No database found to handle jdbc:mysql://localhost:3306/ -> [Help 1]

To resolve this we have added dependency flyway-mysql in plugin.

Plugin configuration : pom.xml with driver specific dependencies quarkus-jdbc-postgresql, flyway-sqlserver, flyway-mysql, flyway-database-oracle, flyway-database-postgresql, flyway-database-db2, flyway-database-derby, flyway-database-hsqldb

<plugin>
    <groupId>org.flywaydb</groupId>
    <artifactId>flyway-maven-plugin</artifactId>
    <version>${flywaydb.flyway.maven}</version>
    <dependencies>
        <!-- Flyway -->
        <dependency>
            <groupId>org.flywaydb</groupId>
            <artifactId>flyway-core</artifactId>
            <version>${flywaydb.flyway.maven}</version>
        </dependency>
        <!-- JDBC driver MariaDB/MySQL specific dependencies -->
        <dependency>
            <groupId>org.flywaydb</groupId>
            <artifactId>flyway-mysql</artifactId>
            <version>${flywaydb.flyway.maven}</version>
        </dependency>
        <dependency>
            <groupId>com.mysql</groupId>
            <artifactId>mysql-connector-j</artifactId>
            <version>${mysql.connector.j}</version>
        </dependency>
    </dependencies>
</plugin>
⚠️ **GitHub.com Fallback** ⚠️