Switching to PostgreSQL from SQLite3 - MagnusBook/flasktutorial GitHub Wiki

This page will give you the steps you need to take for making your application use PostgreSQL, which is a production database system. This will show you how to set it up in Heroku.

Heroku setup

In order to allow the application to use PostgreSQL, we need to add it as an add-on for our Heroku application. You can do this through the website, or more simply through the use of the following command:

heroku addons:create heroku-postgresql:hobby-dev

This will create a free instance of PostgreSQL that you can use for your project. It will define the URL to access the server as an environment variable called DATABASE_URL, which we will use to connect to our database, both for SQLite and SQLAlchemy.

SQLAlchemy conversion

Converting your SQLAlchemy to a PostgreSQL application is quite simple. The only thing you need to change is the URL for the database, SQLAlchemy will handle the rest automatically.

Normally when using SQLite3, you would initialize your database in a way similar to this:

from flask import Flask
from flask_sqlalchemy import SQLAlchemy

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)

In order to use the PostgreSQL database on Heroku, we simply change the SQLAlchemy configuration to get this value from the operating system like this:

from flask import Flask
from flask_sqlalchemy import SQLAlchemy
import os

app = Flask(__name__)
uri = os.getenv("DATABASE_URL")  # or other relevant config var
if uri.startswith("postgres://"): # from SQLAlchemy 1.14, the uri must start with postgresql, not postgres, which heroku provides
    uri = uri.replace("postgres://", "postgresql://", 1)
app.config['SQLALCHEMY_DATABASE_URI'] = uri
db = SQLAlchemy(app)

After making these changes, it should use the value for the configured DATABASE_URL, which in our case will be set by Heroku.

SQLite3 conversion

Converting SQLite3 to use PostgreSQL is a bit more challenging compared to SQLAlchemy. Essentially we need to replace the SQLite3 library in the code with a library called psycopg2.

Installing PostgreSQL

First install the library. To do this, you need to install PostgreSQL on your machine, see this link for instructions. After that, we can install the library using:

pip install psycopg2

Also remember to add this library to your requirements.txt

Changing the code

Using the flasktutorial project as an example, we will see how you can make the changes to use the newly installed psycopg2 library. As a refresher, the part of this application where most of the database logic is defined is in __init__.py. Here is the old version of the file:

from flask import Flask, g
import sqlite3
app = Flask(__name__)

DATABASE = 'database.db'


def get_db():
    db = getattr(g, '_database', None)
    if db is None:
        db = g._database = sqlite3.connect(DATABASE)
    db.row_factory = sqlite3.Row
    return db


@app.teardown_appcontext
def close_connection(exception):
    db = getattr(g, '_database', None)
    if db is not None:
        db.close()


def init_db():
    with app.app_context():
        db = get_db()
        with app.open_resource('schema.sql', mode='r') as f:
            db.cursor().executescript(f.read())
        db.commit()


def query_db(query, args=(), one=False):
    cur = get_db().execute(query, args)
    rv = cur.fetchall()
    cur.close()
    return (rv[0] if rv else None) if one else rv


import flasktutorial.views

First, we will change the imports and the DATABASE variable like so:


from flask import Flask, g
import psycopg2
import psycopg2.extras
import os

app = Flask(__name__)

DATABASE = os.environ['DATABASE_URL']

This imports the psycopg2 library, along with the extras from that same library, which we will use later. We also set the DATABASE variable to be the value of the DATABASE_URL, which will be provided by Heroku.

Next, for the get_db function, we need to replace SQLite with the new module:

def get_db():
    db = getattr(g, '_database', None)
    if db is None:
        db = g._database = psycopg2.connect(DATABASE)
    return db

It should be noted that we removed the row factory specification from this function. That part works a little differently for psycopg2. Other than that, sqlite3 has simply been replaced with psycopg2.

The tear-down function close_connection is exactly the same as before.

Our function init_db creates the application for the first run by running the SQL statements in the schema.sql file. Here the only change is that we use db.cursor().execute instead of db.cursor.executescript

def init_db():
    with app.app_context():
        db = get_db()
        with app.open_resource('schema.sql', mode='r') as f:
            db.cursor().execute(f.read())
        db.commit()

For the query_db function, the only difference is that we explicitly get the cursor object before we can execute the query. Also note that we set cursor_factory to be a DictCursor. This allows us to access the rows like dictionaries, like we did with SQLite3's row factory previously.

def query_db(query, args=(), one=False):
    cur = get_db().cursor(cursor_factory=psycopg2.extras.DictCursor)
    cur.execute(query, args)
    rv = cur.fetchall()
    cur.close()
    return (rv[0] if rv else None) if one else rv

Query syntax

The query syntax is very similar to SQLite3, but slightly different. We will make one change in the views.py file to illustrate this.

The following is what we used in SQLite3:

article = query_db('SELECT * FROM article WHERE id=?;', (str(article_id)), True)

And this is what we use for PostgreSQL:

article = query_db('SELECT * FROM article WHERE id=%s;', (article_id,), True)

Here we change the ? sign in the SQL query to %s. We also simplify things and simply pass in a tuple containing the artcle_id.


Remember that the full code is available from the GitHub repository. Also let me know if you experience any issues regarding this, just let me know, and I will try to help you and update this document accordingly!