Releases and Roadmap - linq2db/linq2db Wiki


Release 4.0.0 Previews 2-7

No new v4-specific features, only changes from underlying v3 releases:


Release 4.0.0 Preview 1

With this release we start to publish previews of Linq To DB v4 (planned for release later this year). Previews will include all features and fixes from current v3 release and new features, fixes and refactorings for v4. This will allow users to use new features that require major version bump due to breaking changes without waiting for next major release.

Based on: v3.4.0. For migration notes check this document

Interceptors

With this release we are starting migration from events to interceptors and introduce first interceptor events. For initial release we concentrating on migration of already existed functionality (e.g. events) to interceptors. New events without prior implementation will be added later or on request.

To see which APIs were replaced with interceptors check migration notes

ICommandInterceptor

This interceptor provides access to events and operations associated with database command.

// triggered after command initialization but before execution
// it provides access to prepared SQL command and parameters
DbCommand CommandInitialized(CommandEventData eventData, DbCommand command);

// triggered before `ExecuteScalar/ExecuteScalarAsync` call on command
// and could replace actual call by returning results from interceptor
Option<object?>       ExecuteScalar     (
                                         CommandEventData eventData,
                                         DbCommand command,
                                         Option<object?> result);
Task<Option<object?>> ExecuteScalarAsync(
                                         CommandEventData eventData,
                                         DbCommand command,
                                         Option<object?> result,
                                         CancellationToken cancellationToken);

// triggered before `ExecuteNonQuery/ExecuteNonQueryAsync` call on command
// and could replace actual call by returning results from interceptor
Option<int>       ExecuteNonQuery     (CommandEventData eventData, DbCommand command, Option<int> result);
Task<Option<int>> ExecuteNonQueryAsync(
                                       CommandEventData eventData,
                                       DbCommand command,
                                       Option<int> result,
                                       CancellationToken cancellationToken);

// triggered before `ExecuteReader/ExecuteReaderAsync` call on command
// and could replace actual call by returning results from interceptor
Option<DbDataReader>       ExecuteReader     (
                                              CommandEventData eventData,
                                              DbCommand command,
                                              CommandBehavior commandBehavior,
                                              Option<DbDataReader> result);
Task<Option<DbDataReader>> ExecuteReaderAsync(
                                              CommandEventData eventData,
                                              DbCommand command,
                                              CommandBehavior commandBehavior,
                                              Option<DbDataReader> result,
                                              CancellationToken cancellationToken);

struct CommandEventData
{
    public DataConnection DataConnection { get; }
}

// convinience base class for custom interceptor implementation
public abstract class CommandInterceptor : ICommandInterceptor
{
    // interceptor implementation as no-op virtual methods
}
IDataContextInterceptor

This interceptor provides access to events and operations associated with database context (built-in class that implements IDataContext, e.g. DataConnection or DataContext).

// triggered when new entity created during query materialization
// (except queries with explicit constructor call)
object EntityCreated(DataContextEventData eventData, object entity);

// triggered before data context instance `Close/CloseAsync` method execution
void OnClosing(DataContextEventData eventData);
Task OnClosingAsync(DataContextEventData eventData);

// triggered after data context instance `Close/CloseAsync` method execution
void OnClosed(DataContextEventData eventData);
Task OnClosedAsync(DataContextEventData eventData);

struct DataContextEventData
{
    public IDataContext Context { get; }
}

// convinience base class for custom interceptor implementation
public abstract class DataContextInterceptor : IDataContextInterceptor
{
    // interceptor implementation as no-op virtual methods
}
IConnectionInterceptor

This interceptor provides access to events and operations associated with database connection.

// triggered before data connection `Open/OpenAsync` method execution
void ConnectionOpening(ConnectionOpeningEventData eventData, DbConnection connection);
Task ConnectionOpeningAsync(
                            ConnectionOpeningEventData eventData,
                            DbConnection connection,
                            CancellationToken cancellationToken);

// triggered after data connection `Open/OpenAsync` method execution
void ConnectionOpened(ConnectionOpenedEventData eventData, DbConnection connection);
Task ConnectionOpenedAsync(
                           ConnectionOpenedEventData eventData,
                           DbConnection connection,
                           CancellationToken cancellationToken);

struct ConnectionOpenedEventData
{
    public DataConnection DataConnection { get; }
}

// convinience base class for custom interceptor implementation
public abstract class ConnectionInterceptor : IConnectionInterceptor
{
    // interceptor implementation as no-op virtual methods
}
Configuration

To register interceptor you can use:

// registration in DataContext
using (var ctx = new DataContext(...))
{
    ctx.AddInterceptor(interceptor);

    // one-time command prepared interceptor
    ctx.OnNextCommandInitialized((args, cmd) =>
    {
        // save next command parameters to external variable
        parameters = cmd.Parameters.Cast<DbParameter>().ToArray();
	return cmd;
    });
}

// registration in DataConnection
using (var ctx = new DataConnection(...))
{
    ctx.AddInterceptor(interceptor);

    // one-time command prepared interceptor
    ctx.OnNextCommandInitialized((args, cmd) =>
    {
        // set oracle-specific command option for next command
        ((OracleCommand)command).BindByName = false;
    });
}

// registration in DataConnection using fluent configuration
var builder = new LinqToDbConnectionOptionsBuilder()
    .UseSqlServer(connectionString)
    .WithInterceptor(interceptor);
var dc = new DataConnection(builder.Build());

Release 3.5.0


Release 3.4.5


Release 3.4.4


Release 3.4.3


Release 3.4.2


Release 3.4.1


Release 3.4.0

Query Tags

Query tag is a comment, attached before query for any reason, but usually to trace source of specific query in logs/execution plans. To attach tag to query, call TagQuery(commentText) method on IQueryable<T>/ITable<T> instance in any place. All comments, attached to query will be aggregated into single comment.

var query = from x in db.Person.TagQuery("first tag").TagQuery("second tag") select x;
query.ToList();
/* first tag
second tag */
SELECT
	x."FirstName",
	x."PersonID",
	x."LastName",
	x."MiddleName",
	x."Gender"
FROM
	"Person" x
Limitations

INSERT ALL/FIRST support

New API added to provide access to INSERT ALL/FIRST multi-table inserts, supported by Oracle.

To define such query over table or subquery, you should apply MultiInsert method to them, define insert operations using Into (unconditional insert) or When/Else (conditional insert) method and execute it with Insert/InsertAll/InsertFirst or their async versions.

// INSERT ALL (unconditional)
await source // query or table
    .MultiInsert()
        .Into(
            db.GetTable<Table1>(),
            x => new Table1 { ID = x.ID + 1, Value = x.N })
        .Into(
            db.GetTable<Table2>(),
            x => new Table2 { ID = x.ID + 3, Int = x.ID + 1 })
        .Into(
            db.GetTable<Table3>(),
            x => new Table3 { ID = x.ID + 3, Int = x.ID + 1 })
    // execute
    .InsertAllAsync();

// INSERT ALL (conditional)
await source // query or table
    .MultiInsert()
        .When(
            src => src.Field1 > 10,
            db.GetTable<Table1>(),
            x => new Table1 { ID = x.ID + 1, Value = x.N })
        .When(
            src => src.Field1 < 5,
            db.GetTable<Table2>(),
            x => new Table2 { ID = x.ID + 3, Int = x.ID + 1 })
        // optional Else
        .Else( // for other records (Field1 in [5; 10] range)
            db.GetTable<Table3>(),
            x => new Table3 { ID = x.ID + 3, Int = x.ID + 1 })
    // execute
    .InsertAllAsync();

// INSERT FIRST
source // query or table
    .MultiInsert()
        .When(
            src => src.Field1 > 10,
            db.GetTable<Table1>(),
            x => new Table1 { ID = x.ID + 1, Value = x.N })
        .When(
            src => src.Field1 < 5,
            db.GetTable<Table2>(),
            x => new Table2 { ID = x.ID + 3, Int = x.ID + 1 })
        // optional Else
        .Else( // for other records (Field1 in [5; 10] range)
            db.GetTable<Table3>(),
            x => new Table3 { ID = x.ID + 3, Int = x.ID + 1 })
    // execute
    .InsertFirst();

Release 3.3.0


Release 3.2.3


Release 3.2.2


Release 3.2.1


Release 3.2.0

Table Options and Temporary Tables

Linq2db has CreateTempTable API that actually creates regular table, which is dropped on table object disposal.

This feature adds support for real temporary tables into linq2db for databases that support temporary tables. Additionally it adds support for TABLE EXISTS checks for CREATE/DROP TABLE statements (APIs).

With this feature we introduce table flags/options, which could be used to mark table as temporary table. This will allow linq2db to generate proper SQL and table name for such tables and corresponding create/drop table SQL (TableOptions.cs):

enum TableOptions
{
    // options not set
    NotSet,
    // default options for regular table
    None,

    // these flags require database support, see list below

    // generates table existense check for CREATE TABLE statement
    CreateIfNotExists,
    // generates table existense check for DROP TABLE statement
    DropIfExists,

    // temporary table flags:
    // temporary table and table content (data) has visibility flags
    // - local: table/data visible only from session that created them (current session)
    // - global: table/data visible to all sessions
    // - transaction: (for data only) data is visible only from current transaction 
    // LinqToDB will select most suitable temp table based on specified flags when database support
    // more than one kind of temp tables, or throw exception when incompatible flags specified

    // local (not visible to other sessions) temporary table
    IsTemporary,

    // table visibility flags
    IsLocalTemporaryStructure,
    IsGlobalTemporaryStructure,

    // data visibility flags
    IsLocalTemporaryData,
    IsGlobalTemporaryData,
    IsTransactionTemporaryData,

    // temporary globally-scoped table:
    // table is visible to all sessions, data visibility depends on database
    IsGlobalTemporary,

}

Changes to existing API surface:

Table existence check support:

Database Drop Table Create Table
Access
DB2
Firebird
Informix
MySQL/MariaDB
Oracle
PostgreSQL
SAP HANA
SQL CE
SQLite
SQL Server
SAP/Sybase ASE

Temporary table support:

Database Local Structure Global Structure Local Data Global Data TransactionData
Access
DB2
Firebird
Informix
MySQL/MariaDB
Oracle
PostgreSQL
SQLite
SAP HANA
SQL CE
SQL Server
SAP/Sybase ASE

Oracle Escaping Changes

Prior to this release, we didn't escaped database identifiers (e.g. table or column name) if they contained lower-cased letters by default. User was able to enable this escaping manually by setting

OracleTools.DontEscapeLowercaseIdentifiers = false;

With this release we switch default value for this option to false, so lowercase identifiers will be escaped by default (as they should).

What code will be broken with this change:

How to fix:

Note: code, generated by T4 templates is not affected as it use names from database

Example:

MYTABLE
(
    ID NUMBER,
    COLUMN NUMBER
)
// table name not specified and defaults to class name "MyTable"
[Table]
class MyTable
{
    // column name not specified and defaults to property name "Id"
    [Column]
    public int Id { get; set; }

    // column name specified but not correct by case
    [Column("CoLuMN")]
    public int MyColumn { get; set; }
}

// correct table name specified
[Table("MYTABLE")]
class MyTable
{
    // property name has same casing as column in db
    [Column]
    public int ID { get; set; }

    // correct column name specified
    [Column("COLUMN")]
    public int MyColumn { get; set; }
}

SequentialAccess support

#2661 adds support for CommandBehavior.SequentialAccess behavior support in query results mapping. Fixes #1185, #2116.

Right now mapping of row to .NET object could read row columns from provider multiple times and in arbitrary order. It works fine with CommandBehavior.Default, which loads whole data row from server into memory, but doesn't work with CommandBehavior.SequentialAccess, which requires consumer to read each column once (or zero) and in order.

With this release we add new option LinqToDB.Common.Configuration.OptimizeForSequentialAccess (false by default), which will enable mapping optimization to read data row columns only once and in proper order to make our mapping compatible with SequentialAccess. In later releases we plan to enable this optimization permanently and remove option.

Note that:

  1. option doesn't enable SequentialAccess behavior for queries. You need to do it itself, e.g. by using custom command processor. See example below
  2. (1) means that option could be enabled even if you don't plan to use SequentialAccess behavior to generate a bit more optimal mapping, but we wouldn't expect any noticable gain from it
  3. linq2db fallbacks to slow mapping mode if data row mapping fails. This will not work with SequentialAccess as it doesn't allow data row re-read. Not really an issue, as valid case of mapping failure occurs only for quite strange queries and indication that there is something wrong with them
  4. if data provider doesn't support SequentialAccess behavior, behavior change will have no effect on it. See list of providers that are known to support SequentialAccess below

Example of custom command processor to enable SequentialAccess behavior:

// register custom command processsor
DbCommandProcessorExtensions.Instance = new SequentialAccessCommandProcessor();

// custom command processor to enable SequentialAccess query behavior
public class SequentialAccessCommandProcessor : IDbCommandProcessor
{
  DbDataReader IDbCommandProcessor.ExecuteReader(DbCommand command, CommandBehavior commandBehavior)
  {
    // override only Default behavior, we don't want to break schema queries
    return command.ExecuteReader(
      commandBehavior == CommandBehavior.Default
        ? CommandBehavior.SequentialAccess
        : commandBehavior);
  }

  Task<DbDataReader> IDbCommandProcessor.ExecuteReaderAsync(
    DbCommand command,
    CommandBehavior commandBehavior,
    CancellationToken cancellationToken)
  {
    return command.ExecuteReaderAsync(
      commandBehavior == CommandBehavior.Default
        ? CommandBehavior.SequentialAccess
        : commandBehavior,
      cancellationToken);
  }

  int IDbCommandProcessor.ExecuteNonQuery(DbCommand command) => command.ExecuteNonQuery();
  Task<int> IDbCommandProcessor.ExecuteNonQueryAsync(DbCommand command, CancellationToken cancellationToken)
    => command.ExecuteNonQueryAsync(cancellationToken);
  object? IDbCommandProcessor.ExecuteScalar(DbCommand command) => command.ExecuteScalar();
  Task<object?> IDbCommandProcessor.ExecuteScalarAsync(DbCommand command, CancellationToken cancellationToken)
    => command.ExecuteScalarAsync(cancellationToken);
}

List of providers that actually support SequentialAccess:


Release 3.1.6


Release 3.1.5


Release 3.1.4


Release 3.1.3


Release 3.1.2


Release 3.1.1


Release 3.1.0

Query parameters improvements

PR #2347 fixes several issues related to parameters:

New APIs

Async CreateTempTable API

Feature #2408 adds async overloads to CreateTempTable API:

Task<TempTable<T>> CreateTempTableAsync<T>(
    this IDataContext db,
    string? tableName    = null,
    string? databaseName = null,
    string? schemaName   = null,
    string? serverName   = null);
Task<TempTable<T>> CreateTempTableAsync<T>(
    this IDataContext db,
    IEnumerable<T> items,
    BulkCopyOptions? options = null,
    string? tableName        = null,
    string? databaseName     = null,
    string? schemaName       = null,
    string? serverName       = null);
Task<TempTable<T>> CreateTempTableAsync<T>(
    this IDataContext db,
    string? tableName,
    IEnumerable<T> items,
    BulkCopyOptions? options = null,
    string? databaseName     = null,
    string? schemaName       = null,
    string? serverName       = null);
Task<TempTable<T>> CreateTempTableAsync<T>(
    this IDataContext db,
    IQueryable<T> items,
    string? tableName             = null,
    string? databaseName          = null,
    string? schemaName            = null,
    Func<ITable<T>, Task>? action = null,
    string? serverName            = null);
Task<TempTable<T>> CreateTempTableAsync<T>(
    this IDataContext db,
    IQueryable<T> items,
    Action<EntityMappingBuilder<T>> setTable,
    string? tableName             = null,
    string? databaseName          = null,
    string? schemaName            = null,
    Func<ITable<T>, Task>? action = null,
    string? serverName            = null);
Task<TempTable<T>> CreateTempTableAsync<T>(
    this IDataContext db,
    string? tableName,
    IQueryable<T> items,
    string? databaseName          = null,
    string? schemaName            = null,
    Func<ITable<T>, Task>? action = null,
    string? serverName            = null);
Task<TempTable<T>> CreateTempTableAsync<T>(
    this IDataContext db,
    string? tableName,
    IQueryable<T> items,
    Action<EntityMappingBuilder<T>> setTable,
    string? databaseName          = null,
    string? schemaName            = null,
    Func<ITable<T>, Task>? action = null,
    string? serverName            = null);
Async BulkCopy API

Feature #2314 adds async overloads to BulkCopy API:

Task<BulkCopyRowsCopied> BulkCopyAsync<T>(
    ITable<T> table,
    BulkCopyOptions options,
    IEnumerable<T> source);
Task<BulkCopyRowsCopied> BulkCopyAsync<T>(
    ITable<T> table,
    BulkCopyOptions options,
    IAsyncEnumerable<T> source);

Note that as usual it requires support from underlying provider, as if it doesn't support required async APIs, execution will be done in synchronous mode. For native bulk copy following providers provide async API:

For 3 other types of bulk copy provider should support ExecuteNonQueryAsync API on command.

Also bulk copy methods, exposed by <DB_NAME>Tools classes were obsoleted, as they doesn't add anything new and just call BulkCopy API internally.

New QueryProc overloads
// new QueryProc overloads to support results of anonymous type
IEnumerable<T> QueryProc<T>(
    this DataConnection connection,
    T template,
    string sql,
    params DataParameter[] parameters);
IEnumerable<T> QueryProc<T>(
    this DataConnection connection,
    T template,
    string sql,
    object? parameters);
SkipOnEntityFetch column mapping flag

Feature #2387 adds new column mapping flag SkipOnEntityFetch to ignore column on entity select queries without explicit column column list specified, e.g. db.Table.ToList().

This flag could be useful if you have columns, you want to select only explicitly, e.g. big blob-like columns.

Flag could be set:


Release 3.0.1