Negative identities and object graph saving - lobodava/artisan-orm GitHub Wiki

Object graph from web client

Let's suppose that a web client sends to our web server a graph consisting of three objects:

GrandRecord
Record
ChildRecord

Let it be a collection of GrandRecords where:

each GrandRecord has a collection of Records,
and each Record has a collection of ChildRecords,
Id value is integer and autogenerated by Database,
and while an object is not saved in Database, Id has value = 0.

Here is the sample of object graph array (or object graphs) :

                      Id, Name
GrandRecord           1,  (A)
    Record            |-- 2, (A)A
        ChildRecord       |-- 3, (A)Aa
        ChildRecord       |-- 0, (A)Ab
    Record            |-- 0, (A)B
        ChildRecord       |-- 0, (A)Ba
        ChildRecord       |-- 0, (A)Bb
GrandRecord           0,  (B)
    Record            |-- 0, (B)A

Or the same in JSON format:

grandRecords: [
    {
        id: 1,
        name: "(A)",
        records: [
            {
                id: 2,
                name: "(A)A",
                childRecords: [
                    {
                        id: 3,
                        name: "(A)Aa",
                    },
                    {
                        id: 0,
                        name: "(A)b",
                    },
                ]
            },
            {
                id: 0,
                name: "(A)B",
                childRecords: [
                    {
                        id: 0,
                        name: "(A)Ba",
                    },
                    {
                        id: 0,
                        name: "(A)Bb",
                    },
                ]
            }        
        ]        
    },
    {
        id: 0,
        name: "(B)",
        records: [
            {
                id: 0,
                name: "(B)A",
                childRecords: []
            }
        ]
    }
]

C# object graph classes

On the web server in our ASP.NET application the above JSON string is deserialized into object graphs of three classes:

public class GrandRecord
{
    public  Int32          Id       { get; set; }
    public  String         Name     { get; set; }
    public  IList<Record>  Records  { get; set; }
}

public class Record
{
    public  Int32               Id             { get; set; }
    public  Int32               GrandRecordId  { get; set; }        
    public  String              Name           { get; set; }
    public  IList<ChildRecord>  ChildRecords   { get; set; }
}

public class ChildRecord
{
    public  Int32   Id        { get; set; }
    public  Int32   RecordId  { get; set; }
    public  String  Name      { get; set; }
}

Note that Record.GrandRecordId and ChildRecord.RecordId after deserialization are 0.

Database tables

The object graphs has to be saved with one stored procedure into three database tables:

create table dbo.GrandRecords
(
    Id    int          not null  identity  primary key clustered,
    Name  varchar(30)  not null
);

create table dbo.Records
(
    Id             int          not null  identity  primary key clustered,
    GrandRecordId  int          not null  ,
    Name           varchar(30)  not null  ,
    
    foreign key (GrandRecordId) references dbo.GrandRecords (Id) on delete cascade
);

create table dbo.ChildRecords
(
    Id        int          not null  identity  primary key clustered,
    RecordId  int          not null  ,
    Name      varchar(30)  not null  ,
    
    foreign key (RecordId) references dbo.Records (Id) on delete cascade
);

User-defined table types as parameters

Artisan.Orm uses the User Defined Table Types to save objects to database. In our case, UDTTs look like this:

create type dbo.GrandRecordTableType as table
(
    Id    int          not null   primary key clustered,
    Name  varchar(30)  not null    
);

create type dbo.RecordTableType as table
(
    Id             int          not null   primary key clustered,
    GrandRecordId  int          not null   ,
    Name           varchar(30)  not null

);

create type dbo.ChildRecordTableType as table
(
    Id        int          not null   primary key clustered,
    RecordId  int          not null   ,
    Name      varchar(30)  not null    
);

Thus the stored procedure for saving the above object graphs starts with:

create procedure dbo.SaveGrandRecords
    @GrandRecords  dbo.GrandRecordTableType  readonly,
    @Records       dbo.RecordTableType       readonly,
    @ChildRecords  dbo.ChildRecordTableType  readonly
as

Repository Save method

The repository Save method is:

public void SaveGrandRecords(IList<GrandRecord> grandRecords)
{
    var records = grandRecords.SelectMany(gr => gr.Records);
    var childRecords = records.SelectMany(r => r.ChildRecords);

    ExecuteCommand(cmd =>
    {
        cmd.UseProcedure("dbo.SaveGrandRecords");

        cmd.AddTableParam( "@GrandRecords", grandRecords );
        cmd.AddTableParam( "@Records"     , records      );
        cmd.AddTableParam( "@ChildRecords", childRecords );
    });
}

Negative Identities

Because our tables in Database are linked by foreign keys GrandRecordId and RecordId, we have somehow to persist that link while converting the object graphs into the separate table paremeters.

What is more, the identities of new objects have to be unique! Otherwise, we can not distinguish the Records of GrandRecord (A) from the Records of GrandRecord (B).

Here Negative Identities come up on stage. Artisan.Orm provides three static classes with only Next property:

Int64NegativeIdentity
Int32NegativeIdentity
Int16NegativeIdentity

The Next property returns a new sequent negative number on every get call starting with [type].MinValue.

So the Negative Identities provide ever increasing Ids for new objects in the graph.

CreateDataRow Mapper methods

Artisan.Orm cmd.AddTableParam extensions use Mappers and those Mappers do the magic with the Negative Identities in CreateDataRow methods:

// GrandRecordMapper

public static Object[] CreateDataRow(GrandRecord obj)
{
    if (obj.Id == 0) 
        obj.Id = Int32NegativeIdentity.Next;

    foreach (var record in obj.Records)
        record.GrandRecordId = obj.Id;

    return new object[]
    {
        obj.Id    ,
        obj.Name
    };
}

// RecordMapper

public static Object[] CreateDataRow(Record obj)
{
    if (obj.Id == 0) 
        obj.Id = Int32NegativeIdentity.Next;

    foreach (var childRecord in obj.ChildRecords)
        childRecord.RecordId = obj.Id;

    return new object[]
    {
        obj.Id             ,
        obj.GrandRecordId  ,
        obj.Name
    };
}

// ChildRecordMapper

public static Object[] CreateDataRow(ChildRecord obj)
{
    if (obj.Id == 0) 
        obj.Id = Int32NegativeIdentity.Next;

    return new object[]
    {
        obj.Id        ,
        obj.RecordId  ,
        obj.Name
    };
}

Data in table valued parameters

So after the Mappers do their work, the stored procedure receives the following table valued parameters:

@GrandRecords

Id	Name
1	(A)
-2147483648	(B)

@Records

Id	GrandRecordId	Name
2	1	(A)A
-2147483647	1	(A)B
-2147483646	-2147483648	(B)A

@ChildRecords

Id	RecordId	Name
3	2	(A)Aa
-2147483645	2	(A)Ab
-2147483644	-2147483647	(A)Ba
-2147483643	-2147483647	(A)Bb

Object graphs saving technique

In order to update existing, insert new and delete old data, SQL Server uses the MERGE statement.

The MERGE statement has the OUTPUT clause. The OUTPUT in MERGE statement can collect just inserted Ids as well as Ids from the source (parameter) table.

If a record exists in a table inserted.id contains the existing Id and MERGE does UPDATE.

If a record does not exist in a table inserted.id contains the new Id and MERGE does INSERT.

So the technique "to save all three tables with correct foreign keys" is to collect InsertedId-ParamId pairs from the first table and translate those values for the second. And do the same for the second and third tables.

Here is the stored procedure which saves our object graphs:

create procedure dbo.SaveGrandRecords
    @GrandRecords  dbo.GrandRecordTableType  readonly,
    @Records       dbo.RecordTableType       readonly,
    @ChildRecords  dbo.ChildRecordTableType  readonly
as
begin
    set nocount on;

    declare @GrandRecordIds table (  -- translation table
        InsertedId  int  primary key, 
        ParamId     int  unique
    );
    
    declare @RecordIds table (       -- translation table
        InsertedId  int     primary key, 
        ParamId     int     unique, 
        [Action]    nvarchar(10)
    );

    -- save GrandRecords 

    merge into dbo.GrandRecords as target
        using 
        (
            select Id, Name from @GrandRecords
        ) 
        as source on source.Id = target.Id

    when matched then
        update set                
            Name = source.Name        

    when not matched by target then                                                         
        insert ( Name )
        values ( source.Name )

    output            -- collecting translation ids
        inserted.Id,
        source.Id
    into @GrandRecordIds ( 
        InsertedId , 
        ParamId    );


    -- save Records 

    merge into dbo.Records as target
    using 
    (
        select
            Id             , 
            GrandRecordId  =  ids.InsertedId,   -- Id translation target
            Name    
        from
            @Records r
            inner join @GrandRecordIds ids 
                on ids.ParamId = r.GrandRecordId -- Id translation source
    ) 
    as source on source.Id = target.Id

    when matched then
        update set
            GrandRecordId  =  source.GrandRecordId, 
            Name           =  source.Name    

    when not matched by target then                                                         
        insert (    
            GrandRecordId , 
            Name          )
        values (
            source.GrandRecordId , 
            source.Name          )

    when not matched by source 
        and target.GrandRecordId in (select InsertedId from @GrandRecordIds) then
           delete

    output                 -- collecting translation Ids
        isnull(inserted.Id, deleted.Id),
        isnull(source.Id, deleted.Id), 
        $action
    into @RecordIds (
        InsertedId  , 
        ParamId     , 
        [Action]    );

            
    delete from @RecordIds where [Action] = 'DELETE';


    -- save ChildRecords

    merge into dbo.ChildRecords as target
        using 
        (
            select
                Id        ,
                RecordId  =  ids.InsertedId,    -- Id translation target
                Name        
            from
                @ChildRecords cr
                inner join @RecordIds ids 
                    on ids.ParamId = cr.RecordId -- Id translation source
        ) 
        as source on source.Id = target.Id

    when matched then
        update set
            RecordId = source.RecordId , 
            Name     = source.Name

    when not matched by target then
        insert (    
            RecordId , 
            Name     )
        values (
            source.RecordId , 
            source.Name     )
        
    when not matched by source and target.RecordId in (select InsertedId from @RecordIds) then
        delete;
end;

Important notice

In the MERGE statement the source and the target tables 
  MUST have clustered indexes on their join columns!
This prevents deadlocks and guarantees insertion order.

The join columns are in as source on source.Id = target.Id line of MERGE statement.

That is why the User Defined Table Types have primary key clustered in their definitions.

That is why the Negative Identities are ever increasing and start with MinValue.

Note also the definition of foreign keys in the permanent tables. They contain on delete cascade clause, which helps to delete child records when deleting parent records in MERGE statement.