The qbo4.Data.Parquet plugin has been extended to support serializing IDataReader.GetSchemaTable to the ParquetWriter.CustomMetadata property. This enables downstream code to recreated the native RDBMS (SQL) types, avoiding (among other things) the loss of nvarchar precision (nvarchar(50) => System.String => nvarchar(max).

If the downstream consumer of the Parquet file does not handle the CustomMetadata appropriately, this feature can be disabled by setting the ParquetWriterOptions.IncludeColumnMetadata to false.

The IConfigurationBuilder.AddConfigurtionMethods will now ignore configuration methods that:

• have a Section defined, and
• do not have the Section present in appsettings.config

This allows a developers to control whether to load configuration by bootstrapping section names in appsettings.json.

If you wish to consitionally execute a ConfigurationMethod, specify a Section and include the section in appsettings.json:

[ConfigurationMethod(Section="OnlyLoadIfInAppsettings")]
public static IConfigurationBuilder AddOnlyIfInAppsettings(IConfigurationBuilder builder)
{ ... }

To call AddOnlyIfInAppsettings, include it's Section in appsettings.json:

{
    "OnlyLoadIfInAppsettings": true
}

The value of the setting can be anything, including child nodes:

{
    "OnlyLoadIfInAppsettings": {
        "Thing1": "Foo",
        "Thing2": "Bar"
    }
}

The value can even be false (which is non-intuitive):

{
    "OnlyLoadIfInAppsettings": false
}

To ignore AddOnlyIfInAppsettings, omit it's Section from appsettings.json:

{
}

SQS Queue Plug-In now creates FIFO queues by default. When updating to the latest plug-in there is risk of data loss if SQS queues contains items during the update. To locate and migrate previous queue items:

Queue/SQSMigrateNonFifo (migrates all items for all SQS queues - suggested if queue counts < 1000)
Queue/SQSMigrateNonFifo?Queue={QueueName} (filters on a queue)

More information can be found in the qbo3.Amazon README

Search Terms for {Module} and {Module}ID have been moved from the embedded config files to setup packages. This will require running all setup packages that end with "Search Term" to populate these missing search terms.

The qbo.Excel.Extension.js has been updated in qbo3.OpenXml to use a custom route handled by qbo4.Document.Bridge.AsposeCells.CellsHandler. This enables creating custom Excel overrides to method signature output. However, the setup package is an embedded resource, so it may need to be installed manually. If this is not installed, the Options > Export to Excel will raise a 404 (Not Found) error.

To install:

<?xml version="1.0" encoding="utf-8" ?>
<ConfigurationEntryCollection>
	<!-- CustomRoute -->
	<ConfigurationEntryItem>
		<ConfigurationEntry>CustomRoutes/Excel</ConfigurationEntry>
		<Source>CustomRoutes.config</Source>
		<ConfigurationType>qbo.Application.Configuration.CustomRouteCollection</ConfigurationType>
		<ConfigurationKey>Excel</ConfigurationKey>
		<ConfigurationXml>
			<CustomRoute Name="Excel" Url="excel/{class}/{operation}" Version="1" Handler="qbo4.Document.Aspose.Bridge.CellsHandler, qbo4.Document.Aspose.Bridge, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
		</ConfigurationXml>
	</ConfigurationEntryItem>
</ConfigurationEntryCollection>

qbo HealthCheck Service has been updated to utilize an IPerson provider (LocalUserManager and SecurityMap) that do not require System.Web.SecurityMembership. This requires additional setup in appSettings.json

Setup instructions can be found in the readme.

qbo HealthCheck has been updated to support locking a check when the check is executed. This is an "opt-in" setting which ensures that when multiple application servers are performing the same check, the check is performed only once by the application server that acquires the lock.

HealthCheck has also been refactored to store HealthCheck entries in a dedicated table vs ConfigurationEntry. For existing or custom checks, there is a built in conversion method that will automatically convert existing checks to table storage.

The UI has been refactored to conform to qbo3 dashboard patterns and by default healthchecks are not processed when the UI draws. This has to be explicitly performed by the user.

Information can be found in the readme.

qbo HealthCheck has been updated to allow checks to execute asynchronously and centrally cache the result data. This allows resource intensive HealthChecks to process off thread and source cached results across servers. The asynchronous checks are executed using a background service which is native to the qbo HealthCheck plug-in. Queue Service does not execute the asynchronous checks. The caching of the healthcheck results now supports IDistributedCache. The first implementation of IDistributedCache is SqlServer.

In order to update, please reference the latest NuGets:

• qbo3.HealthCheck
• qbo4.Caching.SqlServer
• qbo4.Caching
• qbo.Fintech.Dacpacs (installs Sql caching store)

Please see respective readme files for configuration.

The qbo4.Configuration.HttpApplication project now directly references qbo4.Configuration.AssemblyBinding, calling AssemblyBinding.Redirect.Start prior to loading configuration and dependency injection. This ensures that assembly binding issues are resolved even if the IIS pipeline invokes HttpApplication.Startup prior to AssemblyBinding.Redirect.Start.

The following qbo3 packages now include a build targeting .netstandard2.0:

• qbo3.Accounting
• qbo3.Application
• qbo3.Attachment
• qbo3.Contact
• qbo3.Decision
• qbo3.Exception
• qbo3.Import
• qbo3.Logging (targets .net48 only - see below)
• qbo3.Message
• qbo3.Process
• qbo3.Report
• qbo3.Score
• qbo3.Security

This allows projects target .netcore, including .net6.0 and .net7.0 pre-release to leverage qbo3 data and API tier code, but not web-tier code.

The .netstandard builds omit functionality based on the System.Web namespace, including the qbo3 HttpAsyncHandler.

If you wish to build netcore-based UI, use qbo4.UI.

Developer note: the qbo3 repo now includes a qbo.Core.Netstandard solution as the host solution for these projects. The "project-specific" solutions (qbo.Application.sln, qbo.Attachment.sln, etc.) are now deprecated, as are their corresponding pipelines. Pipeline builds for these projects are now handled by the qbo3.Core.NetStandard pipeline.

qbo3.Logging note: the underlying Microsoft.Enterprise.Library.Logging v 5.0.5 classes are not compatible with .netstandard. Thus, future .netcore projects should implement ILogger / qbo4.Logging equivalents to the functionality found in qbo3.Logging. This includes a handful of custom references in qbo3.Import to read from the ImportLog table.

Filters and Search Terms moved from config files to setup packages.

• Run packages with a pattern of Setup.*Filter.xml to add all filters
• Run packages with a pattern of Setup.*SearchTerm.xml to add all search terms

Filters for Entity, EntityParent and Extranet remain in the config files and support overwriting. Search Terms for {Table} and {Table}ID remain in the config files and support overwriting.

A sample seed script to load all core packages that have been abstracted can be found here.

Child Classes, Dimensions, Listeners and Services moved from config files to setup packages.

• Run packages with a pattern of Setup.*Child.xml to add all child classes
• Run packages with a pattern of Setup.*Dimension.xml to add all custom dimensions
• Run packages with a pattern of Setup.*Listener.xml to add all data and method listeners
• Run packages with a pattern of Setup.*Service.xml to add all services

The qbo3.Score module has been updated to implement CalculateAsync:

• IScoreEngine now required a CalculateAsync method
• AbstractEngine now implements a variety of CalculateAsync methods, which by default call Calculate

No action is required; this is not a breaking change. However, if you are upgrading an existing IScoreEngine implementation or creating a new one, ensure you implement CalculateAsync:

public override Task CalculateAsync(...) 
{
  // do the work here
}

In another step toward net50 migration, qbo.Application.csproj and qbo.Score.csproj have been updated to use SDK-style project files. This vastly simplifies the project structure, including flagging embedded resources by folder:

<ItemGroup>
  <EmbeddedResource Include="Config\*.config" />
  <EmbeddedResource Include="Templates\Score\*.xslt" />
</ItemGroup>

If you are porting a csproj to a SDK style projects, please review code migration notes.

qbo3 Behaviors now use MooTools version 1.6 which is included in mootools.core.js, mootools.more.js and mootools.behavior.js. The following MooTools 1.4 files must be manually removed: mootools.js, mootools.min.js and mootools.overrides.js.

qbo3.Security has been extended to utilize templates for SecurityAccess. SecurityAccessTemplates have been created for the user functions: Register, ConfirmAccount and PasswordReset.

The following steps must be taken to adopt the latest version:

• Seed SecurityAccessTemplate Setup Data

The qbo3 Amazon Library has been retrofitted to utilize qbo4.Configuration.Bridge and qbo4.Amazon.Security for credentials. This update addresses the current deficiency of developers unable to connect to AWS resources locally and continues qbo3/qbo4 settings evolution. This change impacts ALL AWS plug-ins.

The following steps must be taken to adopt the latest version:

• one click must target net48
• ensure appsettings.json exists at root of one-click (with Build Action: Content) containing json:

{
    "ApplicationStartup": {
        "Options": {
            "Project": "qbo3"
        }
    },
    "qbo3": {
        "ConfigFile": "default",
        "ConfigurationEntry": {
            "ConnectionString": "{qbo3:ConnectionStrings:qbo.Default}"
        }
    },
    "AWSCredentialServiceOptions": true
}

• Ensure IIS Application Pool Setting: Advanced Settings -> Process Model -> Load User Profile is set to True

The AWSCredentialServiceOptions triggers loading the AWSCredentialService on application startup. If this is being run from an AWS container, no further configuration should be necessary. If this is being run from outside AWS, one can set an AccessKey or SecretKey:

"AWSCredentialServiceOptions": {
    "AccessKey": "aaa",
    "SecretKey": "sss"
}

Please refer to the qbo3 Amazon ReadMe for additional details.

S3FileObject has been updated to address an AWS pathing limitation. This results in a breaking change for existing S3 File Objects in use. For new file objects that have not yet saved any files to S3 there are no breaking changes.

The fix for existing file objects is to append the Settings attribute with Settings="S3RootFolder=qboRoot". Eg.

<FileObject Name="AmazonS3" Type="qbo.Attachment.Amazon.S3FileObject, qbo.Attachment.Amazon" Compression="false" Uri="https://site.s3.amazonaws.com/" Settings="S3RootFolder=qboRoot"> ... </FileObject>

This update will append qboRoot to the root path of Amazon Key / PathURL to allow existing pathing to function. For newer installs, qboRoot is excluded from root path.

QBO core has been extended to support DateTimeOffset (DTO) datatypes in app, data and web tiers. Supporting DTO provides additional date precision that allows the application to function in disparate time zones. Note DBMS in newer installs is targeted to UTC. For more information on choosing the correct date types when defining QBO objects, please refer to QBO.wiki -> Defining Data Types.

Each respective application requires conversion to support this update. Please read this entire post and then focus on the Steps To Update section.

• Data Tier conversion to DTO
• Custom statement conversion to DTO (where applicable)
• Web / App Tier conversion to DTO
• Bulk Import Engines

Data Tier conversion to DTO

The Core database solution is equipped to convert QBO database schemas. For most, smaller installs follow the update steps. For large installs, please consult Quandis Professional Services for an optimized database script that will consolidate the conversion and update of DTO columns and reduce conversion time. Conversion times using the standard install process should not exceed 4 hours.

Smaller Databases (less than 100 GB)

qbo.Fintech.Dacpacs now contain objects that support DateTimeOffset. Each respective database schema must be updated to support DTO. The general pattern to update is:

• Use the DB Solution to convert the database schema (tables, columns, triggers) to DTO data type. The conversion script is automatically generated by the DB project. By converting columns to DateTimeOffset SQL Server implicitly sets the time zone for each value to UTC (Eg. 1/1/2020 00:00 becomes 2020-01-01 00:00:00.0000000 +00:00)
• Update EVERY table record value to set correct time zone

Record updates are performed when referencing Core.dacpac's Script.PreDeployment.sql and Script.PostDeployment.sql:

Script.PreDeployment.sql

• All core tables are flagged for conversion if {Table}.CreatedDate was type DateTime and is now DateTimeOffset
• Tables flagged for conversion are logged to a new table dbo.DateTimeOffsetConversion which is used for conversion tracking

Script.PostDeployment.sql

• For each table flagged, a one-time conversion (invoking pTableUpdateDateTimeOffset) will update blocks of rows (chunked) to correct time zone. It autodetects all datetimeoffset columns and will include in update sql. Target timezones can be globally set with SystemDefaults:

• SystemDefault: DateTimeOffset.TimeZone, Value Pacific Standard Time - used with SQL Server 2016 or later

• SystemDefault: DateTimeOffset.TimeZoneOffset, Value -07:00 - used with pre SQL Server 2016

It's preferred for the conversion to occur on SQL Server 2016 or later as AT TIME ZONE will be utilized to set the time zone. This ensures any dates which fall into Daylight Savings Time are converted using the operating system. SQL Server will automatically toggle the time zone based on date value. For pre 2016, date conversion is routed through SQL QBO UDF dbo.SQL2012DTOConvert which checks a lookup table DSTMapV2 to convert the value. DSTMapV2 is automatatically seeded with latest database updates.

The practical risk (although very low) is date values will be converted to a time zone off by one hour causing the day to roll to a previous day.

• Eg. ImportForm.ActualCompletion is set to 12/1/2019 00:00 -07:00 when it should be 12/1/2019 00:00 -08:00 if the application is set to run in PDT. * 12/1/2019 00:00 -07:00 is actually 11/30/2019 23:00 -08:00

Custom statement conversion to DTO

Custom statements may require an update to conform to DTO data types. For example, if there is a custom statement that queries Decision.ActualCompletion based on a date range, the Db parameter types should change from DateTime to DateTimeOffset. ConfigurationEntry/EntryDateTimeDetect can be used as a tool to identify custom statements that contain DateTime patterns. This statement works in conjunction with ConfigurationEntry/EntryDateTimeConvert which converts statements.

Usage:

• Execute /Report/Excel.ashx/ConfigurationEntry/EntryDateTimeDetect- This will identify any statements that contain DateTime patterns.
• Examine each statement and modify any DateTime references OR optionally invoke ConfigurationEntry/EntryDateTimeConvert. The report will provide a link for each convert statement.
• EntryDateTimeConvert will string replace DateTime with DateTimeOffset and create a backup of original statement with a datetime stamp that can be accessed from the application

Statement conversion should be performed in a lower environment and tested. EntryDateTimeDetect will exclude any statements that have already been converted

Web / App Tier conversion to DTO

All statements part of embedded resources in Core objects have been converted to support DTO and are included as part of new build. Custom statements and/or statement overrides need to be examined to ensure they're DTO compatible.

Bulk Import Engines

Bulk Import Engines operate by using SQL Bulk Import to read text or Excel based files into a transient table and then invoking a custom statement to update QBO from the transient table. Text and Excel bulk engines have been updated to import all fields detected as DateTimeOffset. This allows date precision to be captured on import and optionally carried into respective QBO tables. For example:

• Text File may contain birthdate and QBO statement targets updating Contact.BirthDate. transient.birthdate will be type DTO and existing statement will implicitly convert DateTimeOffset to DateTime

• Text File may contain ScheduledSaleDate and QBO statement targets updating ImportForm.ActualCompletion. transient.ScheduledSaleDate will be type DTO and existing statement will correctly ImportForm.ActualCompletion timezone

• SqlBulkEngine related imports may require an update to their ImportFileMap entries. For any mapping that previously targeted a DateTime column which has been converted to DateTimeOffset, set ImportFileMap.DataType = DateTimeOffset. For example, any mapping that targets Process.OpendedDate should have ImportFileMap.DataType = DateTimeOffset.

Assembly Guidelines

• All Core, CoreWeb, Queuing NuGets have been rebuilt with fields that have been converted to DTO (Eg. CreatedDate, UpdatedDate)
• All plug-ins should be rebuilt referencing a version of Core that supports DTO

Please perform the following in a lower environment:

• Update respective one-click NuGets. All Core/Web NuGets, DacPacs and plug-ins should update.
• Backup Entity/Entity Parent in target database by scripting from SSMS
• Generate a DB Script based on new DacPacs. Examine script and look for DateTimeOffsetConversion. This ensures Pre and Post scripts are correctly referenced. In some cases script will not generate due warnings on Changes to [dbo].[Table] might introduce run-time errors in [dbo].[Entity]. At this point you'll need to drop Entity and EntityParent and regenerate
• Execute database script to convert database schema
• Target custom DateTime statement conversion using ConfigurationEntry/EntryDateTimeDetect
• Restore Entity/EntityParent by backup or rebuild

Please note:

• If the production environment is significantly larger than lower environments please perform a dry run database conversion on backup of production.
• NuGet plug-in dates should reflect 10/29/2020 or later. If a NuGet does not update, in most cases it should be rebuilt with latest references and republished.

Previously extranet access was checked using the following logic:

• Not enabled: Created Person OR Parent
• Parent enabled: Created Person OR Parent
• Form enabled: Created Person OR (Template AND Parent)

Extranet access is now checked as follows:

• Not enabled: Template
• Parent enabled: Template AND Parent
• Form enabled: Template AND ImportForm Access

The qbo.ApplicationWeb project has included CodeMirror as part of the QBO source to assist with browser-based editing for code: SQL, XML, XSLT. To keep this code up to date:

• all UI references in the qbo3 repository have been updated to reference CodeMirror via a CDN (content delivery network)
• the CodeMirror folder has been removed from the qbo.ApplicationWeb project
  • the content has not been deleted from GIT, in case we need a ready reference to compare to

To render CodeMirror:

<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/codemirror/5.58.1/codemirror.min.css" integrity="sha512-xIf9AdJauwKIVtrVRZ0i4nHP61Ogx9fSRAkCLecmE2dL/U8ioWpDvFCAy4dcfecN72HHB9+7FfQj3aiO68aaaw==" crossorigin="anonymous" />
<script src="//cdnjs.cloudflare.com/ajax/libs/codemirror/5.58.1/codemirror.min.js" integrity="sha512-WWC1A/JchDFZ2ZGaNyMC7CmPFXGLI/6Ih7WN6YG0DX1NGMkW5lqCVFxCmEx3e56Z7iqdQGpO0f+m2t9CJhdb2Q==" crossorigin="anonymous"></script>

To include extensions like xml helpers, include the extensions similar to:

<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/codemirror/5.58.1/addon/hint/show-hint.min.css" integrity="sha512-OmcLQEy8iGiD7PSm85s06dnR7G7C9C0VqahIPAj/KHk5RpOCmnC6R2ob1oK4/uwYhWa9BF1GC6tzxsC8TIx7Jg==" crossorigin="anonymous" />
<script src="//cdnjs.cloudflare.com/ajax/libs/codemirror/5.58.1/addon/hint/show-hint.min.js" integrity="sha512-ge9uKCpgPmuJY2e2zPXhpYCZfyb1/R7KOOfMZ3SzSX3ZayWpINs3sHnI8LGEHUf6UOFX/D03FVHgR36uRL8/Vw==" crossorigin="anonymous"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/codemirror/5.58.1/addon/edit/closetag.min.js" integrity="sha512-8Ai5rTwZGpdbVS8LHO3VegWyY7OauoOBLgz7xnrEbpivphLu2x/TShlyVfN3l4Rjx0+XE3hWWJFyzopv3Lo1yA==" crossorigin="anonymous"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/codemirror/5.58.1/addon/hint/xml-hint.min.js" integrity="sha512-XtLGFClKrm3hNY3bS01LPiIkF64i9CnlxCqj5O+TSQq7UW8kFhFIc3kOR3bJ98h4ThxFaKdJA9PpQC76LvD/oQ==" crossorigin="anonymous"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/codemirror/5.58.1/mode/xml/xml.min.js" integrity="sha512-k1HnoY9EXahEfPz7kq/lD9DltloKH9OrB9XNKYoUQrNz9epe5F4mQP5PfuIfeRfoXHkNrE0gF3Mx4LhC5BVl9Q==" crossorigin="anonymous"></script>

Use of href="//[...]" and src="//[...]" instructs modern browsers to load the content using the same protocol (http or https) as the hosting page. This avoids security warnings about mixed content.

The current version of CodeMirror has slightly different .css, which required a small change to qbo.less:

.CodeMirror span {
    padding-top: 0px !important;
}

There is a small risk that a client's network blocks the CDN we've chosen to reference (cdnjs.cloudflare.com).

This Jasmine test verifies that all Javascript and CSS libraries load without errors. Available on the Specifications page under Application/Libraries.

MySQL support is available for QueueLog. This allows instrumentation for Queue processing to occur in a separate DBMS. Please review qbo3.Logging.MySQL readme for implementation details.

Queuing has been updated to store queues in a dedicated SQL table. This allows for increased agility on creating and managing queues. Queues must be seeded into Queue table from Configuration entry.

Steps to migrate/seed Queues:

• Run latest DB project. The Pre /Post Deployment scripts in Core will create the necessary data structures AND update any records in ObjectQueue table.

• Navigate to /Application/Queue.ashx/ConvertQueues. This will save and convert the Queues in ConfigurationEntry to the Queue table. It will also perform a second update on ObjectQueue table to bind records to the current Queue Uri values.

• Rebuild Entity View: Design > Data Tuning > Entity View > Rebuild.

Optional Queue Log Reconnect:

After the update, QueueLog will be disconnected from the respective queues for pending items. This will not impact processing ability, but will disable search ability for these items. Users can optionally reconnect any pending QueueLog records with the following script:

DECLARE @BaseUrn nvarchar(50) = [insert BaseURN here];

-- update FutureQueue

DECLARE @FutureQueue nvarchar(50) = 'FutureQueue'; 

DECLARE @FutureQueueLoggingUriID bigint = (SELECT QueueLogLookupID FROM QueueLogLookup WHERE uri IN (SELECT Queue.LoggingUri FROM Queue WHERE Queue = @FutureQueue)) 

DECLARE @FutureQueueBaseUrnID bigint = (SELECT QueueLogLookupID FROM QueueLogLookup WHERE uri = @BaseUrn)

UPDATE QueueLog      SET QueueLog.LoggingUriID = @FutureQueueLoggingUriID,     QueueLog.BaseUrnID = @FutureQueueBaseUrnID

FROM Queue,QueueLog INNER JOIN QueueLogCurrent     ON    QueueLog.QueueLogID = QueueLogCurrent.QueueLogID WHERE QueueLog.Queue = Queue.Queue     AND    QueueLog.Queue = @FutureQueue     AND QueueLog.LoggingUriID IS NULL

-- update FailureQueue

DECLARE @FailureQueue nvarchar(50) = 'FailureQueue'; 

DECLARE @FailureQueueLoggingUriID bigint = (SELECT QueueLogLookupID FROM QueueLogLookup WHERE uri IN (SELECT Queue.LoggingUri FROM Queue WHERE Queue = @FailureQueue)) 

DECLARE @FailureQueueBaseUrnID bigint = (SELECT QueueLogLookupID FROM QueueLogLookup WHERE uri = @BaseUrn)

UPDATE QueueLog      SET QueueLog.LoggingUriID = @FailureQueueLoggingUriID,     QueueLog.BaseUrnID = @FailureQueueBaseUrnID

FROM Queue,QueueLog INNER JOIN QueueLogCurrent     ON    QueueLog.QueueLogID = QueueLogCurrent.QueueLogID WHERE QueueLog.Queue = Queue.Queue     AND    QueueLog.Queue = @FailureQueue     AND QueueLog.LoggingUriID IS NULL

New queue functionality supported:

• Queue/Reset - Allows an existing Queue to be "reset" which clears the items and log. This is useful when a queue is erroneously loaded and the items need to be abandoned and not processed. The reset queue can be accessed and previous, valid items can be searched and moved to the active version of the queue. The command can be executed from the UI and will not hang the browser session.

• Queue/ResetLog - Allows an existing QueueLog to be "reset". This will not remove the items in the queue but reset the log so the pending items are cleared

• Queue/AutoCreate - Queues can be created dynamically with {ClassName}/Queue/{Operation}?QueueName={NewQueue}. New queues are defaulted with system settings and have an expiration applied so they can be removed later via automation

• Queue History - QBO standard history pattern is now applicable to Queue table so that users can view history outside of Queue log

• Queue/Delete will delete the queue record but will retain the underlying queue store and queue log for audit purposes. The command can be executed from the UI and will not hang the browser session.

UI Updates

Queue.ashx has been updated to traditional QBO home page pattern:

• Dashboard pattern implemented which allows filtering and sorting of queues
• Search panel now supports summary pop out with Reset and History as commands off Edit button
• Multiple queues can be selected for action. Previously only one queue was supported.
• Queue Item search menu commands: Process,Delete and Move are only supported if Queue store supports ID searching (eg. ObjectQueue, ServiceBroker)

Queuing Items From Data Tier

For any SQL queries that insert items directly into ObjectQueue, the queries must set ObjectQueue.Uri to CurrentQueue.Uri or they will be ignored by Queue Service. For example:

INSERT INTO ObjectQueue({Columns}, Uri)
	SELECT {Columns}, Queue.Uri
FROM {Tables}, Queue

WHERE {Tables.WHERE}
        AND Queue.Queue = 'CurrentQueue'

The login page has been refactored in the following way: Introduction of qbo.Application.Properties.Settings.RenderPassword defaulting to True, and qbo.Application.Properties.Settings.RenderRegistrationdefaulting to False.

In Theme.Core.xslt, template MainMenuLogin which renders the top bar of the login page, as well as template Login which renders the login controls have been updated to check for these particular settings. If RenderPassword is set to true, the password reset link "Forgot Your Password?" will render in both places, and RenderRegistration in the same vein will render the registration link if its value is true.

Since RenderRegistration is defaulted to False, if your product requires anonymous registration from this page, you must add a SystemDefault accordingly.

Register User supports creating a new user and applying roles.

The Design menu has been broken out by section to account for appropriate system role access.

Menu				Permission
----				----------
Security			Security

	User Accounts		PersonHome
	Roles			SystemRoleHome
	Group Access		ObjectAccessHome
	IP Addresses		SystemAddressHome

Templates			Templates

	{Template}s		{Template}Home

Rules				Rules

	Matrices		MatrixHome
	Linked Objects		ObjectLinkHome
	Business Rules		BusinessRuleHome

* Configuration			SystemConfiguration

	Calendars		CalendarHome
	Configuration Entries	ConfigurationEntryHome
	Data Tuning		DatabaseHome
	Encryption		EncryptionHome
	Logging			LoggingHome
	Modules			ObjectConfigurationHome
	Module Designer		ImportFileNewObjectHome
	Packages		ImportFilePackageList
	Queuing			QueueHome
	Routes			CustomRouteHome
	Schedules		ScheduleHome
	States			StateHome
	Style Sheets		StyleSheetHome
	ZIP Codes		ZipCodeHome

* Menu Designer			SystemConfiguration

* Recycle Application		ApplicationRecycle

* Next Item			SmartWorklistMemberNextItem

* Logout

* - Icon used in place of text for menu option.

IncludeIServiceOperationAsLoggingCategory now exists as an Application setting in qbo.Application that determines whether or not to add the operation/name of the IService as a category when logging after invoking the service. This is turned off by default, and allows easier control over IService logging as opposed to setting listeners for each of the different IService operations in a system.

The USPSCertified and AddressTracker Delivery Services that existed in DeliveryService.config within qbo3.Attachment now exist as configuration entries in a setup package contained in qbo3.AttachmentWeb named Setup.AttachmentDeliveryService.xml.

If your environments utilize these, ensure that this new setup package gets run.

• Support message recipient limits for outbound email providers
• MessageRecipients are sourced from MessageRecipient for existing messages

Outbound email providers (eg. SES) impose limits on the amount of recipients per email. Previously, messages were being sent to a respective provider with too many recipients causing rejection or inability to deliver to some recipients. Sugar has been added to allow Message cloning based on recipient limits of each provider to address the issue. Note Outbound provider usage is intended for the use cases where QBO sends emails with recipient counts in the hundreds. For mass email / marketing campaigns (> 1000) please consider SendGrid or MailChimp.

MessageTemplates allow configuring recipients dynamically from method signature, query or static list. When a message is created and saved, recipients are saved to MessageRecipient table and only dynamically sourced during creation. For subsequent message access, recipients are auto-filled from MessageRecipient. This is a optimization that eliminates unnecessary calls to auto-bind MessageRecipient when a message is subsequently accessed.

Queue Service now supports AWS Lifecycle hooks when application servers are created from an auto scale group. This allows for proper initialization and termination of application servers.

qbo.Configuration.Amazon should be included for all installations that target AWS.

For more information on configuring auto scale groups for QBO, please refer to Configure QBO to use AWS Autoscale Groups.

Queuing has been extended to support additional impersonation types and queue access is now enforced for elevated security queues.

Impersonation Types:

Non-Elevated Security

• Queued User - (Pre-existing). Queue will process items under the same security context that item was queued.

Elevated Security

• Inherited User - (New). Queue will process items under the same security context item was queued additionally under running as inherited user. This is useful when elevated permissions are necessary and extranet is required.
• Specific User - (New). Queues can be set to run under a specific user account. This is useful when queues must process items under a set security context.
• AdminUser - (Pre-existing/Deprecated). Specified administrator account. This enumeration will be deprecated and all queues running with this impersonation type should be updated to Specific User. Queues set to run under AdminUser can be converted by editing the queue from the application and convert to a SpecificUser.

Queue Access - Breaking Change

Elevated Queues require explicit access to submit items. Elevated Queues require permission granted to a SystemFunction with naming pattern:

Queue{QueueName}ElevatedAccess

Eg. for SecurityQueue the SystemFunction is QueueSecurityQueueElevatedAccess

Access can be granted under Role Management > Permissions.

The SecurityQueue is the first implementation of an elevated security queue. SecurityQueue allows users (both authenticated and non-authenticated) to process messages using queuing. As a result, SecurityQueue requires elevated security, and executes as a specific user with specific permissions.

Conversion Considerations

If a system has queues configured to run under AdminUser and the intent is to refactor permissions, consider the following:

• Update the queue to run under SpecificUser. Ensure the user specified only has permissions to process items designated for the queue. Also ensure the user is part of QueueAdministrators role which allows items to retry with correct permissions
• Identify what users and roles are submitting to the queue
• For each role grant access to Queue{QueueName}ElevatedAccess

Applications with queues specified as AdminUser must engage the security configuration. There is a helper method that locates all elevated security queues and grants access to specific roles.

Usage:

Queue.ashx/ElevatedBulkAccess?Queues={Queue1,Queue2}&Roles={Role1,Role2}&ExcludeRoles{OptionalRole1,OptionalRole2}

This method signature requires the Queues and Roles parameters.

Examples:

This function can be run multiple times and will not create duplicate SystemFunction or SystemPermission records. Caution should be used when granting access to all roles as this can introduce security holes.

The Functions.Substitute overload that is called with an XDocument and a queryString has been updated to fix a bug where Xslt Extension calls that evaluated to an empty string would throw an exception on a subsequent malformed evaluate. This bug has been fixed and is included in a new qbo3.Application package, along with multiple unit tests added to XDocumentExtensionFacts.cs.

This version of substitute regex matches on curly brace parameters and assumes that the matched value is an XPath expression, or a nested set of curly braces. For each match, it tries to XPathEvaluate the expression based on the XDocument provided. If the result isn't empty, it replaces the occurrences of the match with the result.

Within this logic, there is a ‘retry’ code path after the initial XPathEvaluate that checks if the result is null or empty, and that the original matched expression doesn't start with a / character. If these conditions were met, it was assumed that the curly brace expression exists in the form of the simple field name pattern such as ..&Contact={Contact}&..., therefore the code prepends the XPath global selector // to the beginning of the expression and evaluates the result, before the final check to see if we have a valid result to replace match occurrences with in the final string.

The issue occurs when the expression to evaluate is an Xslt Extension call instead, and the initial evaluation results in an empty string. The ‘retry’ code path conditions would be met, and would append // to an extension call such as //Formatting:formatDate(//CalendarItem/CreatedDate), for which the 'retry' evaluation throws an exception, as it should for this malformed XPath. This issue would also occur if the result of the extension call was intended to return an empty result.

The ‘retry’ code path now checks to see if the expression matches the dynamically generated XPathRegex in Application.Utilities.Extensions.ExtensionMethods based on configuration – and if it does, instead of appending ‘//’ and reevaluating, replace the occurrences of the match in the original string with an empty string.

Copy has been marked with an Obsolete attribute in favor of CopyAsync.

CopyAsync now checks to see if FileNameFormat is null or empty before trying to replace occurrences of the old template name with the new template name. The same check and replace has been added for PathURLFormat. This prevents errors when trying to copy a template that does not have a value set for either of these fields. The copy method is most commonly called from the front end, under the edit drop down menu.

In a recent commit, summary pages that should show a Group Access panel have been updated to call a new setting GenericPanelAccess to ensure this panel gets rendered as opposed to the default. The affected xslt files did not include a proper namespace xmlns:data="urn:qbo3-data in order to use the data extension function to retrieve the value of the setting. The affected templates have been updated, and new packages built. Please consume the following packages built today:

• qbo3.Accounting
• qbo3.Application
• qbo3.Attachment
• qbo3.Contact
• qbo3.Decision
• qbo3.Import
• qbo3.Message
• qbo3.Process
• qbo3.Report
• qbo3.Score
• qbo3.Security

In a recent commit, a new panel was added to the generic panel set 'Group Access' to allow power users easier access to managing access on an object. This was problematic, as the only Summary page that the Group Access panel should be rendered on are the objects that are normally managed by Group Access. An easy way to see the list of these is to investigate the filters on the main Group Access page.

This was fixed by creating a new qbo.Application setting GenericPanelAccess which acts in the same way as GenericPanelDefault with the exception that it contains Group Access as one of the values. Further, all Summary.Xslt files for objects that are managed with Group Access now call PanelGeneric with the value grabbed from the new setting so that they can continue rendering the Group Access Panel.

Updates to the following modules with packages built today (10/11/2019) must be taken in tandem to avoid any issues related to this change:

• qbo3.Accounting
• qbo3.Application
• qbo3.Attachment
• qbo3.Contact
• qbo3.Decision
• qbo3.Import
• qbo3.Message
• qbo3.Process
• qbo3.Report
• qbo3.Score
• qbo3.Security

All current app-tier and web-tier modules have been updated to embed core XSLT files into the built app-tier assembly. This comes with a change to how qbo3 looks for these files, where if a file is not found within the application root, it will attempt to look for it as an assembly resource. This means that overrides placed at the application root will still be respected.

The next time you update your qbo3 solution, ensure that you pull NuGet packages for the following no older than 10/7/2019:

• qbo3.Accounting
• qbo3.AccountingWeb
• qbo3.Application
• qbo3.ApplicationWeb
• qbo3.Attachment
• qbo3.AttachmentWeb
• qbo3.Contact
• qbo3.ContactWeb
• qbo3.Credit
• qbo3.CreditWeb
• qbo3.Debt
• qbo3.DebtWeb
• qbo3.Decision
• qbo3.DecisionWeb
• qbo3.Exception
• qbo3.Import
• qbo3.ImportWeb
• qbo3.Logging
• qbo3.LoggingWeb
• qbo3.Message
• qbo3.MessageWeb
• qbo3.Mortgage
• qbo3.MortgageWeb
• qbo3.Process
• qbo3.ProcessWeb
• qbo3.Report
• qbo3.ReportWeb
• qbo3.Score
• qbo3.ScoreWeb
• qbo3.Security
• qbo3.SecurityWeb

qbo3.Fintech and qbo3.Fintech.Full NuGet packages have been updated accordingly for this change. In both solutions and packages, the total file count has dropped drastically, which should in turn reduce the time it takes to update either package. For the first run to consume this change, the update run-time should be theoretically halved, as it will delete the files it no longer contains in the solution, and not reintroduce them. After the first run, update speeds are expected to be significantly shortened.

qbo.Attachment has been updated in relation to Attachment.cs and AbstractFile.cs to be more consistent in calling async methods when already within an async method. This may provide some lift when working with Attachment objects.

CopyFileAsync in Attachment.cs has been updated to be consistent with a change in CopyFile, where instead of returning the raw result of calling GetRelativePath(relativePath), it now returns PathURL. This is important because plugins can manipulate this field using the returned AttachmentInfo object.

qbo.Application has been updated to address items becoming "stuck" when processing under an inactive user. Common use cases:

• Future items processing as they become current
• Items in non system queues
• Items in FailureQueue

All items will route to the FailureQueue with appropriate message. Note that if the user is re-activated, items can be processed from FailureQueue and automatically routed to original queue.

Additional seeding of Person, SystemMember and Role is required. Setup.QueueInitialize.xml has been updated to include user: [email protected], SystemRole: QueueAdministrators

AbstractObject/InstanceList has been extended to accept a parameter to override the default Select statement current used to initialize Objects. The purpose is to allow more control over the statement with the initial use case to reduce the amount of un-necessary joins to initialize objects.

This extension has been utilized in the following classes:

• qbo.Decision - Decision/Process and DecisionStep/Start InstanceList now configured to initialize from project setting InstanceListDefaultStatement which defaults to SelectBasic vs Select

• qbo.Import - ImportFileQueue/Process InstanceList now configured to initialize from project settting InstanceListDefaultStatement which defaults to SelectBasic vs Select

**Both of these updates will exclude a join to EntityView for initialization.

Using a new qbo.Attachment setting SetAttachmentToFileName, new Attachment objects created from an AttachmentTemplate will default to the value of FileName instead of AttachmentTemplate.AttachmentTemplate at the end of Attachment.cs/SetDefaults if FileName is not null or empty.

This setting is turned on by default, and will mean that new Attachment objects will not contain the name of the associated template, but instead contain the FileName set on the object or derived earlier in Attachment.cs/SetDefaults (using FileNameFormat, PathUrl, so on).

Along with this change, if 'FileName' is empty Attachment.cs/SetDefaults will make a final attempt at setting FileName by deriving the value from PathURL using System.IO.Path.GetFileName(PathURL).

Recall that the Attachment column is displayed in the hyperlink text to the user, and that FileName is used as the Content-disposition header when responding to a request with file data (therefore the file name if downloaded). If there is a need to default Attachment/the hyperlink as the AttachmentTemplate (in keeping with the previous logic), you may create a SystemDefault for SetAttachmentToFileName and set it to False.

The SetDefaults override in Attachment.cs has been changed in regards to setting the Attachment.Attachment field with a new Attachment setting SetAttachmentToFileName. This will set the Attachment.Attachment field to be the value of the FileName at the end of SetDefaults, regardless if Attachment.Attachment is empty or not.

Detailed Change:

The method first starts to fill the different fields of the Attachment object based on the values of an AttachmentTemplate only if the field is empty or null. It then proceeds to call GetRelativePath() on fields if they need Function Substitution. This logic has not changed.

Here is where the logic changed.

At the end of SetDefaults, if Attachment.Attachment was still empty at this point but FileName was populated, then set Attachment.Attachment to the FileName. Finally, it called GetRelativePath() on the Attachment to ensure values were substituted.

Now with this change, Attachment.Attachment will be set to the FileName if Attachment.Attachment is empty just like before, OR if a new property SetAttachmentToFileName is set to true. It is set to true by default. Else if Attachment.Attachment is not empty, it will call GetRelativePath() on the value to ensure substitution.

In order to progress development with the Statement Builder and the underlying SqlPattern class, a virtual method ChildJoinClause has had its method signature changed. The classes that override this method have been modified to consume this change.

The following modules must be updated in parallel, with NuGet versions built at the very earliest by 9/16/2019:

• qbo.Application
• qbo.Accounting
• qbo.Contact
• qbo.Decision

qbo.Import has been updated to support two new optional parameters:

• OnCompleteStatus
• ActualStart

OnCompleteStatus allows ImportFile.Status to be set to a status other than Complete when the import completes. This is useful when launching workflows after an import completes and the management of the ImportFile state flow is transitioned to the workflow. ImportFile.Status can be set to a value that will not mislead the consumer.

ActualStart allows workflows launched IAW ImportFileTemplate to be future queued for processing. This is useful when granularity is required to allow an import to process but stage the workflow tied to the IFT until a future date. Since the tracking of records is using queuing, tractability exists in the queue module. This is facilitated by setting Decision.ActualStart to a future date.

qbo.Decision now supports future queuing workflows upon creation if Decision.ActualStart is future dated. If Decision.ActualStart is not future dated, the workflow will process immediately for backward compatibility.

UncFile.cs has been updated to support connectivity to Amazon FSx. FSx requires active directory which requires a domain to be specified in the UNC credential. Connectivity to FSx follows the same pattern as existing UNC connectivity:

• Windows account with matching username and password must also exist on the server initiating the connection. Since most of our web and application servers are not part of an active directory, the account can be created with the username and password excluding the active directory name
• Windows account with matching username and password must exist on the target
• Account must have necessary permissions to access the resource

// FileObject
<FileObject Name="AWS FSX" Uri="//{FSXIP}/root/" Type="qbo.Attachment.FileObjects.UNCFile, qbo.Attachment".../>

// Credential
<Credential UriPrefix="file://{FSXIP}/root/" AuthType="Basic" Username="user" Password="pass" Domain="domain" />

The UNCFile class will detect a domain is specified and adjust the LogonType and LogonProvider settings to authenticate with Active Directory. Also note that in FileObject settings, LogonType and LogonProvider can be specified to override default behavior

Theme.Core.Panel.xslt, ObjectAccess.ACL.xslt and Matrix.Select.xslt have been updated to refactor out the 'Manage Access' screen to a new core panel. This is originally found on Matrix.ashx/Summary under the 'Edit' drop down. This was done to solve an issue with pagination in popups, but the bonus value-add this addition brings is the panel can now be used for object types other than Matrix.

This update requires the latest qbo.ApplicationWeb and qbo.SecurityWeb.

qbo.Application has been extended with a new application setting qbo.Application.Properties.Settings.QueueCountOrderBy which defines the default OrderBy parameters utilized in Queue.Summary.List.xst, and therefore the ordering of the data set used initially/on first UI draw. The default for this setting is QueueLogID, which will effectively produce the same behavior as before.

This update requires the latest qbo.Application and qbo.ApplicationWeb.

This allows the configuration of choosing the most-desired OrderBy behavior, so that users don't have to continuously click and wait for the screen to load twice. For example, you can set the default order to the latest Queued Date by setting the value to -QueuedDate. This value will utilize the SqlPattern class functionality and produce an ordering of QueuedDate by descending order.

S3FileObject.List has been found to be subject to pagination via the underlying request and response object in the AWSSDK. It was causing issues when a particular S3 path contained more files than the paging limit during use-cases where the pattern passed was not a wildcard pattern, but a filename; the file can be missed since it might not exist in the first page.

This use-case can exist in custom API calls to FileObject/ListFiles that specify a filename as the pattern, but the main area this was affecting was in the implementation of CachedFile, where parts of the relativePath were broken up into the FilePath and FileName, and being passed as path and pattern respectively to BaseFileObject.List (S3 in this case).

S3FileObject now contains a property qbo.Attachment.Amazon.Properties.Settings.ListWithPattern defaulted to false; when this property is set to true, List checks to see if the pattern passed contains Wildcards, and if it does not it appends the value of pattern to the AWS request object to narrow down the search, therefore circumventing the need to work with the pagination, worrying about large data sets when trying to list a particular file in a heavily populated S3 path.

Configuration Monitoring method signature has been updated. Classes that implement Monitor will need to be updated. The following core classes have been updated:

• qbo.Application
• qbo.Attachment
• qbo.Security

Please ensure to get latest packages when pulling in latest qbo.Application. Please also be sure to update any classes that implement Monitor as Configuration Monitoring will fail if any of the dlls do not support the update.

SystemDefault configuration monitoring has been updated to support the Revert function and update config files accordingly. Previously Revert would only remove the record from the SystemDefault table and leave overridden settings in web.config that would require manual clean up.

SystemDefault configuration monitoring contained a bug where null DB values would cause an infinite AppDomain and Queue Service reset within the application. This has been addressed by masking null DB values as empty string to match web.config serialization. Configuration monitoring has also been updated to disqualify any invalid settings by setting SystemDefault.Filter = Invalid and allow configuration monitoring to proceed with setting updates.

qbo.Application and qbo.ApplicationWeb Queuing.config no longer contain system queues. System queues are defined as: DefaultQueue, CurrentQueue, CallbackQueue, FutureQueue and FailureQueues. These queues are now sourced from the Queue initialization file Setup.QueueInitialize.xml. When updating to latest qbo.ApplicationWeb, the target system will lose system queues. Here is the path to updating the target system.

New Installations

• Run Setup.QueueInitialize.xml

Existing Installations

Minimal concern if existing System queues and Calendar settings are NOT critical:

• Run Setup.QueueInitialize.xml. This will overwrite any updates to Standard Queue Calendar and system queues

High concern if existing System queues and Calendar settings are critical:

Setup.QueueInitialize.xml will have to be manually reviewed and items will be selectively included. The sections in the seed script to consider are Calendar and Queues.

• Verify there are no updates to Calendar Standard Queue Calendar. If so, export Standard Queue Calendar
• Verify there are no updates to system queues. This can be done by querying ConfigurationEntry:

SELECT * FROM ConfigurationEntry WHERE Source = 'Queuing.config' AND ConfigurationKey IN ('FailureQueue', 'FutureQueue', 'CallbackQueue', 'CurrentQueue', 'DefaultQueue', 'SystemMaintenance')

Exclude any Queues that exist in ConfigurationEntry when manually importing along with any Calendar overrides.

ConfigurationEntry has been extended with a new [DbXmlReaderMethod] Validate. Validate is a tool designed to assist developers and power users ensure that ConfigurationEntry rows in a database are ready for an installation of QBO to use. The method returns data describing the results of validation for the particular ConfigurationEntry rows checked.

You may optionally supply an ConfigurationEntryID list parameter to narrow down the validation as opposed to all applicable rows.

You may also pass another optional parameter MarkInvalid=true to set and update the Filter to "Invalid" for the rows that failed validation.

The way Validate performs its check is to test to see if calling SetProperties on a ConfigurationEntryObject in memory succeeds without throwing an exception. SetProperties is defined as a custom overridden method for a ConfigurationEntryObject as opposed to relying on AbstractObject's version of it. This override is not new and has been in the code base for some time. The override attempts to resolve the underlying configuration class's type using reflection, and then calls SetProperties on that particular type with the ConfigurationXml of the ConfigurationEntry in question using its attributes. SetProperties on configuration classes performs internal checks while populating fields, such as resolving a passed Type. If this succeeds, the override continues by calling the Type's WriteXml and saves the results as the ConfigurationEntry's ConfigurationXml. The primary use case of the override is picking up custom logic implemented in a particular configuration class's WriteXml.

Under normal circumstances, saving a ConfigurationEntry should fail if it has bad data or is malformed, but as a systems code base changes, often times a disconnect surfaces between what exists in a build and the database's ConfigurationEntry records. This tool gives the ability to catch these disconnects after a deploy and fix them before configuration errors catch developers and power users off guard. An example of this is a missing plugin assembly after a deploy.

JasmineValidate has been added to ConfigurationEntry.cs as well, with an accompanying Jasmine test spec under Application>ConfigurationEntry. JasmineValidate takes all the parameters for Validate, calls it, and returns a simple data structure noting if the Validate call returned all valid items or not. The method utilizes qbo.Application.Properties.Settings.JasmineValidateExceptionSubstringIgnoreList to check error messages against in the case that you want to ignore a particular exception message and still return a success status for the Jasmine spec.

The DelimitedGenerator creates delimited file attachments (.csv typically) from a Dataset. Since a Dataset can contains multiple DataTables, multiple attachments can be generated. If the FileName and PathURL may include a {Table} expression, DelimitedGenerator will substitute {Table} with an inferred table name, so the files names are intuitive.

The bug arose when the AttachmentTemplate.PathUrl / FileNameFormat use {Table}, but the call to Functions.Substitute included null parameters.

As part of this fix, DelimitedGenerator was changed to:

• Default the FileNameFormat to {Table}.{DateTimeStamp}.csv', instead of {{Table}}.{DateTimeStamp}.csv'

Potential breaking change: file names will now omit extra {}: Loan.2019-05-24.csv instead of {Loan}.2019-05-24.csv

• Substitute Attachment.FileName first
• Substitute Attachment.PathURL second
• Substitute Attachment.Attachment last

The XsltExtensionConfiguration section was updated to derive from BaseConfiguration, which in turn requires defining a SectionName property. This raised a conflict:

• The embedded resource follows the qbo/{Extension} naming pattern of all other resources (e.g. qbo/XsltExtension)
• Legacy web.config files break from this convention, using qbo/XsltConfiguration instead of qbo/XsltExtension

To ensure that XsltExtensionConfiguration will work with both web.config and the embedded resource, you must modify legacy web.config files as follows:

// Legacy version
<section name="XsltConfiguration" type="qbo.Application.Configuration.XsltExtensionConfiguration, qbo.Application" />
// ...
<XsltConfiguration configSource="config\XsltExtension.config" />

// New version
<section name="XsltExtension" type="qbo.Application.Configuration.XsltExtensionConfiguration, qbo.Application" />
// ...
<XsltExtension configSource="config\XsltExtension.config" />

Lastly, if you are not using the embedded sources, ensure that you have the current version of XsltExtension.config from qbo.ApplicationWeb > Config.

Global.asax has been extended to allow a new scheme of authorization revolving around the OpenID specification involving Bearer tokens that come in Authorization HTTP headers. This change involves modification of the Global.asax file itself in qbo.SecurityWeb, a new class structure in qbo.Security to handle header authorization, and a new plugin solution qbo.OpenID which includes a project qbo.Security.OpenID that implements the new scheme.

These are not breaking changes. Basic authentication works exactly as before, and the plugin calls are via reflection. In order to use the new functionality, be sure to include the plugin assembly qbo.Security.OpenID and have updated qbo.Security and qbo.SecurityWeb modules as of this date.

The new Bearer login scheme implemented allows a highly trusted OpenID identity server to provide JWT Access tokens for clients that allows the user to either log in to QBO, or automatically create a new user and log them in to QBO, all based on the claims set by the identity server.

After installation, the following properties can be overridden to enable functionality:

Attachment/Generate has been updated to explicitly Dispose of the data object used when generating documents. In the case of data object invoking an IDataReader, the database connections would stay open and under high concurrency this would ultimately lead to database timeouts when the application tries to obtain connections from the connection pool.

This will be available in Version 3.0.7 of qbo3.Attachment and is available now in SVN/Trunk.

Version 3.0.6 of qbo3 nuget packages have been released to our public Nuget feed. Key changes include:

• Debug symbols are now included in the qbo3 Nuget packages
• Embedded configuration files are part of each application tier module

All core configuration sections are now configured to read from embedded configuration files. This means that web.config no longer needs to contain the qbo section handler for ObjectConfiguration-based config files, such as Matrix.config, Contact.config, Person.Config, etc.

If web.config contains a section for any configuration, that will be used instead of the embedded configuration. However, use of external .config files should be deprecated in favor of using the embedded configuration files mixed with ConfigurationEntry populated by setup packages.

Config files such as Css.config and Javascript.config are available as embedded resources, but the web tier components that load them have yet to be updated to leverage the new embedded configuration. Developers wishing to leverage embedded configuration should change:

ConfigurationManager.GetSection(section) as T

to

BaseConfiguration<T>.Load()

Migration of all calls to ConfigurationManager to BaseConfiguration will be available in the 3.0.7 release.

Configuration sections based on qbo.Application now support loading from embedded .config resources. This means that the web.config file does not need to include explicit references to Abstract.config, Matrix.config, Css.config, etc. To support embedded loading of EnterpriseLibrary resources, the qbo.Exception project now includes a EnterpriseLibraryConfiguration class that will load default EnterpriseLibrary configuration from code.

Marked CacheManager (based on EnterpriseLibrary) as obsolete; use Cache (based on qbo.Application.Interfaces.ICache instead).

Created IExceptionHandler interface, and implemented:

• GenericExceptionHandler: routes to one of the below handlers, based on OutputMethod
• HtmlExceptionHandler: default handler, renders HTML-based exceptions as QBO has always done
• XmlExceptionHandler: renders exceptions as XML if the OutputMethod == OutputMethod.Xml
• JsonExceptionHandler: renders exceptions as Json if the OutputMethod == OutputMethod.Json
• qbo.Msp.ExosExceptionHandler: provide custom error handling for ServiceLink's Exos applications

Extended CustomRoutes to:

• Support route constraints :date and :isodate
• Support an IExceptionHandler for custom exception handling
• Added test interface to UI to evaluate a URL and return all matching routes

Extended the SqlService plugin to support passing entire Request.Body to a stored procedure.

When serializing method signatures to JSON, the default behavior is to omit null values. This default behavior is controlled from a new application setting qbo.Application.Properties.Settings.Default.JsonSerializeNullValues. If true, null values will be emitted, otherwise they will be omitted.

On a per-request basis, the setting can be altered by passing a custom header:

JsonSerializeNullHeader: true | false

or by passing a custom parameter on a query string:

&JsonSerializeNullHeader=true|false

The configuration files for AbstratObject-derived classes can now be included in application code as embedded resources, and omitted from the web site's web.config file. See qbo.Application.Configuration.ObjectType.config as an example.

qbo.Attachment has been modified to allow additional granularity when setting Attachment PathURL and FileName. There are use cases that require Attachment.FileName and the filename contained in Attachment.PathURL to be different.

AttachmentTemplate has been extended to contain PathURLFormat. PathURLFormat replaces the functionality of the existing FileNameFormat column and the purpose is to set a pattern for Attachment.PathURL. AttachmentTemplate.FileNameFormat is now used (for exactly what it indicates which is) to set Attachment.FileName. AttachmentTemplate.FileNameFormat is an optional parameter and Attachment.PathURLFormat is a required value.

This is a breaking change for existing systems and a database update must be run to set AttachmentTemplate.PathURLFormat to existing AttachmentTemplate.FileNameFormat and then set AttachmentTemplate.FileNameFormat to NULL. qbo.Db.Stardard pre-and-post deployment scripts have been updated to detect and run a one-time update to AttachmentTemplate. By running the qbo.DB script, Attachment naming will function the same as before.

The BaseObjectConfiguration class has been extended to include a hook to allow derived classes to read configuration information from an embedded resource (.config file) instead of from web.config / external .config files. This allows core configuration files to be deployed with application-tier modules, without needing web.config. This should enabled much better isolation of tests.

To prove this concept, the following steps were taken:

• FileObject.config was added to the qbo.Attachment.Configuration folder, with Build Action marked as Embedded Resource.
• override void FileObjectConfiguration.LoadDefault() was added to FileObjectConfiguration.cs, and reads configuration from the FileObject.config embedded resource:

public static System.Reflection.Assembly _assembly = typeof(qbo.Attachment.Configuration.FileObjectConfiguration).Assembly;
public override void LoadDefault()
{
  base.LoadDefault();
  // the embedded XML file cannot have an XML declaration
  using (var stream = _assembly.GetManifestResourceStream("qbo.Attachment.Configuration.FileObject.config"))
  {
    using (var reader = XmlReader.Create(stream))
    {
      DeserializeSection(reader);
    }
  }
}

• FileObjectCollection.cs was modified to define the Source file, since it's embedded:

public override string Source => "FileObject.config";

Note that there should be no xml declaration section in embedded configuration files. Microsoft's ConfigurationSection.DeserializeSection() method will throw an error if the beginning of the document is not positioned on an element.

QBO operations (methods, statements and services) may now return a JsonReader, in addition to the existing DataSet, DataReader, XmlReader, Object, Collection and Void return types. Two uses cases for this include:

• IService plugins that pull JSON data from a remote source don't need to convert to an object or XML, and
• Statements that use SQL's FOR JSON clause

For example, the following SQL can return JSON data:

SELECT Contact.FirstName, Contact.LastName 
FROM Contact 
WHERE ContactID=1
FOR JSON PATH, WITHOUT_ARRAY_WRAPPER

returns

{ 
  'Contact' : {
    'FisrtName': 'John',
    'LastName': 'Doe'
  }
}

Note that HttpAsyncHandler can convert a native JsonReader to XML or pass to an XSLT (by converting to XML first).

Custom routes now support HttpMethods, so they can be bound to one or more HTTP methods (get, post, put, delete, patch).

If a SQL query raises an exception, the SqlService can map the exception to a native .NET exception, leveraging HttpAsyncHandler's translation of .NET error codes to RESTful HTTP responses.