Core Features - Gadreel/divconq GitHub Wiki

dcSchema

Use dcSchema to define how structured data is moved around within DivConq - be that between servers, between Java and dcScript, via web services or to stored procedure calls. In the end DivConq treats all data as JSON-like structures. These structures can be validated before they are allowed to be passed to other services, scripts, procedures, etc.

dcBus

With DivConq we promote putting most application (business) logic into a service. DivConq provides the features needed to make it possible to send JSON like data structures as parameters to Services during a service call. Also, DivConq makes it easy to return JSON like data structures from a service. In addition there is a validation layer so that you may validate parameters before the service call is made and you may validate the return value before the calling code accepts the result.

Services are Java classes or Groovy scripts and their definitions are published in the schema file (dcSchema). When multiple servers are deployed services may load balanced. External applications may (with the correct credentials) make "native like" calls to the DivConq service bus by passing in JSON to the web interface. JSON is returned from the service call result.

dcScript

A simple, resource light and secure scripting language for use within DivConq applications. DivConq provides the features needed to make it possible to send JSON like data structures as parameters to scripts during a call. Also, DivConq makes it easy to return JSON like data structures from a script. In addition there is a validation layer so that you may validate parameters before the script call is made and you may validate the return value before the calling code accepts the result.

dcWeb

DivConq's service bus is exposed to the world via web services - both through HTTP RPC calls and through Web Socket calls. Given the full-duplex nature of web sockets and HTTP/2, it is possible to achieve a nearly native dcBus feel within external applications connecting to the web server. However, external applications are not trusted the same as internal application components are and as such only a subset of services are available to external applications.

A simple web framework is available for developing web applications that use DivConq's features.

dcWorkpool

DivConq promotes the use of a work pool for threaded applications. Don't create your own threads, instead allow the work (thread) pool to be tuned to handle the workload. When using the work pool the task context (user id, authorization, task id, debug and locale settings) automatically transfers to the new thread for the span of that work.

If the work pool is overloaded, adding more threads probably won't help. Most areas of DivConq are designed to be non-blocking so when a thread is busy it probably IS busy. As such, allow the work pool to act as a natural throttling tool - work will get done ASAP, and if to much work is requested you'll just have to wait. Spin up another server if the work load is too severe.

There is also a scheduler if you need a task to repeat at regular/calculable intervals.

dcWorkQueue

The work queue is like the work pool in that you submit a task and it runs. However, work queue supports a "farm" of DivConq servers. Tasks in the work queue may be picked up and run by any of the DivConq servers in the farm. Thus while the Task is run on only one server, the mechanism for running tasks is distributed.

Logging

A debug logger comes built-in to DivConq. Each task gets an id and that id is included in the debug log. Although the log entries are interleaved in the log file, given the task id it is easy to extract the messages specific to that task. Further, log entries are (typically) translated so when a translation file exists for your language it is possible to have a debug log in your native language. Translation codes are included in the log entry, thus making it possible to partially translate an existing log file into another language.

dcRepository (future)

Admin's may build customized projects by combining a variety of DivConq components developed by third parties or in-house.

dcDB (future)

Combines concepts from SQL and noSQL into a single data store. Provides data replication with eventual data consistency across multiple active databases in multiple data centers. Is capable of replication even over high latency networks.

For your relational (table) like needs DivConq provides dcTables. dcTables has a schema and, although not SQL compliant, it supports many familiar features such as Insert, Update and Select. dcTables does not promote Delete but instead supports Retiring and Reviving records (rows). There are a number of other features in dcTables, such as time-oriented values (data changes over time, but you don't want to lose the old values).

DivConq also supports dcSimpleData, a basic multi-key + value storage feature. Queries in dcSimpleData may retrieve one or more keys based on filters. dcSimpleData is schemaless like many other noSQL databases.

One of the goals of dcDB is to be able to query a large and complex result set in a single call. To this end dcDB supports stored procedure calls that can return any JSON like structure (objects, lists, values). A single query may collect data from both dcTables and dcSimpleData. Stored procedures are written in Groovy.

dcDB dramatically reduces the complexity of database deployment. All-in-one data persistence makes development and support much easier, and replication leap-frogs many traditional SQL engines where even configuring mirrored database servers is a pain. The one caution is that stored procedures which update dcSimpleData must be carefully coded to support replication.

DivConq will support many other helpful features such as live database backups, import and export tools, and even replication from dcTables to a relational (read-only) database.