DI: Data Integration - boxalino/rtux-integration-magento2 GitHub Wiki

The Magento2 Data Integration Layer is available as a stand-alone repository & plugin https://github.com/boxalino/data-integration-magento2

The use of the Data Integration Layer requires to use the samples & guidelines newer than tag 3.0.0 (or master branch) of the Integration Layer Guidelines https://github.com/boxalino/rtux-integration-magento2/tree/3.0.0. This will, respectively, update the Framework Layer (rtux-magento2) to version ~2.4.0

The complete documentation is available on the data-integration-magento2 wiki page

The integration mode refers to the integration strategy:

1. Full : the full content for the given type and the adjacent documents required for it.
2. Delta : partial content for the given type, based on the sync check (when was the last sync)
3. Instant (not released) : targeted content update (special fields with high impact on real-time response), based on the changed ids. !!!Requires integration review & test by the client!!!

The integration type refers to the content to be synchronized:

1. Product
2. Order
3. User

For other integration types (blogs, etc) from 3rd party sources, please review the Transformer service documentation

---

Due to the custom nature of the data integration, the used interfaces are to be defined in the Boxalino Integration layer repository from the project`s codebase.

For a generic out-of-the-box integration, the following aspects are to be configured: https://github.com/boxalino/rtux-integration-magento2/blob/3.0.0/etc/di_config/di-full.xml

There are 2 ways to ensure that the full & delta running daily:

php bin/magento boxalino:di:full:product

The available commands, once integrated, follow the pattern boxalino:di:mode:type

Check the wiki pages from the Data Integration Layer repository on:

1. Full Data Integration
2. Delta Data Integration
3. Instant Updates

For generic integration options reviews and purpose, use the Integration Strategies confluence page

The DI layer includes the following elements:

1. Console : the CLI commands for triggering different mode&types of data sync . You can use the base to create custom CLI commands as well.

2. Integration Handlers : they are declared per content (product, order, user) & mode (full, delta, instant). They have a single role: to run the integrate logic.

For ex: the ProductIntegrationHandlerInterface manages the integration of product full exports. It receives configuration for docHandlers (handlers to export different kind of doc_X content: ex: doc_attribute (DocAttributeHandlerInterface), doc_attribute_value (DocAttributeValueHandlerInterface), doc_language (DocLanguageHandlerInterface), doc_product (DocProductHandlerInterface) ). Once these files have been integrated, it makes the SYNC REQUEST. The SYNC REQUEST triggers the computation on Boxalino DI endpoints.

Each of the document handlers mentioned above is responsible to export the JSONL for the respective data structure. Due to this, they are entirely customable.

Every attribute in the data structure (ex: doc_product) is managed by a property handler element. The propertyHandlers are declared in the integration layer, as part of the DocProductHandlerInterface definition.

Each property handler is using a data provider (in terms of Magento2: a model and a resource). The data provider is responsible for accessing the DB and giving access to the content.

The mapping between the property handler and the data provider is done with the help of a resolver, based on the item name in the di.xml.

This is done in 3 steps:

1. Create your data provider model & resource
2. Update the docProductSchemaDataProviderResolver definition in di.xml for the property to be updated (ex: to replace visibility, <item name="visibility" xsi:type="object">Boxalino\DataIntegration\Model\DataProvider\Document\Product\Visibility</item> becomes <item name="visibility" xsi:type="object">BoxalinoClientProject\BoxalinoDataIntegration\Model\DataProvider\Document\Product\Visibility</item>)

The custom model&resource from your integration layer can use the default/baseline available in boxalino/data-integration-magento2. This will ensure that the other functionalities (ex: setting configuration, switching export modes full-delta-instant, data format) is maintained.

One full data integration per day

One delta per hour or as often as every 20-30min (project-based)

In case your project is sensitive to data updates (stock, price, categories), activate the instant data updates as often as every 5min.

One full data integration with the initial setup

MINIMUM One delta per day (to update the order status) DESIRED A delta per hour (if subscribers are active)

One full data integration with the initial setup

One delta per day (to add new users or update the existing ones)