InspeXtor Communication Architecture - AK-Khalis/mht-inx-wiki GitHub Wiki

An Instance represents a singular unit within the InspeXtor system, whether it serves a residential or business setting. The On-premise instance orchestrates the connection of various Devices (e.g., Lights, Shades, RGBW) and Sensors (e.g., Motion sensor, Steinel, O3 sensor) within the client's location.

Each On-premise instance is linked with a common cloud computing infrastructure referred to as Cloud InspeXtor. This cloud instance manages subscriptions and expiration for On-premise instances.

Communication between On-premise and Cloud InspeXtor instances occurs continuously, employing various methods depending on specific settings. This documentation outlines the communication settings and diverse communication types:

1. User Request Handling:

   • User request is received from the On-premise system and processed in the backend.

2. Backend Processing:

   • Backend sends a request to CommonFunction::send_packet_to_python() with the packet containing necessary details.
   • A log data entry is created in the pmi_instance_to_cloud_log table, utilizing the log primary key ID for status verification.

3. Socket Connection:

   • Connects to the Python web socket 127.0.0.1:9998 and passes the data to the socket.

4. Python Processing:

   • Python web socket 127.0.0.1:9998 accepts and processes the data in a threaded manner.

5. AWS IoT Core Connection:

   • Python establishes a connection with the AWS IoT Core through the MQTT protocol.
   • Publishes to the AWS topic inx/activate.
   • If the connection or publish fails, it retries up to 3 times before returning the status as success or failed.

• Subscription:
  • AWS IoT Core subscribes to the topic inx/activate.
• Republishing:
  • AWS IoT Core republishes the packet to the topic inx/activate/cloud.

• Subscription:
  • Cloud InspeXtor subscribes to the topic inx/activate/cloud.
• Processing:
  • Packet is sent to the PHP processor bgprocs/processActivateInstance.php.
• Response:
  • Result data is returned to On-premise through the AWS topic inx/<ContractId>/activate.

• aws-cloudsync-socket-proc (Script: AwsCloudSyncSocketProcess.sh)
• aws-cloudsync-subscribe-proc (Script: AwsCloudSyncSubscribeProcess.sh)

For activation:

• JSON Data Preparation:

  {
      "c":"Sync",
      "a":"activate",
      "instanceId":"<InstanceID>",
      "instancePass": "<Instance Password>",
      "contractID" : "<Instance Contract ID>",
      "type" : "<Instance Type>",
      "name" : "<Instance Name>",
      "location" : "<Instance Location>",
      "phone" : "<Instance Phone>",
      "ta" : "activate"
  }
  
  

For deactivation:

• JSON Data Preparation:

  {
      "c":"Sync",
      "a":"deregisterInstanceAction",
      "instanceId":"<InstanceID>",
      "instancePass": "<Instance Password>",
      "contractID" : "<Instance Contract ID>",
      "type" : "<Instance Type>",
      "name" : "<Instance Name>",
      "location" : "<Instance Location>",
      "phone" : "<Instance Phone>",
      "ta" : "activate"
  }
  

1. User Request Handling:

   • User request for deregistration is received from the On-premise system and processed in the backend.

2. Backend Processing:

   • Similar to registration, backend sends a request to CommonFunction::send_packet_to_python() with the packet containing necessary details.
   • Log data entry is created in the pmi_instance_to_cloud_log table.

3. Socket Connection:

   • Connects to the Python web socket 127.0.0.1:9998 and passes the data to the socket.

4. Python Processing:

   • Python web socket 127.0.0.1:9998 accepts and processes the data in a threaded manner.

5. AWS IoT Core Connection:

   • Python establishes a connection with the AWS IoT Core through the MQTT protocol.
   • Publishes to the AWS topic inx/activate with the appropriate payload for deregistration.

• Subscription:

  • AWS IoT Core subscribes to the topic inx/activate.

• Republishing:

  • AWS IoT Core republishes the packet to the topic inx/activate/cloud.

• Subscription:
  • Cloud InspeXtor subscribes to the topic inx/activate/cloud.
• Processing:
  • Packet is sent to the PHP processor bgprocs/processActivateInstance.php.
• Response:
  • Result data is returned to On-premise through the AWS topic inx/<ContractId>/activate.

• aws-cloudsync-socket-proc (Script: AwsCloudSyncSocketProcess.sh)
• aws-cloudsync-subscribe-proc (Script: AwsCloudSyncSubscribeProcess.sh)

The Ping procedure is a continuous backend process used to check the connection existence between the on-premise and cloud InspeXtor. This process operates based on the setting of the CLOUD_CONNECT module.

• JSON Data Preparation:

  {
    "contract_id": "<Contract ID>",
    "time": "<Current time>"
  }
  

• Topic: inx/ping
• Scripts: inx_aws_iot/inx_socket.py and inx_aws_iot/publish.py

1. After successful publish, wait for the cloud response by MQTT subscribing to the topic inx/<Contract ID>/ping.
2. The message is sent to the PHP processor bgprocs/processPingResponse.php through a post method from Python in inx_aws_iot/pahosub.py (function: send_pkt_to_php()).
3. Update the response from the cloud to On-premise InspeXtor.

• aws-cloudsync-ping-proc (Script: AwsCloudSyncPingProcess.sh)
• aws-cloudsync-subscribe-proc (Script: AwsCloudSyncSubscribeProcess.sh)

• AWS IoT Core subscribes to the topic inx/ping.
• AWS IoT Core republishes the packet to the topic inx/ping/cloud.

• Cloud subscribes to the topic inx/ping/cloud in inx_aws_iot/pahosub.py.
• Send packet to the PHP processor bgprocs/processPingCloud.php in inx_aws_iot/pahosub.py (function: send_pkt_to_php()).
• After processing the result data, return to On-premise through the AWS topic inx/<ContractId>/ping.

Database synchronization is a continuous backend process to send database changes from on-premise to cloud InspeXtor, which operates based on the existence of the CLOUD_CONNECT module.

The database synchronization involves two main processes:

1. Database Change Query Builder: This process creates changed SQL queries files and compresses them into a ZIP file.
2. Sending to Cloud through MQTT: The compressed files are sent to the cloud using MQTT.

The database builder consists of three different and independent processes to build database records for the cloud based on DDL queries and table priorities.

• On-premise:
  • Screens: proc-db-struct-hp-builder (Script: DBSyncHPStructQueryBuilder.sh)
  • PHP Script: bgprocs/procSyncDBStructureBuilder.php
  • Checks for any database DDL changes every 120 seconds.
  • If any change is found, it creates an SQL file named <timestamp>_S_H_<Contract ID>_compare.sql under the directory dbdiff/DataSyncFolder/.
  • The file is then compressed as a ZIP file and stored in dbdiff/DataSyncFolder/ZipFile/.

• On-premise:
  • Screens: proc-db-data-hp-builder (Script: DBSyncHPDataQueryBuilder.sh)
  • PHP Script: bgprocs/procSyncDBHDataBuilder.php
  • Checks for any database data changes every 60 seconds.
  • If any change is found, it creates an SQL file named <timestamp>_D_H_<Contract ID>_compare.sql under the directory dbdiff/DataSyncFolder/.
  • The file is then compressed as a ZIP file and stored in dbdiff/DataSyncFolder/ZipFile/.

• On-premise:
  • Screens: proc-db-data-lp-builder (Script: DBSyncLPDataQueryBuilder.sh)
  • PHP Script: bgprocs/procSyncDBLDataBuilder.php
  • Checks for any database DDL changes every 300 seconds.
  • If any change is found, it creates an SQL file named <timestamp>_D_L_<Contract ID>_compare.sql under the directory dbdiff/DataSyncFolder/.
  • The file is then compressed as a ZIP file and stored in dbdiff/DataSyncFolder/ZipFile/.

The database sender consists of two different and independent processes to send database records to the cloud based on priority.

• On-premise:

  • JSON structure:

    {
        "c":"Sync",
        "a":"setQueryFile",
        "cid": <Contract ID>,
        "iid": <instance ID>,
        "ips": <Instance Password>,
        "content": <zip file content>,
        "fn": <zipfile_name>,
        "ta": "send_sync_query_high"
    }
    

  • Screens: proc-db-hp-sender (Script: DBSyncHPQueriesSender.sh)
  • Python Scripts: inx_aws_iot/sync_high_sender.py, inx_aws_iot/publishdbfile.py
  • Sends zipped structure queries and high-priority data files to the cloud.

• In Cloud InspeXtor:

  • Processes received packets from the topic inx/<Contract ID>/send_sync_query_high.
  • Sends packets to the PHP processor bgprocs/procSyncDBStructureProcessor.php.
  • Unzips the file, and runs SQL queries in the cloud.

• On-premise:

  • JSON structure:

    {
        "c":"Sync",
        "a":"setQueryFile",
        "cid": <Contract ID>,
        "iid": <instance ID>,
        "ips": <Instance Password>,
        "content": <zip file content>,
        "fn": <zipfile_name>,
        "ta": "send_sync_query_low"
    }
    

  • Screens: proc-db-lp-sender (Script: DBSyncLPQueriesSender.sh)
  • Python Scripts: inx_aws_iot/sync_low_sender.py, inx_aws_iot/publishdbfile.py
  • Sends zipped structure queries and high-priority data files to the cloud.

• In Cloud InspeXtor:

  • Processes received packets from the topic inx/<Contract ID>/send_sync_query_low.
  • Sends packets to the PHP processor bgprocs/procSyncDBStructureProcessor.php.
  • Unzips the file, and runs SQL queries in the cloud.

Users can manipulate devices, schedule manipulations, other system settings, and send commands from the Cloud UI to the On-premise in InspeXtor.

• Send commands and device and schedule manipulations from UI to Cloud backend.
• Cloud prepares a packet with the corresponding Contract ID.
• The packet is sent from the PHP process to the Python processor InstanceController.php (Function: InstanceController::send_packet_to_python()).
• The Python web socket 127.0.0.1:9999 accepts and processes the packet as threaded (Script: inx_aws_iot/inx_socket.py).
• Python connects to the AWS IoT Core through MQTT protocol and sends a publish to the AWS topic inx/<Contract ID of On-Premise>/action (if the connection fails, it retries 3 times and returns status success or failed) - (Scripts: inx_aws_iot/inx_socket.py and inx_aws_iot/publish.py).

• Accepts packets from AWS IoT Core with the topic inx/<Contract ID of On-Premise>/action (Script: inx_aws_iot/pahosub.py).
• Sends the packet to the PHP processor bgprocs/processCloudRequest.php in inx_aws_iot/pahosub.py (Function: send_pkt_to_php()).
• After processing the result data with database changes if exists, returns to On-premise through the AWS topic inx/instance/return.

• Processes received packets from the topic inx/instance/return.
• Sends the packet to the PHP processor bgprocs/processInstanceReturn.php.
• Runs SQL queries in the cloud if they exist.

Users can manipulate devices, schedule manipulations, other system settings, and send commands from the Mobile Application to On-premise in InspeXtor. The mobile app connects directly to Cloud InspeXtor.

• Send commands and device and schedule manipulations from the Mobile Application to the Cloud backend.
• Cloud prepares a packet with the corresponding Contract ID.
• The packet is sent from the PHP process to the Python processor.
  • Source: InstanceController.php -> InstanceController::send_packet_to_python().
• Sent to a Python web socket 127.0.0.1:9999.
  • The Python web socket 127.0.0.1:9999 accepts and processes the packet as threaded (inx_aws_iot/inx_socket.py).
• Python connects to the AWS IoT Core through the MQTT protocol and sends a publish to the AWS topic inx/<Contract ID of On-Premise>/action.
  • If the connection fails, it retries 3 times and returns the status (success or failed) (inx_aws_iot/inx_socket.py and inx_aws_iot/publish.py).

• Accepts the packet from AWS IoT Core with the topic inx/<Contract ID of On-Premise>/action (inx_aws_iot/pahosub.py).
• Sends the packet to the PHP processor bgprocs/processCloudRequest.php in inx_aws_iot/pahosub.py (function: send_pkt_to_php()).
• After processing the result data with database changes (if exists), returns to On-premise through the AWS topic inx/instance/return.

• Processes the received packet from the topic inx/instance/return.
• Sends the packet to the PHP processor bgprocs/processInstanceReturn.php.
• Runs SQL queries in the cloud if they exist.

InspeXtor operates through two distinct components: On-premise and Cloud.

An On-premise InspeXtor instance is a unit of the system that resides at the client's location, connecting various devices and sensors.

Each On-premise instance connects to a common cloud computer known as Cloud InspeXtor, responsible for managing instance subscriptions and expirations.

Communication between On-premise and Cloud InspeXtor instances is vital for system functionality. This communication occurs through various channels, including:

1. Registering and Deregistering an On-premise Instance
2. Ping to Cloud
3. Database Synchronization from On-premise to Cloud
4. Send Control Message from Cloud to On-premise
5. Send Control Message from Mobile App to On-premise

Each communication type is designed to serve specific functionalities within the InspeXtor ecosystem, ensuring seamless operation and data synchronization between On-premise and Cloud instances.

By effectively managing these communication channels, InspeXtor provides users with a reliable and responsive system for controlling devices, managing schedules, and maintaining system settings across different platforms and environments.