Home - accionlabs/acciom GitHub Wiki

Introduction

Acciom is a data quality assurance tool, efficient enough to qualify your data against various quality parameters. Quality parameters include completeness, consistency, uniqueness, validity, accuracy. Quality qualifying tests can be executed against various types of data sources like Postgres, MySQL, SQL Server, Amazon Redshift(Datawarehouse software), etc. Systems running Data Migrations, ETLs are the right audience to product.

Challenges faced for data quality qualification

  • Inaccurate analytical results
  • Time consumption
  • Cost
  • Manual testers compromise data quality by following the data sampling method.
  • Dependency on SQL query skills.

The 4 Vs of Acciom

  • Volume refers to the tremendous volume of the data.
  • Velocity means that data are being formed at an unprecedented speed and must be dealt with in a timely manner.
  • Variety indicates that big data has all kinds of data types, and this diversity divides the data into structured data and unstructured data.
  • Value that data brings in for a business.

How does Acciom help?

  • 99.9% efficiency
  • Faster results
  • Leave query building to us
  • Supports heterogeneous databases

Use Cases

New Acciom user

  • User Register by filling in the required credentials
  • If email-id already registered then throw an appropriate error
  • If registered successfully then redirects to the login page.
  • User would have role-based access to resources

Test Suite Upload

  • Acciom user prompted for credentials
  • On successful authentication, the session token is issued and the user is redirected to the file upload page.
  • User uploads excel
  • On Successful upload, the user can run the suite.

Test Suite Run

  • Acciom user prompted for credentials
  • On successful authentication, the session token is issued and the user is redirected to the home page.
  • All the suites save by the user are displayed.
  • User can run a single quality check or all checks.
  • UI displays the result of each quality parameter in different colors Test Case Pass: Blue Test Case Fail: Red Test Case waiting: Orange Test Case Error: White
  • Verify current system is capable of qualifying data for below parameters
    • Completeness
      • Count Check
      • Duplicate Check
      • Null Check
    • Accuracy
      • Field-to-Field Comparison.
    • Validity
      • DDL Check
      • Constraint Check

Architecture

Acciom Architecture Below are the main components

  • User Interface
    • User Interface for the product which can be used by testers for qualifying the data.
  • Identity Service
    • This service intends to do authorization and authentication. Issuing tokens.
  • Rest API Layer
    • Rest API layer is the abstraction to the core services and gives flexibility for 3rd party integration.
  • Core MicroServices
    • These services intend to create task queues, queues manager.
  • Workflow Services
    • Task executors, Task Schedulers
  • Logging
    • Centralized logging service.