Extract Information - urcuqui/Data-Science GitHub Wiki

Extract Information

The idea is to extract and share the information in batch to the cloud provider, in the different cases we will have three tools:

  • SDK, they are the different programming languages like Python, JavaScript, C++, Java, others.
  • CLI, they are command shells
  • Services

AWS Python

Cloud9 allows us to make environments

AWS Solutions

  • The idea is to use the API gateway in order to manage all the data providers which generate the data that are going to go to the AWS Cloud services.
  • Another AWS tool is the storage gateway, it is a solution where we can use it in our datacenter or solutions on-premise, we can implement a VM which is going to get the logs of our systems and to take them to the cloud.
  • Kinesis for data streaming solutions, some of the agents are producers, stream, shards, and consumer.