Milestone 3 - SCG-TeamNpComplete/Storm-Detector GitHub Wiki

#Storm Detector Milestone 3

##Storm Detector URL:

Production Server: http://ec2-35-161-48-143.us-west-2.compute.amazonaws.com:11000/SG_MICROSERVICE_SERVICEGATEWAY/gateway/servicegateway/stormdetector

Upon hitting the Storm Detector's end point, XML message will displayed as output. Storm Detector invokes the Storm Detector.(https://github.com/SCG-TeamNpComplete/Storm-Detector)

##Docker

Storm Detector Dockerhub Link: https://hub.docker.com/r/kedargn/stormdetector/

For milestone 3, Storm Detector is running inside Docker container and can be accessed through Service Gateway.

  • Dockerfile contains the instructions for docker image.
  • Travis CI build the docker image according the Dockerfile and pushes it to Dockerhub.
  • Travis CI invokes Amazon Code Deploy. Amazon Code Deploy in turn invokes scripts/install.sh & scripts/before_install.sh script.
  • scripts/before_install.sh kills the removes Storm Detector's docker image and container, if it exists.
  • scripts/install.sh script pulls the docker image from Dockerhub and runs it inside docker container in Amazon EC2 instance.

##Load Balancing

  • The distributed zookeeper application helps load balance the microservices. The microservices registers themselves to the Zookeeper Server during startup. For every request to microservices, the Service Gateway then contacts the zookeeper server and fetches all registered instances of the requested microservice. It then picks an available instance in a round robin fashion.

##Fault Tolerance

  • Zookeeper facilitates fault tolerance. We have deploy ZooKeeper in a cluster/ensemble of two ec2 instances. So even if one ec2 instance crash, the other will take care of the incoming requests.

Instances and Microservices

  • We have 3 ec2 instances:

    ec2-35-161-48-143.us-west-2.compute.amazonaws.com

    ec2-35-160-137-157.us-west-2.compute.amazonaws.com

    ec2-52-15-90-97.us-east-2.compute.amazonaws.com

  • Zookeeper server is running on each ec2 instance and they form an ensemble.

  • There are 5 microservices running inside docker on each ec2 instance: Dataingester, StormDetector, StormCluster, ForecastDetector and Forecast

  • ServiceGateway is run as microservice on ec2-35-161-48-143.us-west-2.compute.amazonaws.com which is a high availability instance. This is also a single point of failure if this ec2 instance goes down.

Testing Fault Tolerance

  • System is tolerant to failure of both e2c instance and microservice running inside docker in ec2.

    Failure of Service in Docker: The Service Gateway contacts zookeeper server running in ec2 to find a running instance of requested microservice. Service Gateway picks ec2 instance in a Round Robin fashion. Request will be successfully addressed as long as at least one instance is running on any of the ec2.

    Failure of EC2: As the zookeeper servers are replicated over a set of 3 ec2 instances, as long as a majority of the servers are available, the ZooKeeper service will be available.

##Capacity Test

  • We used Apache JMeter to run the capacity Tests. We tried to overwhelm the system by running multiple threads in burst mode. The test results with thread count, error percentage and throughput can be found below:

TestResult