MileStone 3 - SCG-TeamNpComplete/Storm-Clustering GitHub Wiki

#Storm Cluster Milestone 3

Storm Cluster URL:

Production Server: http://ec2-35-161-48-143.us-west-2.compute.amazonaws.com:11000/SG_MICROSERVICE_SERVICEGATEWAY/gateway/servicegateway/stormcluster

Upon hitting the Storm Cluster's end point, XML message will displayed as output. Storm Cluster invokes the Forecast Detector.(https://github.com/SCG-TeamNpComplete/Forecast-Detector)

##Docker

Storm Clustering Dockerhub Link: https://hub.docker.com/r/kedargn/stormcluster/

For milestone 2, Storm Cluster is running inside Docker container.

  • Dockerfile contains the instructions for docker image which can be accessed through service gateway.
  • Travis CI build the docker image according the Dockerfile and pushes it to Dockerhub.
  • Travis CI invokes Amazon Code Deploy. Amazon Code Deploy in turn invokes scripts/install.sh & scripts/before_install.sh script.
  • scripts/before_install.sh kills the removes Storm Cluster's docker image and container, if it exists.
  • scripts/install.sh script pulls the docker image from Dockerhub and runs it inside docker container in Amazon EC2 instance.

##Load Balancing

  • The distributed zookeeper application helps load balance the microservices. The microservices registers themselves to the Zookeeper Server during startup. For every request to microservices, the Service Gateway then contacts the zookeeper server and fetches all registered instances of the requested microservice. It then picks an available instance in a round robin fashion.

##Fault Tolerance

  • Zookeeper facilitates fault tolerance. We have deploy ZooKeeper in a cluster/ensemble of two ec2 instances. So even if one ec2 instance crash, the other will take care of the incoming requests.

Instances and Microservices

  • We have 3 ec2 instances:

    ec2-35-161-48-143.us-west-2.compute.amazonaws.com

    ec2-35-160-137-157.us-west-2.compute.amazonaws.com

    ec2-52-15-90-97.us-east-2.compute.amazonaws.com

  • Zookeeper server is running on each ec2 instance and they form an ensemble.

  • There are 5 microservices running inside docker on each ec2 instance: Dataingester, StormDetector, StormCluster, ForecastDetector and Forecast

  • ServiceGateway is run as microservice on ec2-35-161-48-143.us-west-2.compute.amazonaws.com which is a high availability instance. This is also a single point of failure if this ec2 instance goes down.

Testing Fault Tolerance

  • System is tolerant to failure of both e2c instance and microservice running inside docker in ec2.

    Failure of Service in Docker: The Service Gateway contacts zookeeper server running in ec2 to find a running instance of requested microservice. Service Gateway picks ec2 instance in a Round Robin fashion. Request will be successfully addressed as long as at least one instance is running on any of the ec2.

    Failure of EC2: As the zookeeper servers are replicated over a set of 3 ec2 instances, as long as a majority of the servers are available, the ZooKeeper service will be available.

##Capacity Test

  • We used Apache JMeter to run the capacity Tests. We tried to overwhelm the system by running multiple threads in burst mode. The test results with thread count, error percentage and throughput can be found below:

TestResult