Home - Paulmulonzia/Dspace-test GitHub Wiki

DSpace Deployment in AWS

Weekly Status Report

For week ending 20/10/2017

Milestone	Planned Date	Actual Date	Comments
Category 6 and 7
Task1:Components & features isolation PoCs and Pilot testing	16-Oct-17	20-Oct-17	In Progress
Task 2:Asynchronous component interaction experimentation & tests	16-Oct-17	20-Oct-17	In Progress

Week’s Plan
Category 6 and Category 7

Category 6's Summary
Components & features isolation PoCs and Pilot testing.

Tasks
Task 1: Testing using jmeter on App & EFS (continuation)
Task 2: Cloudwatch/Grafana Monitoring/Datadog (continuation)

Accomplishment

Ongoing Test on Java WebApp & EFS performance.

Category 7's Summary

Asynchronous component interaction experimentation & tests.

In-progress - import of data to DSpace accounts.

Accomplishment

(i) Component Interaction. Different components Apache, webapp (java), Dspace(accounts) storage on both EFS & Postgres DB interact seamlessly.

(ii) Dummy Data Generation: Used the DSpace "dummy data" script (dsogen) to generate sample production data. We generated 700,000 PDFs each with a minimum of 10 pages and maximum of 15 pages. The total size of the data is approximately 56 GB.

(iii) Monitoring/Tests. We have actively tested using jmeter simulating between 300 and 500 users on all DSpace accounts concurrently for approximately 30minutes initially and monitoring using AWS CloudWatch, Grafana and Datadog. Tests conducted will give us an accurate assessment of a DSpace production environment workloads.The import progress as as the time of this report was as follows:
- dspace.dddke.net - 249028 PDFs (10 - 15 pages) - (Approximately 19.2 GB)
- dspace1.dddke.net - 128991 PDFs (10 - 15 pages) - (Approximately 10.3 GB)
- dspace2.dddke.net - 38532 PDFs (10 - 15 pages) - (Approximately 2.7 GB)

Issues
1. Generation and import of huge amounts of dummy data takes considerably lot of time {upload -appx. for 20GB longer that expected}.
2. PostgresDB CPU maxing-out for the period of testing (t2.large RDS instance)
3. EFS Burstcreditbalance significantly reduces during the period of test, over a few next tests will undertake longer periods with all dspace accounts

Plans for next week
Complete Category 6 and Category 7
Start Category 8 and Category 9

Testing & Monitoring
For week ending 20/10/2017

1. EFS MONITORING METRICS

Before, During and After Testing

(i) During the test, EFS TotalIOBytes stood at maximum of 3BilBytes, this is from 615MilByte before the test. </>Throughput: 3,000,000,000 MilBytes/ (5601024*1024 Bytes) =9.5MiB/s File system size (in GB): Dividing the average throughput number (in KB/s) by the baseline throughput number (50 KB/s/GiB) Apprx. {(1,627KiB/s)/(50 KB/s/GiB)} = 195GiB
(ii) The BurstCreditBalance does not change given the same period data upload it remain at 2.3 TB but dropped upto 2.2 TB immediatley after the test
(iii) At this point the number of connections that were hitting the EFS increased from 1 to 3 with datawriteIO hitting a max of 2.258 Bil

![alt text](https://INSERT LINK")

2. Postgres Metrics Before, During and After Testing
Analysis

![alt text](https://INSERT LINK")

CPU UTILIZATION
Before test (12:35- 12:41) = 3%
During test (12:42-13:15 hrs) = 97.6%
After test (13:16….. hrs) = went back to normal (3%).
Memory Utilization (Total 5.6 GiB) Before test (12:35- 12:41) memory available was = 4.825 GiB
During test (12:42-13:15 hrs) memory available was = 4.328 GiB Used Memory was = 0.5 GiB

Throughput
Write:
Before test = 138 kBps
Read
Before Test = 6 kBps
During test (12:42-13:15 hrs) Average =8 kBps

3. CloudWatch Monitoring - Memory Utilization
Analysis
Before, During and After Testing
Summary
Initial memory utilization before test noted to be 65% for single (1 instance)
During test memory increases between 70 to 75 %, one additional instance is registered.
At 75% to 80% another instance was added. In total three (3) instances launched
Scaled down to one(1) instance when memory <68% for a period of 5 minutes

![alt text](https://INSERT LINK")