OpenMigrate Metrics - tsgrp/OpenMigrate GitHub Wiki

#Overview: To help locate performance bottlenecks and deduce the optimal number of threads, additional log messages will be added into key places in OM to denote both the time every listener took to execute as well as the total time for source and target listeners, respectively. Rather than accomplishing this by manually typing out log messages and timers at the beginning and end of each method, this will make use of aspect oriented programming.

###AOP Bean Config:

1. Open up App-ctx.xml
2. At the top of the App-ctx.xml comment out <!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN 2.0//EN" "http://www.springframework.org/dtd/spring-beans-2.0.dtd">
3. Add the following at the top of the file <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.5.xsd">
4. At the bottom of the App-ctx.xml add the following
5. Add the "bean id" and point it to the UseLoggerAnnotation class which contains all of the log statements.
6. Add the "aop:config" tags to specify that it is using AOP
7. Add the "aop:aspect" tags pointing at the UseLoggerAnnotation
8. Add "aop:pointcut" to point at specific packages
9. Add "aop:around pointcut-ref" to point at specific methods

    <bean id="loggerAnnotations" class="com.tsgrp.migration.annot.UseLoggerAnnotation"/>
      <aop:config>
         <aop:aspect ref="loggerAnnotations">
             <aop:pointcut id="sourceListeners" expression="execution(* com.tsgrp.migration.source.*.*(..))" />
           	<aop:around pointcut-ref="sourceListeners" method="logMethod" />
         </aop:aspect>
      </aop:config>

##Files of Note:

1. UseLoggerAnnotation.java

• This is where the logging takes place.

1. AbstractMigrationSource.java and AbstractMigrationTarget.java

• Contain all of the summation and timing logic. There are static loggers within these methods to catch the total time of all the listeners as well as the individual listener times. Both AbstractMigrationSource and AbstractMigrationTarget are Multi-thread safe.

##ELK Setup: For Setting up ELK please refer to the OC Enhanced Logging Github page. ###Additional Information:

1. The json and conf files are located in http://svn.tsgrp.com/repos/OpenMigrate/trunk/metrics
2. In order to print stuff to the console while testing instead of hitting elastic search add output{ stdout{ codec => rubydebug } at the bottom of the sample-om-logs.conf file.
3. To create custom patterns within logstash create a new folder called 'patterns' in the logstash root directory and within this folder add a file called "customPatterns".

• Within the "customPatterns" file add custom patterns for logstash.

1. To filter these custom patterns go to the "sample-om-logs.conf" and add regex pattern to the "match=>" section

• First block catches the summations
• Second block catches the individual timings
• Third block catches the migration log statistics
• Fourth block is a generic log message

1. Logstash works for multiple lines via the grok patterns.

• The multiline codec will collapse multiline messages and merge them into a single event resulting in readable output.
• The Migration Statistics are parsed in these grok patterns as well to show 'Migrated Count', 'Ignored Count', 'Failed Count', 'Aborted Count', and 'Items left in log at close'.

1. Kibana Graphs

• Average Time for End Import Node Listener (Bar graph)
• Average Thread time (Bar graph)
• Failed Migrations, Migrated Count, Ignored Migration Count, Aborted Count, Closed Count (Number)
  • These are all summations
• Method Fires (Bar graph)
• Max Class Time Execution (Bar graph)
• Starting Migration Count (Bar graph)
• Average Method Run time (Bar graph)
• Number of Migrations (Bar graph)
• Average Execution Time (Area graph)
• Standard Deviation of Method Times (Line graph)
• Class hits (Bar graph)
• Deviations of Classtime executions (Line graph)
• Max time of Methods (Bar graph)
• Average Execution Time (Bar graph)
• Percentage of Source Execution Time (Area graph)