Summary & Index:

During this week the group continued to explore the dataset. The main focus was to have an overview about how terrorism has been evolving over time. The main findings for this weeks are:

• Terrorist attack localization evolves over time.
• There is a huge number of terrorist groups and its activity is evolving over time.
• There are around 40 features with a substantial number of missing values.

Wiki Status - concluded

Index

####1 - Evolution of terrorism over time #####1.1 - Research Questions #####1.2 - Number of attacks over time #####1.3 - Geo-Referential Analysis #####1.4 - Which groups of terrorists are active between 1970 and the present? ####2 - Exploratory analysis #####2.1 - Correlation analysis #####2.2 - Action on Previous Feedback - Outliers #####2.3 - Action on Previous Feedback - Missing Values ####3 - Weekly Presentation ####4 - Perceived Feedback

Weekly work

The work for this week was separated into two different parts. The first one was to describe terrorism over time and the second one was to continue with the exploration of the data from the last week.

1 - Evolution of terrorism over time

1.1 - Research Questions

In this week the group focuses mainly on describing the terrorism in general. We were interested in discover how it has been evolving over time. In a presidential meeting we collect several research questions:

• how is the number of attacks evolving? Are they increasing or decreasing?
• how is the number of kills per attack evolving?
• how are weapons used evolving?
• how are attack types evolving?
• Are the attacks changing region over the years?
• the attacks in the different regions are changing over time? (concerning weapons, type of attack, number of victims, target, group)
• which groups of terrorists have been active between 1970 and the present?

Concerning these questions, we conducted much analysis that is presented in the next subchapters.

1.2 - Number of attacks over time

As we can see in the figure below (click the link to have a better view of the image), terrorist attacks have been increasing over the last years. However, this was not the case from 2014 to 2015.

1.3 - Geo-Referential Analysis

With the Geo-referential Analysis, our goal is to consider the interaction of different attributes and tried to visualize it in a map over some Animation/Video. Visualization is done considering different features:

• The number of incidents per year and how they are fluctuating.
• Number of deaths that happened in these incidents.
• Number of incidents happened per region, type of attacks (Assassination, Bombing, Kidnapping...) terrorists use.
• Number of incidents relating to the attack type.

Number of Incidents since 1970

In order to understand the evolution of terrorist attacks since 1970, we conceived a video. In this Youtube Link it is possible to see a clear analysis. In the figure below there is a frame of the video. Each yellow dot in the map corresponds to a terrorist attack.

As we can see from the video, there has been a sudden increase in the number of incidents from the mid-1980s and then a decrease from mid-1997 to 2005, after that the incidents increases dramatically. We can see some tendencies more in detail:

• Terrorism has been a constant in North America, Central Europe, and the Philippines.
• From 1980 to 2000 terrorist attacks occurred in Latin America countries (like Chile or Peru). But after this period we can see that it is still active until the present in Colombia.
• Also in South America, we can see some significant activities from 1970 and become rare after 2000.
• In 1977 terrorist attacks become more frequent in the Middle East.
• In 1988 in India.
• In 1978 it starts to appear in the Southern part of Africa, becoming more general all over the continent in 1991, and significantly diminishing in 2004. From 2006 to the present it becomes more common in Central Africa.
• Finally, in Australia terrorist attacks are quite rare.

Number of Deaths over the years

In order to understand the evolution os deaths in terrorist attacks since 1970, we conceived a video. In this Youtube Link it is possible to see a clear analysis. In the figure below there is a frame of the video. Each dot in the map corresponds to a terrorist attack.

As we can see from the video we can conclude that:

• Though incidents decreased from 1997 to 2005, the incidents happened during this time have more Red circles, therefore this means that the number of deaths was huge in these events.
• From 2003 to the present the incidents with more victims (blue and red circles) appear mainly in the central part of the globe.

Number of Incidents per Region

In order to understand the number of incidents per region, we conceived one more video. In this Youtube Link it is possible to see a clear analysis. In the figure below there is a frame of the video. Each dot in the map corresponds to a terrorist attack.

The regions which are adversely impacted are 'Middle east', 'North America', 'South America', 'South and South-east Asia', 'Africa' and 'Western Europe'.

Type of Attacks evolving since 1970

In order to understand how the types of attacks are evolving since 1970 per region, we conceived one more video. In this Youtube Link it is possible to see a clear analysis. In the figure below there is a frame of the video. Each dot in the map corresponds to a terrorist attack. And the colors correspond to different attack types.

No particular pattern had appeared from this video. However, it could be interesting to check in the next weeks if the type of attacks correlates with the terrorist group.

Number of Incidents per Attack Type

In order to understand the number of incidents per Attack Type per region, we conceived one more video. In this Youtube Link it is possible to see a clear analysis. In the figure below there is a frame of the video. From the above two videos regarding attack types, we can see that major type of attacks used by terrorists is 'Armed Assault', 'Assassination', 'Bombing/Explosion', 'Infrastructure Attack' and 'Kidnapping'.

1.4 - Which groups of terrorists are active between 1970 and the present?

In order to understand how terrorist attacks evolved over the years, we decided to analyze the active periods of terrorist groups in several regions. We define here the active period of a terrorist group as the time period between the first recorded attack and the last in the terrorism database.

It's important to notice that 44% of the terrorist attacks do not have information regarding the group/individuals behind it, with the field being filled with the keywords "Unknown" or "Other". Also, 3% of the terrorist attacks are not associated with an organized terrorist group ("Unaffiliated Individual(s)"). Given the scope of this analysis, both these situations were not considered in this chapter.

For all of the plots present in this chapter, the top 30 terrorist organizations with bigger active periods were considered. Information regarding the number of attacks was also added in the plot (next to the organization's name).

1.4.1. Global data

Considering all the attacks around the globe, the top 30 terrorist organizations activity is presented in the figure below.

From the figure, it is possible to conclude that the groups with the bigger active periods are still active in the present (with their last attack happening between 2014 and 2015). It's also noticeable that most of them started operating in the 1970s.

The organizations with a larger number of attacks are:

• Shining Path (a communist militant group in Peru).
• Basque Fatherland and Freedom (an armed Basque nationalist and separatist organization in northern Spain and southwestern France, also known as ETA).
• Irish Republican Army (armed movements who believe in the independence of Ireland).
• Voluntary Armed Forces of Colombia (guerrilla movement involved in the continuing Colombian armed conflict).
• New People's Army ( the armed wing of the Communist Party of the Philippines).

1.4.2. Australasia & Oceania

In the figure below it is possible to see the top 30 terrorist organizations with more attacks in Australasia & Oceania.

In Australasia and Oceania, both the number of terrorist organizations and the number of attacks are very low when compared with the other regions of the globe. All of these attacks are also very recent (the older one happening in 1991)

The organizations with a larger number of attacks are:

• Bougainville Revolutionary Army (armed organization seeking independence from Papua New Guinea).

1.4.3. East Asia

In the figure below it is possible to see the top 30 terrorist organizations with more attacks in East Asia.

Almost all of the terrorist organizations operating on East Asia have small active periods, except for 3 (which also conducted the largest number of attacks):

• Uighur Separatists
• Chūkaku-ha (a Japanese far-left revolutionary group).
• Kakurōkyō (a leftist group in Japan).

1.4.4. Central Asia

In the figure below it is possible to see the top 30 terrorist organizations with more attacks in Central Asia.

Almost all of the terrorist organizations operating in Central Asia have small active periods and appeared recently (the 1990s). The organization with a larger active period (with only two attacks registered) is the Abkhazian Separatists group.

1.4.5. South Asia

In the figure below it is possible to see the top 30 terrorist organizations with more attacks in South Asia.

Most of the terrorist's organizations operating in South Asia are fairly recent (first attacks in the 1990s). It's also noticeable the very large number of attacks by the Taliban between 1995-2005.

1.4.6. Southeast Asia

In the figure below it is possible to see the top 30 terrorist organizations with more attacks in Southeast Asia.

Terrorist organizations in Southeast Asia usually have a reduced active period when compared to the other regions (although with a few exceptions).

The organizations with a larger number of attacks are:

• New People's Army ( the armed wing of the Communist Party of the Philippines).
• Abu Sayyaf Group ( the jihadist group from the southwestern part of the Philippines).

1.4.7. North America

In the figure below it is possible to see the top 30 terrorist organizations with more attacks in North America.

Terrorist organizations in North America seem more frequently associated with racist motivations (Ku Klux Klan, White Extremists, Neo-Nazi Group, Black liberation Army). Almost all of the organizations were extinct by 2000.

1.4.8. Central America & Caribbean

In the figure below it is possible to see the top 30 terrorist organizations with more attacks in Central America & Caribbean.

Most of the terrorist activity in Central America was between 1978 and 1997, with only two groups being active after 2000.

The organizations with a larger number of attacks are:

• Frente Farabundo Martí para la Liberación Nacional (was one of the two major political parties in El Salvador).

1.4.9. South America

In the figure below it is possible to see the top 30 terrorist organizations with more attacks in South America.

Terrorist organizations in South America usually have a reduced active period, when compared to the other regions (although with a few exceptions).

The organizations with a larger number of attacks are:

• Shining Path (a communist militant group in Peru).
• National Liberation Army (group involved in the Colombian armed conflict).

1.4.10. Eastern Europe

In the figure below it is possible to see the top 30 terrorist organizations with more attacks in Eastern Europe.

The terrorism in Eastern Europe follows a different pattern from the remaining regions, as most of its terrorism activity happened after 2000.

The organizations with a larger number of attacks are:

• Chechen Rebels.
• Donetsk People's Republic.

1.4.11. Western Europe

In the figure below it is possible to see the top 30 terrorist organizations with more attacks in Western Europe.

Terrorist organizations in Western Europe are usually active for large periods, some with recent attacks.

The organizations with a larger number of attacks are:

• Basque Fatherland and Freedom (an armed Basque nationalist and separatist organization in northern Spain and southwestern France, also known as ETA).
• Irish Republican Army (armed movements who believe in the independence of Ireland).

1.4.12 Middle East & North Africa

In the figure below it is possible to see the top 30 terrorist organizations with more attacks in the Middle East & North Africa.

Terrorist organizations in the Middle East are usually active for large periods, some with recent attacks.

The organizations with a larger number of attacks are:

• Palestinian Organizations.
• Hamas.

1.4.13 Sub-Saharan Africa

In the figure below it is possible to see the top 30 terrorist organizations with more attacks in Sub-Saharan Africa.

Terrorist organizations in the Sub-Saharan Africa are usually active for large periods, most of them with recent attacks.

The organizations with a larger number of attacks are:

• Lord's Resistance Army.
• African National Congress.

2 - Exploratory analysis

In order to have more insights about the data we also continued doing some exploratory analysis.

2.1 Correlation analysis

We use Pearson correlation and respective plots to check if several variables are associated.

Number of Kills vs Attack Type

In the figure below it is possible to see the plot for the obtained values in Number of kills and Attack type, and also the Pearson correlation between both variables.

Number of kills vs Number of attackers

In the figure below it is possible to see the plot for the obtained values in Number of kills and Number of the perpetrator, and also the Pearson correlation between both variables.

Success vs Number of kills

In the figure below it is possible to see the plot for the obtained values in Success and Number of Kills, and also the Pearson correlation between both variables.

Success vs Number of perps

In the figure below it is possible to see the plot for the obtained values in Success and Number of perpetrators, and also the Pearson correlation between both variables.

In order to achieve conclusions about the correlations, it is important to extract outliers, and define which variables are the most important ones.

2.2 - Action on Previous Feedback - Outliers

From the last week, we already have discovered that there is very few events have resulted in >1000 deaths.
For this week, from the visual analysis of the correlation plots, we can see that there are outliers present in the data. This values can have a great impact on the correlation value, therefore, this should be analyzed and solved in the future.

2.3 - Action on Previous Feedback - Missing Values

Besides, we also conducted analysis about all the missing values frequencies. In the next plot we can see the percentage of missing values for each variable.

The distribution of missing percentages is presented in the next plot:

We concluded that around 40 columns are very sparse.

It is also important to point out the presence of a field called "doubters", which indicates if there is doubt about an incident being classified as terrorism. We choose that we will leave the incidents with doubt out of the analysis.

3 - Weekly Presentation

https://docs.google.com/presentation/d/1sINPJo6x7QrBDslyvvKr03Hi2idQ9oHjR7RoFqgY6bk/

4 - Perceived Feedback

4.1 - General Feedback

• Make your stories interesting.
• Explain diagrams well. So that everyone understands.

4.2 - Feedback

• Make fonts bigger.
• Maps: Attacks in populated cities. Check from the data whether you can make the correlation of the number of attacks vs the population of the city. Attacks in areas where the population is small, find the reason. Maybe because of an oil pipe/other interest.
• If possible, show the maps on a slider. JS visualization.
• Unknown - Unaffiliated. Are they updated later?
• Group Names: Muslims, Tamils, Gunmen, Rebels, Albanians ~~ They're not organizations. What to do. Hierarchies of groups should be created.