Thursday 15/04/2020

Presentation of testPerceval.py

This little program allow you to search information in [email protected] mailbox archive

What can you do in the module :

1. Display all the messages of the archive
2. Count the mails send by a person (you will specify her name)
3. Print the 5 first messages of the archive (so you can know the structure of a message)
4. Search a keyword (this will search a word/string in the content of all the messages)
5. Build a JSON with N first messages
6. Save the mails to Elastic
7. Delete all the date in ElasticSearch
8. Print Info on how to launch ElasticSearch
9. Execute a Json query
10. Print a random thread
11. Build threads and index it
12. Stop the program

Inputs

Depends on what action you're gonna do, you will be asked to enter a name, keyword, number, mailing list

Output

All the results are printed in the console.

If you choose to build a JSON, it's save at sandbox/data/json.

If you choose to save the mails into elasticSearch, the index is mails.

How to use it

You will need to have Python 3.6 installed on your machine.

You will need to have elasticSearch on your machine, this the steps to do this :

• sudo apt-get install apt-transport-https
• wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
• add-apt-repository "deb https://artifacts.elastic.co/packages/7.x/apt stable main"
• sudo apt-get update
• sudo apt-get install elastic search

How to launch elasticsearch :

• sudo /bin/systemctl enable elasticsearch.service
• sudo systemctl start elasticsearch.service

Also, i created a python virtual environment and i recommend you to do the same.

In this GitHub, you will find in sandbox/python/testPerceval :

• classes directory : where you can find python modules
• test directory : where you can fin the main who are usings the modules
• requirements.txt

You will need to pip install -r requirements.txt.

IMPORTANT : you have to be at ../sandbox/python/testPerceval/test

Then for launching the module : python testPerceval.py

Then you will be asked to type a number corresponding of the action you wanna do. Depending to your choice, you will be asked to type another String or nothing.

If you have saved the mails into elasticsearch you will be able to do your own query on the browser starting with :

• localhost:9200/mails/_search/

Basics Use

One you've launched mainTestPerceval, you will have different choices, here i will present you to test them all First type the mailingList you want. Then for different actions : 1 : Just type 1 2 : Just type 2 - Type for example "Laurent Michel" without the quotes 3 : Just type 3 4 : Just type 4 - Type for example "nasa" without the quotes 5 : Just type 5 - Type for example 5 to build a json with 5 messages 6 : Just type 6 - This will build elastic search index 8 : Just type 8 - This will delete all elastic search indexes 9 : Just type 9 - This will only print info on elastic search 10 : Just type 10 - Type 3 to have result both printed and stored - Type temp3 to query a random message 11 : Just type 11 - This will print the thread of a random message 12 : Just type 12 - This will build the index of thread If this is your first use of the module : Please do the action 8, then 6, then 12 just to have all the indexes updated.