The most important stretch of the project as it wouldnt be smart to leave alot in the fourth sprint in case anything goes wrong. Theres a decent baseline for the front end and backend, all that needs to be done is some additions, adding a sort of rating/scale based on the results and refinement of the front end that was completed in sprint 2.

I have a front end developed but it just needs some additions + one or two last additions to the front end and spring 4 should be okay.

• 2 apis are completed and have been added to the front end
• add at least a total of 3 apis
• Test limits a bit further
• Start to host the tool

References:

• https://flask.palletsprojects.com/en/3.0.x/tutorial/factory/

• https://www.digitalocean.com/community/tutorials/how-to-create-your-first-web-application-using-flask-and-python-3

• https://medium.com/techcrush/how-to-render-html-file-in-flask-3fbfb16b47f6

• https://www.digitalocean.com/community/tutorials/how-to-make-a-web-application-using-flask-in-python-3

• https://flask.palletsprojects.com/en/3.0.x/api/#flask.flask

• https://flask-limiter.readthedocs.io/en/stable/ ( flask limiter package/module documentation)

• Taking python scripts from individual apis and converting them into one file that combines the individual scripts: See the apps code here

    @app.route('/', methods=['GET', 'POST'])  #routes the below code to the index.html file, essentially how the search bar functions on the homepage
    @limiter.limit("3/minute")  # Rate limit: max 3 requests per minute 
    @app.route('/contact')    # Routes to the individual pages, there's also one for the about page

• The functions work best with just a simple domain, for example:

https://domain.com/directory/directory2+=bflehehife124j1riji/directory3/daakdaskjfe is going to run into errors while domain.com will work alot more smoothly for the tool. What the function does through the use of specific actions to the package "urllib.parse" is parse the SLD and TLD from the full URL given which is whatever's entered into the search bar.

Reference: https://docs.python.org/3/library/urllib.parse.html

Reference:

• (VT package documentation) https://github.com/dbrennand/virustotal-python

This has been setup for awhile now and overall functions well but is also the main reason for the limiting on the application. The current limits set by VT is 4 per min, 500 per day. My application has been tested to run 500 urls in one swoop via excel, it actually seems to run around 120 URLs at once using the following script.

Reference:

• (OTXv2 package documentation) https://github.com/AlienVault-OTX/OTX-Python-SDK

• The OTX script worked well by itself from sprint 2 and was overall easy to integrate into the application. I have the OTX script tested out with the VT script and it seems to function as well counting proper counts of pulses on URLs, its limited in the sense that it too stops when the VT script hits its limit. The limit for OTX is 4000 per day which is a great starting amount for the public app ill be building.

Reference:

• (pysafebrowsing package documentation) https://github.com/Te-k/pysafebrowsing
• https://developers.google.com/safe-browsing/v4/usage-limits#:~:text=There%20is%20no%20cost%20for,Safe%20Browsing%20API%20community%20forum.

The Google Safe Browsing API has been simple to setup with the APIs that we already have setup in the previous sprints. Google safebrowsing pretty much detects within their own database whether URLs are malicious or not, and even warns users with the big red warning screen before users visit it. Having this API works great with the tool. Having a malicious rating from google widely affects my risk score rating on the tool.

According to official google documentation there is no limit set on the safebrowsingapi, but i've seen other numbers saying a maximum of 10 per second and even a cap at 10,000. So far google safe browsing is the most scalable option.

This was a great third api to add and focuses on a more categorical rating on the domains/URLs rather than quantities. Google has a wide stretch of data/information based on how many people use google to search the internet therefore has a decent gauge of maliciousness.

1. Setting up the flask app wasnt terrible after using the documentation. HTML and CSS ive done before so really all that needed to be done was setting up the routes to the specific html pages (index, contact and about for now) as well as setting the variables of each result of the functions set inside of the search bar on the index page.

2. Index.html is essentially app.py and the main homepage where the search bar is that actually utilizes the various apis

3. about.html and contact.html are linked on the menu bar and are extra pages created with basic html and css. About page is just a little more information about the tool and contact is just to provide feedback to myself regarding the tool.

4. results.html is where the actual results are displayed, needs to be in same directory as the app and then can be linked (vt_result below is linked on app.py):

<body>
    <div id="content">
        <h2>VirusTotal Results</h2>
        <p>{{ vt_result }}</p>

app.py: return render_template('results.html', vt_result=vt_result, otx_result=otx_result, score_result=score_result, sb_result=sb_result, risk_score_percentage=risk_score_percentage, error_message=error_message)

• https://help.pythonanywhere.com/pages/Flask/
• https://www.pythonanywhere.com/details/flask_hosting

I've run into some issues but this was a last issue part of this sprint, this would obviously be a main task for sprint #4.

• https://www.pythonanywhere.com/pricing/

• Essentially of creating a variable and setting the initial value to 0, if the Virus total result was above 0 then +25 is added to rating score, if OTX has more than 0 pulses than another 25 is added to the rating score. A malicious categorization from google safe browsing results in a +50 to the rating score.

• On the about page ill be adding an explanation of how the rating score works so the user has complete transparency

• A personalized message based on what the rating score is will be displayed:

        try:  # based on the results of the risk score display a custom rating//message
            if risk_score_percentage >= 50:
                score_result = f"This URL is seen as a malicious threat // Please use maximum caution"
            elif risk_score_percentage >= 25:
                score_result = f"This URL is seen as a possible risk, based off the individual results and your own research continue "
            else:
                score_result = f"We have not detected anything but please always continue with caution when using the internet :)"
        except Exception as e:
            print(f"An error occurred: {e}")
            score_result = f"An error occurred when calculating risk score: {e}"

So far the only thing I really need is to get this thing hosted, refine the front end aswell as add atleasat 1 more api to the backend. I think in regards to sprint 3 I did some large tasks that really have brought this thing together.

1. Google Safe browsing api
2. Create the application via flask aswell as create routes to html pages for the front end
3. Integration of VT and OTX scripts into the application
4. Domain parse function to allow application to run more smoothly.
5. Created a limiter that sets the application to only be used 3 times per minute (still needs work in sprint 4)
6. Created a rating system out of 100 based on the results
7. Created a custom message/final recommendation based on the rating results of the tool.