Setting Up Your Reddit Scraper - professorf/data-analytics GitHub Wiki

One Time Setup

Step 1. Register for an account on Reddit

The first step is to go to https://reddit.com and register for an account. We will assume you have done so, and not show these steps in detail. Write down your reddit username and reddit password, you will need it for the next step.

Step 2. Register your app

Next go to: https://www.reddit.com/prefs/apps and register your app. This requires first clicking on the create app button. Then

a. Give your app a name, e.g., SCK Scraper
b. Describe your app, e.g., A scraper for my research on SCK
c. Provide an about url, e.g., just use your school's url
d. Provide a redirect url, e.g., just use your school's url
e. Check the **I'm not a robot** box

After doing all that, click on the create app button. A screen will appear containing "API keys", an app's version of a username and password. The first key highlighted in yellow under the label personal use script is known as your client id. The second key, next to the label secret, is known as your client secret.

Copy and paste your client id and client secret and save it in a file. You will need this information for the code that actually scrapes your subreddit. NOTE: DO NOT USE THE CLIENT ID AND CLIENT SECRET IN THE IMAGES

Step 3. Create a .env file in your visual studio code analytics folder

Create a .env file in the folder that will hold all the connection information for your python discussion scraper (to be discussed next):

reddit_client_id     = "YOUR CLIENT ID GOES HER"
reddit_client_secret = "YOUR CLIENT SECRET GOES HERE"
reddit_user_agent    = "multiplatform:APPNAME:v1.0 (by /u/USERNAME)"

For example, if your Reddit username is "professorf", and your client id is "NTulTWRpSOT5CmmJ8FviQw" and your client secret is"8d1STfwtsItgDpfQdC0ma-YoHidZCw" and your app's name is "SCK Scraper", then your app is:

reddit_client_id     = "NTulTWRpSOT5CmmJ8FviQw"
reddit_client_secret = "8d1STfwtsItgDpfQdC0ma-YoHidZCw"
reddit_user_agent    = "multiplatform:SCK Scraper:v1.0 (by /u/professorf)"

In an editor like Visual Studio Code, this would look like:

If you are reading this as part of one of my classes, take a screenshot and save it as Lastname-Firstname-reddit.png