SMART methodology

1. Specific : Specific questions are simple, significant and focused on a single topic or a few closely related ideas.
   • "Are kids getting enough physical activities these days?"
   • "What percentage of kids achieve the recommended 60 minutes of physical activity at least five days a week?"
2. Measurable : Measurable questions can be quantified and assessed.
   • "Why did a recent video go viral?"
   • "How many times was our video shared on social channels the first week it was posted?"
3. Action-oriented : Action-oriented questions encourage change.
   • "How can we get customers to recycle our product packaging?"
   • "What design features will make our packaging easier to recycle?"
4. Relevant : Relevant questions matter, are important and have significance to the problem you're trying to solve.
   • "Why does it matter that Pine Barrens tree frogs started disappearing?"
   • "What environmental factors changed in Durham, North Carolina between 1983 in 2004 that could cause Pine Barrens tree frogs to disappear from the Sandhills Regions?"
5. Time-bound : Time-bound questions specify the time to be studied.
   • "What environmental factors changed in Durham, North Carolina, between 1983 and 2004 that could cause Pine Barrens tree frogs to disappear from the Sandhills Regions?"

Question. Questions leading to answers that can be quantified and assessed align with which component of the SMART methodology?

• Tangible
• Measurable
• Reasonable
• Appropriate

Correct. Questions leading to answers that can be quantified and assessed align with the measurable component of the SMART methodology.

Question. While considering a research question, a data analyst follows the SMART methodology. They limit their analysis to include data from July 2012 to August 2012. What component of the SMART framework describes this decision?

• Thoughtful
• Topical
• Time-bound
• Targeted

Correct. Limiting analysis to a certain time period describes time-bound questions. They help limit the range of analysis possibilities and enable data analysts to focus on the most relevant data.

Fairness : ensuring that your questions don't create or reinforce bias / crafting questions that make sense to everyone

---

More about SMART questions

Companies in lots of industries today are dealing with rapid change and rising uncertainty. Even well-established businesses are under pressure to keep up with what is new and figure out what is next. To do that, they need to ask questions. Asking the right questions can help spark the innovative ideas that so many businesses are hungry for these days.

The same goes for data analytics. No matter how much information you have or how advanced your tools are, your data won’t tell you much if you don’t start with the right questions. Think of it like a detective with tons of evidence who doesn’t ask a key suspect about it.  Coming up, you will learn more about how to ask highly effective questions, along with certain practices you want to avoid.

Highly effective questions are SMART questions:

• S-pecific: Is the question specific? Does it address the problem? Does it have context? Will it uncover a lot of the information you need?
• M-easurable: Will the question give you answers that you can measure?
• A-ction oriented: Will the answers provide information that helps you devise some type of action plan?
• R-elevant: Is the question about the particular problem you are trying to solve?
• T-ime bound: Will the answers allow you to solve the problem sooner rather than later?

Examples of SMART questions

Here's an example that breaks down the thought process of turning a problem question into one or more SMART questions using the SMART method: What features do people look for when buying a new car?

• Specific: Does the question focus on a particular car feature?
• Measurable: Does the question include a feature rating system?
• Action-oriented: Does the question influence creation of different or new feature packages?
• Relevant: Does the question identify which features make or break a potential car purchase?
• Time-bound: Does the question validate data on the most popular features from the last three years?

Questions should be open-ended. This is the best way to get responses that will help you accurately qualify or disqualify potential solutions to your specific problem. So, based on the thought process, possible SMART questions might be:

• On a scale of 1-10 (with 10 being the most important) how important is your car having four-wheel drive?
• What are the top five features you would like to see in a car package?
• What features, if included with four-wheel drive, would make you more inclined to buy the car?
• How much more would you pay for a car with four-wheel drive?
• Has four-wheel drive become more or less popular in the last three years?

Things to avoid when asking questions

Leading questions: questions that only have a particular response

• Example: This product is too expensive, isn’t it?

This is a leading question because it suggests an answer as part of the question. A better question might be, “What is your opinion of this product?” There are tons of answers to that question, and they could include information about usability, features, accessories, color, reliability, and popularity, on top of price. Now, if your problem is actually focused on pricing, you could ask a question like “What price (or price range) would make you consider purchasing this product?” This question would provide a lot of different measurable responses.

Closed-ended questions: questions that ask for a one-word or brief response only

• Example: Were you satisfied with the customer trial?

This is a closed-ended question because it doesn’t encourage people to expand on their answer. It is really easy for them to give one-word responses that aren’t very informative. A better question might be, “Tell me about the customer experiences from the trial.” This encourages people to provide more detail besides “It went well.”

Vague questions: questions that aren’t specific or don’t provide context

• Example: Does the tool work for you?

This question is too vague because there is no context. Is it about comparing the new tool to the one it replaces? We just don’t know. A better question might be, “When it comes to data entry, how much time does the tool save you?” This question gives context (data entry) and helps frame responses that are measurable (time).

Self-Reflection: Data analyst scenarios

Question 1

Consider the following scenario. You are three weeks into your new job as a junior data analyst. The company has just collected data on their weekend sales. Your manager asks you to perform a “deep dive” into this data. In order to get this project kicked off, this means you need to ask some questions and get some information.

Which questions are most important to you in order to get started?

You may want to jump right in and get started on the deep dive, but it is a wise choice to stop and ask questions.

Here are a few questions you might want to ask:

• When is the project due? As a data analyst, you will never have an infinite amount of time to finish a task and you will often have multiple tasks that need to be completed. Knowing the deliverable’s due date is essential to managing your time effectively.
• Is there anything specific to keep in mind?
• Who are the major stakeholders for this project, and what do they expect this project to do for them?
• Who am I presenting the results to?

These questions can help you focus on techniques and analyses that produce results of interest to stakeholders.

As a data analyst, ask questions constantly. If someone requests work on a project, you need to ask questions that align with the plan and the goals. As you explore the data, you also need to ask questions. The more questions you ask, the more you learn about your data, and the more powerful your insights will be at the end of the day.

Question 2

The type of questions you ask as you begin this "deep dive" are very important. Some common questions are:

• Objectives: What are the goals of this deep dive? What, if any, questions are expected to be answered?
• Audience: Who are the stakeholders? Who is interested or concerned about the results of this deep dive? Who will you be presenting to?
• Time: What is the time frame for completion? By what date does this need to be done?
• Resources: What resources are available to accomplish the deep dive's goals?
• Security: Who should have access to the information?

Think about how specific questions can unlock useful information in each of these areas.

How does asking detailed, specific questions benefit you when planning for a project? Can vague or unclear questions harm a project?

Asking good questions means clarifying details until you get to concrete requirements. With clear requirements and goals, it’s much easier to plan a successful data analysis project.

Good data analysts are methodical in their questions. They follow a process to get all the information they need to be successful. Unclear requirements lead to inevitable problems later in a project, which lead to time-consuming and often expensive changes. Understanding the impact of asking good questions helps data analysts remember to be thorough and specific when clarifying requirements.

Self-Reflection: SMART questions in real life

Asking real-world questions

In this activity, you are going to have a "data conversation" with someone you know, either in person, on the phone, or on a video chat. Think of someone in your life who interacts with data to make better decisions. This might be a family member who runs a small business, a friend who runs a committee for the Parent Teacher Association, or a neighbor who teaches piano lessons. All of these people turn to data in some way to be more effective in their roles.

You can let them know you're training to be a data analyst, and are hoping to have a "data conversation" with them to practice your asking skills. Hopefully, you'll end up with some useful recommendations that will benefit you both.

Step 1: Planning

Start by taking a few minutes to think about who you are going to speak with. Use the SMART question framework to make sure each question you ask makes sense based on their field. Each question should meet as many of the SMART criteria as possible. As a reminder, SMART questions are:

• Specific: Questions are simple, significant, and focused on a single topic or a few closely related ideas.
• Measurable: Questions can be quantified and assessed.
• Action****oriented: Questions encourage change.
• Relevant: Questions matter, are important, and have significance to the problem you’re trying to solve.
• Time-bound: Questions specify the time to be studied.

For instance, if you are having a conversation with someone who works in retail, you might lead with questions like:

• Do you currently use data to drive decisions in your business? If so, what kind(s) of data do you collect, and how do you use it?
• Are there business decisions or changes that you would make if you had the right information? For example, if you had information about how umbrella sales change with the weather, how would you use it?

If you are having a conversation with a teacher, you might ask different questions, such as:

• What kind of data do you use to build your lessons?
• Do you use others’ data to support the concepts students are learning?
• How do you use grading data?

Some of these questions ask about specific types of data, or about the context of the data. These questions are relevant to the role of the person you are asking, and encourage more complete and informative answers.

Step 2: Creating questions

Overall, your goal is to come up with questions to help you understand more about the data the individual usually interacts with, limitations of the data they have, and their business goals. For this step, review the following advice:

• Avoid technical jargon.
• Prioritize your questions: Ask the most important and impactful questions first to save time.
• Make your time count: Stay on subject during the conversation.
• Clarify your understanding: To avoid confusion, briefly summarizing the given answers to make sure you understood it correctly. This will go a long way in helping you avoid mistakes.

For example, in a conversation with a teacher, you might check your understanding with a statement like, “Just to double check that I understand what you’re saying correctly, you currently use grading data in the following ways…”

Depending on the field the person is in, they may not be comfortable sharing more detailed data with you, and that's ok! Be sure to respect what the person is willing to share. And if they don't share detailed data (for example, maybe a business owner won't want to share sales spreadsheets), you could always create some sample data yourself, and use that as a starting point for your analysis.

Step 3: Take good notes!

This may seem obvious, but it’s a common mistake and one easily avoided by sticking to a process of asking questions, clarifying your understanding of their response, and then briefly recording it in your notes.

Remember, if a question is worth asking, then the answer is worth recording. Challenge yourself to take great notes during your conversation. At a minimum, good notes include:

• Facts: Any concrete piece of information is usually worth writing down. Dates, times, names, and other specifics that pop up.
• Context: Facts without context are useless. Note any relevant details that are needed in order to understand the information you gather.
• Unknowns: Sometimes you may miss an important question during a conversation. Make a note when this happens so you know to figure out the answer later.

For example, if we had a conversation with an ice cream shop about collecting data on customer flavor preferences, our notes might appear something like this:

• Project: Collect customer flavor preference data.
• Overall business goal: Use data to create more popular flavors.
• Two data sources: Cash register receipts and completed customer surveys (email).
• Target completion date: Q2
• To do: Call back later and speak with the manager about the location of survey data.

The notes you decide to take are going to differ greatly based on the data conversation you have. The important thing is that your notes are specific, organized, and concise.

Test your knowledge on crafting effective questions

TOTAL POINTS 4

Question 1

A data analyst uses the SMART methodology to create a question that encourages change. This type of question can be described how?

• Action-oriented

• Stimulating

• Motivational

• Results-focused

Correct. In the SMART methodology, questions that encourage change are action-oriented.

Question 2

A time-bound SMART question specifies which of the following parameters?

• The topic or subject of the analysis

• The desired change the analysis should produce

• The metrics or measures related to the analysis

• The era, phase, or period of analysis

Correct. A time-bound SMART question specifies the era, phase, or period of analysis.

Question 3

A data analyst working for a mid-sized retailer is writing questions for a customer experience survey. One of the questions is: “Do you prefer online or in-store?” Then, they rewrite it to say: “Do you prefer shopping at our online marketplace or shopping at your local store?” Describe why this is a more effective question.

• The first question is leading, whereas the second question could have many different answers.

• The first question contains slang that might not make sense to everyone, whereas the second question is easily understandable.

• The first question is vague, whereas the second question includes important context.

• The first question is closed-ended, whereas the second question encourages the respondent to elaborate.

Correct. Vague questions do not provide context. The second question clarifies that the data analyst wants to learn exactly how and where customers prefer to shop.

Question 4

A data analyst at a social media company is creating questions for a focus group. They use common abbreviations such as PLS for “please” and LMK for “let me know.” This is fair because the participants use social media a lot and are likely to be technically savvy.

• True

• False

Correct. Fairness means asking questions that make sense to everyone. Even if a data analyst suspects people will understand abbreviations, slang, or other jargon, it’s important to write questions with simple wording.