Analyzing many teams in the teams dashboard - theliberators/columinity.docs GitHub Wiki

This step-by-step guide explains how to analyze the results from many teams in the Teams Dashboard. A Demo Teams Dashboard is available here. The examples in this guide will use the "Agile Team Effectiveness" model. If you pick another model, the factors will be different. But the principles are the same.

Who is this guide for?

• Users that are on our "Professional"- or "Enterprise"-plans and have access to the Teams Dashboard.

⚠️ Using Columinity is a change initiative (whether you like it or not) ⚠️

Columinity provides powerful insights into patterns across many teams in your organization. This strength is also a risk. If teams sense that people outside the team are misusing the data, they will start answering the questions in a way that those people want to hear, and Columinity will lose its usefulness. Using Columinity is a change initiative. As with any change initiative, it is necessary to clarify the intent, how it helps people, and who will benefit from it. We recommend you do at least the following:

1. Before starting with Columinity, create buy-in with teams first.
2. Create work agreements about who can access the Teams Dashboard.
3. Review how we protect anonymity and privacy if your teams have questions about this.
4. Create a plan/structure for how you will embed this change initiative. Who is leading it? How often do people meet? What happens with the data?

1. Go to your Teams Dashboard

1. Go to https://teamsdashboard.columininity.com.
2. Log in with your credentials.
3. If you can access multiple Teams Dashboards, choose the relevant one.

2. Teams, snapshots, and scientific models

Teams typically and ideally run multiple surveys over time to track their improvements. Each survey is a snapshot, tied to a particular scientific model. We offer several scientific models to help teams become more effective, each focusing on a few core factors and sub-factors. Teams often benefit from multiple models, each offering its perspective on the same team. You can toggle between models with this selector in the header, which effectively filters the entire Teams Dashboard to just the snapshots and results for that model:

3. Understanding the overview

Once logged in, you are greeted with the Teams Dashboard overview:

The overview (in blue) at the top shows you essential metrics for the current period. The length of the current period defaults to the past 3 months but can be adjusted with "Settings". Remember that a more extended period will naturally include more outdated data. The following metrics are shown:

• Active teams: The number of active teams in your Teams Dashboard compared to your available team volume. If you have not reached the available volume, you can continue adding teams until reached.
• Recent snapshots: The number of snapshots started in the current period.
• Open actions: The number of improvement actions created by teams, categorized into "Open", "Done" and "Total".
• Due actions: The number of improvement actions created by teams that are due, meaning their expected completion date has passed. Click the number to see details.
• Team Help Requests: The number of improvement actions marked by teams as impeded. These are actions that teams are actively asking for help with. Click the number to see details.
• Participation rate: The average participation rate for snapshots in the current period. This is calculated for all snapshots where a "participation goal" has been set, allowing Columinity to estimate the number of completed participants against the number of expected participants (the goal).
• The bars for "Responsiveness", "Continuous Improvement" and so on show the average scores, the trend and the 15-85% scoring range on selected factors in the current period. You can change the factors shown here under "Settings".

4. Review Insights

1. Click "Insights".
2. You now see a visual with scores. The guide below will walk you through this step-by-step.
3. The top bar tells you which benchmark is selected ("Most teams in our database").
4. The top bar also tells you what comparison mode is used and the associated color-coded legend.
5. For this example, the results in "Insights" are based on two samples: snapshots from teams in the current period compared to snapshots from teams in the previous period. Periods are typically three months, counting back from today. To review the periods and the teams whose snapshots are included, please feel free to hover over the i-icons in the top bar.
6. The settings discussed in the previous steps can be changed under "Settings". Please look over our guide on statistical settings to learn more. Additionally, review this guide on how to do this.

Now, let's begin making sense of what everything else means.

5. Start with Team Effectiveness

With Columinity, we want to help your team become more effective. Teams are effective when they can satisfy their stakeholders through a work process that is also enjoyable to team members. Team Effectiveness is the starting point for any analysis in Columinity. All the other factors we measure contribute to team effectiveness in one form or another, according to scientific studies.

1. Locate the factor "Team Effectiveness" (in blue) to review the average score of your teams (52).
2. Review the sub-factors that compose team effectiveness, in this case, "Team Morale" (55), "Stakeholder Satisfaction" (52), and "Stakeholders: Team Value" (55). Our algorithm combines the scores of these sub-factors into a score for team effectiveness. Consult this guide for more detail.

What it means

• The score for Team Effectiveness (52) reflects the (weighted) average of snapshots from included teams. Consult this guide to understand in detail how this aggregation works.
• The green bar behind for Team Effectiveness reflects the 15-85% percentile to indicate the range of scores. A wide range indicates disagreement and/or different perspectives, meaning the score must be interpreted cautiously. If the range is wide, a warning message will be shown as a "limitation" in the dialog that opens when you click a factor.
• The bar for Team Effectiveness is green because the results improved from the previous period (39) by 13 points to the current period (52). Congratulations—teams are becoming more effective! The bar would've been yellow if the results hadn't improved or red when they worsened. This results from the "Comparison Mode" set under "Settings", which is currently set to "Previous period". It can also be set to compare against a selected benchmark, in which case the colors indicate the difference with a benchmark.
• The dots below the green bar for Team Effectiveness indicate the freshness of the data. Each dot represents one month, so this data is several months old. The older the data, the less reliable it is for making informed decisions about what to improve now.
• Click on the blue circle of Team Effectiveness to get a dialog with details. It includes a definition of the factor, a breakdown of scores by benchmark, teams, and segments, and actionable tips for improving. If there are issues with the data that complicate interpretation, they are shown in the yellow area. In this case, there is one limitation.

6. Review core factors that contribute to Team Effectiveness

Team Effectiveness is the outcome we're after with Columinity. But many factors contribute to it. The scientific model you picked when you set up the survey determines which factors are surveyed and analyzed. Each model identifies a handful of core factors that research has shown to be relevant, with each core factor typically having several sub-factors.

With this in mind, let's review the high-level results. The examples will use the "Agile Team Effectiveness" model. If you pick another model, the factors will be different. But the principles are the same.

What it means

• Team Effectiveness is shown on the right, in the blue circle. Any analysis should start here and work back through the core factors (circles) and their sub-factors (squares).
• Each factor shows relevant improvement actions for this factor created by the team. We will talk about this in more detail below. If any actions are marked as impeded by the team, their number is shown in the orange circle (). The number of open and completed actions for a factor is shown in the dark circle (). Click on either circle to go directly to the relevant actions.
• Some factors are measured only among team members (like "Refinement", "Team Morale" or "Sprint Goals"), whereas others are measured only among stakeholders (like "Stakeholders: Team Value"). Other factors are measured across multiple segments of participants, like "Support for Team Autonomy". Consult the breakdown of scores by clicking a factor to understand which groups of participants provided their perspective on it.

ℹ️ Tip: Customize this view with "Settings". These settings are tied to your session and not shared with other visitors. Consult this page to learn more about how the various settings impact the visualization.

ℹ️ Tip: Columinity hides a lot of statistical complexities from you. This can sometimes lead to seemingly counter-intuitive results. If you find something odd, please consult our FAQ which lists many such scenarios.

ℹ️ Tip: If you want to know which questions in the survey link to each factor, please consult this page.

Effects between factors

The arrows between the core factors reflect the effects we've found across them in our academic research. The thicker the line, the stronger the effect. So, Stakeholder Concern has a strong positive impact on Team Effectiveness. Continuous Improvement, in turn, has a strong effect on Stakeholder Concern. On the other hand, Team Autonomy has a small impact on Stakeholder Concern, but a much stronger one on Continuous Improvement. Team Autonomy improves the ability of teams to improve continuously, which in turn improves Stakeholder Concern and Team Effectiveness. These arrows offer an advanced way to think about what you want to improve first.

7. Review tips for improvement actions

The first step in continuous improvement is to know what needs improvement. The second step is discovering how to improve, which is often more challenging. Columinity helps you in both areas. The previous steps helped you identify where issues exist. In this step, we review how Columinity helps you by suggesting actionable improvements:

1. Click "Tips".
2. Review the tips suggested by Columinity. The tips are grouped by factor and sorted by their expected impact (on a scale from 0 - 100). The impact scores take the model-based effect into account, as well as trends in the results. A detailed explanation is provided here.
3. Each tip includes insights from research (expand with "+") and concrete improvement actions (expand with "+"). We always include three broad strategies for improving, followed by a handful of quick tips that are easy to start with. Most tips also include a selection of facilitation guides for self-run team workshops to focus on improving this factor.
4. We always recommend taking the tips as inspiration. Do what seems right, but don't worry if you deviate from the tips. Columinity's algorithm only knows so much about your teams and the context in which they operate.

8. Managing improvement actions

In this final step, we create and track improvement actions. These actions usually result from the analysis you conduct together. If a new scan is performed a month or two later, we can assess the effectiveness of these actions and adjust accordingly.

1. Click "Actions".
2. Create, edit, or delete actions accordingly. Tie each action to one or more factors that you expect will improve because of it. This allows for much better testing of your actions' effectiveness at achieving that.
3. Clicking the flame icon marks an action as impeded (). This indicates that this team needs help from others, such as management or a coach, to resolve the issue. Impeded actions will appear in the overview of the Teams Dashboard so that team coaches and management can act proactively.
4. Click "Export" to export the actions in CSV format. This allows import or analyses in other tools. For example, the CSV could be imported into a tool like JIRA.

9. Exporting data

Columinity offers several exports of data. Exports are always comma-separated files (CSV).

1. To export the data under "Insights" in the form of a CSV:
   • Click "Insights".
   • Click "Export".
   • Select a date range, a summary period, and a benchmark and export the results as CSV.
2. To export participation rate data (CSV):
   • Click "Teams".
   • Click "Tools".
   • Click "Export".
   • In the dialog, choose "Export" for "Export participation report".
3. To export team summaries (CSV):
   • Click "Teams".
   • Click "Tools".
   • Click "Export".
   • In the dialog, choose "Export" for "Export team summaries".

ℹ️ Tip: Exports help integrate Columinity into your ecosystem. You can import the files into PowerBI, Excel, SPSS, or other tooling.

10. Analyzing trends

Once you start accumulating scans for your teams, Columinity allows you to analyze the trends across teams or a selection of teams.

1. Click "Trends".
2. Choose the teams you want to include in the analysis.
3. Select the factors you wish to analyze trends for under "Select Factors".
4. Choose a summarization period. The default is 3 months. This means that all snapshots for teams from the past three months are aggregated into 1 point, and then all snapshots for each of the three months before that. This creates a consistent rhythm. You can shorten the period, but this also means you need very frequent scans (like monthly).
5. Under "Action count", choose what type of action count summary you'd like to include. If your improvement actions are working, you should see improvements along with an increasing number of completed actions.
6. Select a date range to limit to.

Next steps

• Managing users in the Teams Dashboard