Improving Management with Individual and Group-Based Consulting: Results from a Randomized Experiment in Colombia

• Abstract
• Introduction
• Background
• Intervention
• Data
• Results
• Conclusions

---

Abstract

• RQ: Can group-based approaches make scale-up management improvements?
• Methodology: Randomized experiment to Colombian auto parts firms.
  1. expensive one-on-one consulting
  2. consulting to small groups of firms: 1/3 the cost of the first one.
• Results: Both approaches lead to improvements in management practices of 8-10 % points.
  • The potential of group-based approaches is suggested.

---

Introduction

Literature

• A large difference in the management practices is strongly correlated with productivity.
  • Bloom et al. (2016): 30 percent of cross-country productivity differences.
  • Bloom et al. (2013), Bloom et al. (2020): An experiment with textile firms in India
    • intensive individualized consulting can deliver productivity improvements of 17 %.
• Expensive cost of intervention
  • Intervention had a market value of $250,000 per treated firm.

Experiment and Results

• Experiment with the Colombian Government
• Targets: Small and medium enterprises (SMEs).
• Two Interventions
  1. One-on-one consulting using local teams of consultants.
  2. Group-based approach.
• Results: Both treatments improved management by 8-10 % points (administrative panel survey for 3-4 years after implementation.)
  • The group-based intervention clearly superiors to the individual intervention on a cost-benefit basis.
  • Other outcomes: increase in employment, growth in sales, increase in profits, and value-added...

Contribution

1. Improving business and management in firms.
   • Most of the literature has focused on short training courses and microenterprises (McKenzie 2020)
   • The potential of individualized consulting: Bruhn et al. (2018), Higuchi et al. (2017)
2. Interfirm interactions and social learning.
   • emerging literature: Cai and Szenidl, 2018; Chatterji et al., 2018; Lafortune et al. 2018; Dalton et al., 2018; Ray, 2006; Brooks et al., 2018).
   • This paper is distinguished with them by working SMEs that are more complex organization.
     • Interaction with other firm owners + with specialized staff.
   • Many practices are not adopted (Bloom et al., 2013): Adoption of new practices is correlated with the overall learning of the group.
3. How to scale-up policies from promising researcher pilot studies.

Context and Sample

• Labor productivity in Columbia is low: equivalent to 1/4 of the U.S.

Launch of Programa de Extensionismo Tecnológico (2012)

• Offers assistance in improving production practices in order to improve profitability, productivity, and competitiveness, for free.
• Auto-parts sector was chosen as the subjects of the field experiment for:
  • sufficient numbers of firms
  • a number of locations
  • potential for growth
  • similar enough to other industrial sectors.
• The sector consists largely of second-tier suppliers and employs approximately 25,000 workers.
  • exports about $US500 million each year.
• Firms that are legally registered, in business for at least two yrs, a first or second-tier supplier to the automobile industry are eligible.
• Market failure problems:
  1. Many badly managed firms do not know they are badly managed.
  2. They cannot identify which providers can offer good services, lack the financial resources and insurance.
• 218 firms applied and 159 proceeded to the experiment.
  • 3.3% of all Colombian firms in the related categories, and 20% of those classified into ISIC code 2930.
  • Average size and productivity for participants are similar to those of non-participants, but very small/large firms are relatively few.

Random assignment and firm characteristics

• Firms are randomly assigned to three groups of 53 firms each.

  • Using Mahalanobis distance between firms in terms of their geographic location, size, labor productivity, and management practices, they matched triplets to be treated.

• Firm Characteristics

• Management Practices

  • Measured in terms of 141 individual practices based on an assessment developed by the Colombian National Productivity Center: Anexo K
    • Five Areas
    1. Financial practices,
    2. Human resource practices,
    3. Logistics practices,
    4. Marketing practices, and
    5. Production practices. scored on a five-point scale.
  • It is difficult to balance all of the variables because of the size of their sample.
  • However, their overall omnibus tests of joint orthogonality cannot reject that these variables do not jointly predict treatment status.

External validity and comparison to Bloom Van Reenen Management Practices

• World Management Surveys (WMS): random sample of 180 firms representative of the Colombian manufacturing sector.
  1. The mean and distribution of WMS management practices scores for the sample are similar to that of the overall manufacturing sector.
  2. Colombia's average management practice scores are similar to many other developing countries.
  3. WMS data is well comparable to the Anexo management measure.

Macroeconomic context

• The Colombian auto parts sector had grown at an annual average of 5.4% per year over the 2002 to 2012 period.
• A combination of external and internal shocks starting in the rate 2013: Real sales of domestic production were then fairly flat over their study period.

The Intervention

National Productivity Center designed and implemented the program (concluded the contract with the government).

• NPO to be the recipient of training and in-house technical assistance to develop capabilities in implementing managerial consulting services.
• has developed a model of operation and they have been supporting more than 4,000 Colombian companies.
• Two types of intervention: lead consultants, and area consultants.
• The direct cost of implementation: US$2.4M.

Diagnostic phase

• All firms, including the control, received a diagnostic as the first phase.
  • Two-week programs by a team of 6 consultants to evaluate the 141 individual management practices.
  • work with the leaders to finish with a report that analyzed managerial practices for each area, KPI for each area, and recommended practices to prioritize.
• Cost US$3,553 per firm.
• identifies priority practices to be improved by management with the accompaniment of the consultants.
  • Some of the priority areas for improvement in each of the five areas were common to many firms.

Individual Consulting

• 6-months support between March and November 2014.
• Assigned a team of five specialized consultants, along with a leader.
  • 20-hour training to the person in the firm
  • Individual consulting to help the firms implement the improvement plan developed in the diagnostic phase.
  • Monthly meetings to improve and re-define priorities.
• Total of 500 hours and cost US$28,950 per firm.

Group Consulting Treatment

• six months from September 2015 to May 2016.

• To test whether the same gains in management improvements could be achieved more efficiently through working with small groups at a time.

• Lower costs:

  1. Each consultant's time was spread over more firms.
  2. Saving travel cost of the consultants.

• CNP had not previously done group consulting of this form.

1. Meetings of training.

   • 3 to 8 firms participate in at the same time.
   • Firms produce complementary products with similar management problems.
   • Leaders in each area from the firms work together and help each other improve.
   • Contents are almost the same as Individual treatments but are delivered to the group.

2. Group consulting

   • held weekly about one or two specific areas.
   • focus on the implementation of the actions agreed in the improvement plans of each company.
   • prioritized in the Improvement Plan, and its Action Plan would be updated.
   • It is more difficult to directly verify changes made in logistics and production: firms are to bring the photos to the group meetings.
     • Monthly visits by the consultants.
   • Cost US$10,500 per firm: 408 hours of consultants.

Take-up, Data-sources, and Attrition

• 86.8% and 75.4% of the firms assigned to individual and group interventions, respectively, started programs. 67.9% of those assigned to the group treatment completed it.

Data Sources

• Baseline: collected from the application form and diagnostic phase and covered firm characteristics in 2013.
• Three types of follow-up data.
  1. data management practices in the firm.
     • Anexo K management score collected by the CNP in-person visits.
  2. Key performance indicators (KPIs) from the firms collected in-person visits.
     • Innovations for Poverty Action provides an independent check and assistance in collecting data directly from the firms.
  3. Administrative data sources
     • PILA (Unified Register of Contributions): employment outcome
       • health, pensions, and disability insurance paid for workers.
       • Monthly data from pre-intervention through the end of December 2018.
     • The annual manufacturing survey (EAM).
       • mandatory survey for establishment with more than 10 employees.
       • annual sales, value-added, profits, and labor productivity measures from 2010 to 2018.
       • Smaller and younger firms were less likely to be matched.

Impact on Management Practices

$$ \begin{align*} \textit{AnexoK} _ {i, k} =& \alpha + \beta_1 \textit{Individual} _i \times \textit{During} _ t + \beta _ 2 \textit{Individual} _ i \times \textit{Post} _t \\ &+ \gamma_1 \times \textit{Group} _ i \times \text{During} _ t + \beta \gamma _ 2 \textit{Group} _ i \times \textit{Post} _ t + \sum _ {g = 1} ^ {53} \delta _ g \mathbb{1} (i \in g) + \theta \textit{Post}_t + \rho \textit{Anexo K} _ {i, 1} + \epsilon \end{align*} $$

• $t$ indicates three periods: baseline, during the intervention, and post-intervention.
• ANCOVA regression (McKenzie, 2012): controls for the randomization triplets and the baseline level of management practices.

• Four key evidences
  1. Both treatment effect is significant,
  2. Effects of the treatment persist for at least one year post-intervention.
  3. The size of impacts (8-10 percentage points) is similar to one another.
  4. one-third of the Bloom et al. (2013), but twice the size of McKenzie and Woodruff (2014).

Impacts on Firm Outcomes

• Group: A statistically significant increase in employment of 6 workers post-treatment, or 12 percent.

Cost-Benefit and to Policy Maker Expectations.

• Both treatments succeeded to a similar magnitude in improving the set of management practices, and about firm performance outcomes (firm size) for the group treatment, group treatments superior to the individual treatment.
  • USD10,500 per firm for group, while US$28950 per firm for the individual.
  • Firms specialize in different products, suggesting that the internal validity of the estimates should not be invalidated by spillovers.
  • depreciation of the peso.

Possible Mechanisms

1. Small sample problem prevents from controlling firm heterogeneity
   • Positive and significant impacts of the individual treatment on all at the performance measures.
2. Group treatment does have a larger impact.
   • group treatment may provide a way for the improvement that will persist longer, or delivers additional benefits.

Conclusions

• Group treatment model may lower the cost of delivery and also results in an increase in firm performance and firm productivity.
• Small sample size problem prevents them from offering detailed recommendations as to what types of firms it best works.