6: Evaluation - 7-teens/7-teens-DSA3101-2410-Project GitHub Wiki

Evaluation of Model Performance Against Business Objectives

Alignment with Business Objectives

  1. Demand Forecasting Accuracy:

    • The model consistently produced forecasts with RMSE within 10-15% of the actual demand.
    • This level of accuracy enables precise planning for procurement, inventory management, and staffing.

  2. Inventory Cost Reduction:

    • By leveraging Prophet’s upper bounds (yhat_upper) for safety stock calculations, the model provided a dynamic inventory strategy that reduced holding costs while preventing stock-outs during demand surges.

  3. Revenue Optimization:

    • The integration of predicted demand and pricing adjustments resulted in improved revenue capture during high-demand periods.
    • The model outperformed static strategies, especially during promotional campaigns or peak seasons, aligning with the business's objective of maximizing profitability.

  4. Operational Agility:

    • The model’s inclusion of external regressors (e.g., campaigns, holidays) allowed the business to anticipate demand shifts and adjust operations in real time, supporting rapid decision-making.

Limitations of Current Approach

  1. Limited Feature Representation:

    • While the model incorporates key regressors like campaigns and holidays, other potential drivers such as competitor activity or macroeconomic trends are not included, which could improve accuracy in fluctuating markets.

  2. Assumption of Linear Inventory and Pricing Relationships:

    • The current approach assumes consistent relationships between demand, price, and inventory. This may not fully capture non-linear or threshold effects (e.g., drastic changes in demand at certain price points).

  3. Sensitivity to Outliers and Anomalies:

    • Sudden demand spikes or irregular patterns not tied to known events may lead to forecast deviations, affecting safety stock recommendations.

  4. Lag in Response to New Trends:

    • The model relies on historical patterns and may be slower to adapt to emerging trends or unforeseen disruptions in demand.

Suggestions for Model Improvements

  1. Incorporate Additional Regressors:

    • Integrate external data sources such as competitor pricing, macroeconomic indicators, or consumer sentiment to provide a more comprehensive view of demand drivers.

  2. Enhance Non-Linear Modeling:

    • Implement non-linear techniques or hybrid models that better capture complex interactions between demand, price, and inventory.

  3. Real-Time Adaptation:

    • Develop a real-time forecasting pipeline that updates demand predictions as new data becomes available, ensuring agility in responding to sudden changes.

  4. Scenario Testing and Simulations:

    • Add what-if analysis capabilities to simulate different pricing or inventory scenarios, providing actionable insights for decision-makers.

  5. Evaluation Across Sub-Categories:

    • Conduct granular analyses at the sub-category level to identify unique demand patterns, ensuring that the model’s outputs are tailored to specific product segments.

  6. Incorporate Advanced Pricing Strategies:

    • Utilize machine learning models for dynamic pricing to optimize revenue further by adjusting prices based on real-time demand elasticity predictions.

  7. Outlier Detection and Adjustment:

    • Implement automated anomaly detection methods to identify and mitigate the impact of outliers on demand forecasts, ensuring more robust predictions.

Conclusion

The Prophet-driven model demonstrates strong alignment with business objectives, offering significant improvements in inventory management and revenue optimization. However, addressing its limitations through advanced techniques and additional features will further enhance its reliability, adaptability, and business impact.