Backtest Statistics - jaeaehkim/trading_system_beta GitHub Wiki

Motivation

  • ์ „๋žต์˜ ํŠน์„ฑ์„ ํŒŒ์•…ํ•˜๊ณ  ๋น„๊ตํ•˜๊ธฐ ์œ„ํ•œ ์šฉ๋„
  • Backtesting ๋ชจ๋“ˆ์„ ๊ตฌํ˜„ํ•  ๋•Œ ์•„๋ž˜ Stat์„ ๊ณ„์‚ฐํ•˜๊ธฐ ์œ„ํ•œ ์ •๋ณด๋ฅผ loggingํ•˜๋Š”๊ฒŒ ์ค‘์š”

General Characteristics

  • Time Range
    • ์‹œ์ž‘ ๋‚ ์งœ์™€ ์ข…๋ฃŒ ๋‚ ์งœ์˜ ๊ธธ์ด
  • Average AUM
    • ์šด์šฉ ์ค‘์ธ ์ž์‚ฐ์˜ ํ‰๊ท  ๋‹ฌ๋Ÿฌ ๊ฐ€์น˜. ๋งค์ˆ˜,๋งค๋„ ํฌ์ง€์…˜์˜ ๋‹ฌ๋Ÿฌ ๊ฐ€์น˜๋ฅผ ์–‘์˜ ์‹ค์ˆ˜๋กœ ๊ฐ„์ฃผ.
  • Capacity
    • ์ „๋žต์˜ ์šฉ๋Ÿ‰์€ ๋ชฉํ‘œํ•˜๋Š” ํผํฌ๋จผ์Šค๋ฅผ ์„ฑ์ทจํ•  ์ˆ˜ ์žˆ๋Š” ์ตœ๋Œ€ AUM์œผ๋กœ ์ธก์ •. ์ตœ์†ŒAUM ~ ์ตœ๋Œ€AUM ๊นŒ์ง€ ์„ฑ๋Šฅ์ด ์ €ํ•˜๋จ.
  • Leverage
    • ํ‰๊ท  AUM๊ณผ ํ‰๊ท  ๋‹ฌ๋Ÿฌ ํฌ์ง€์…˜์˜ ํฌ๊ธฐ์˜ ๋น„์œจ์„ ๊ณ„์‚ฐ
  • Maximum Dollar Position Size
    • ํŠน์ • ํฌ์ธํŠธ์—์„œ ์ตœ๋Œ€ ๋‹ฌ๋Ÿฌ ํฌ์ง€์…˜ ํฌ๊ธฐ๋ฅผ ๊ณ„์‚ฐํ•จ์œผ๋กœ์จ ํ‰๊ท  AUM ๋Œ€๋น„ ์–ผ๋งˆ๋‚˜ ํŠ€๋Š”์ง€ ํ™•์ธ
  • Ratio of Longs
    • ๋ฒ ํŒ… ์ค‘ ๋งค์ˆ˜ ํฌ์ง€์…˜์˜ ๋น„์œจ์„ ๋ณด์—ฌ์คŒ. ์‹œ์žฅ ์ค‘๋ฆฝ์  ์ „๋žต์ธ Long-Short์€ ์ด์ƒ์ ์œผ๋กœ 0.5์— ๊ฐ€๊น๋‹ค.
  • Frequency of Bets
    • Backtest ๋‚ด์—์„œ ์—ฐ๊ฐ„ ๋ฒ ํŒ… ๊ฐœ์ˆ˜๋ฅผ ์˜๋ฏธํ•จ. ์—ฐ์†์ ์œผ๋กœ ๊ฐ™์€ ๋ฐฉํ–ฅ์˜ ๋ฒ ํŒ…์€ ํ•˜๋‚˜์˜ ๋ฒ ํŒ…์œผ๋กœ ๊ฐ„์ฃผํ•œ๋‹ค. ๋ฒ ํŒ…์€ ํฌ์ง€์…˜์˜ ์œ„์น˜๊ฐ€ ์‚ฌ๋ผ์ง€๊ฑฐ๋‚˜ ๋’ค์ง‘ํž ๋•Œ ๋๋‚œ๋‹ค.
    • ํฌ์ง€์…˜์˜ ๊ฐœ์ˆ˜ >= ๋ฒ ํŒ…์˜ ๊ฐœ์ˆ˜
  • Average Holding Period
    • ๋ฒ ํŒ…์ด ์œ ์ง€๋˜๋Š” ์ผ ์ˆ˜์˜ ํ‰๊ท , ๊ณ ๋นˆ๋„ ์ „๋žต์€ '์ดˆ'๋‹จ์œ„, ์ €๋นˆ๋„ ์ „๋žต์€ ๋ช‡ ๋‹ฌ ~ ๋ช‡ ๋…„.
  • Annualized Turnover
    • ์—ฐ๊ฐ„ ๊ฑฐ๋ž˜๋œ ํ‰๊ท  ๋‹ฌ๋Ÿฌ๋Ÿ‰๊ณผ ํ‰๊ท  ์—ฐ๊ฐ„ AUM์˜ ๋น„์œจ๋กœ ์ธก์ •. ๋‚ฎ์€ ๋ฒ ํŒ…์—๋„ ํฌ์ง€์…˜ ํŠœ๋‹์œผ๋กœ ์ธํ•ด์„œ ๋†’์€ ๊ฑฐ๋ž˜๋Ÿ‰์ด ๊ฐ€๋Šฅํ•จ. ๊ทน๋‹จ์ ์œผ๋กœ ํšŸ์ˆ˜๋Š” ์ ์œผ๋‚˜ ์ตœ๋Œ€ ๋งค์ˆ˜, ์ตœ๋Œ€ ๋งค๋„๊ฐ€ ๋ฐ˜๋ณต๋˜๋Š” ๊ฒฝ์šฐ ๋†’์€ ๊ฑฐ๋ž˜๋Ÿ‰์„ ๋ณด์ธ๋‹ค.
  • Correlation to Underlying
    • ์ „๋žต์˜ ์ˆ˜์ต๋ฅ ๊ณผ ๊ธฐ์ € ํˆฌ์ž ์ƒํ’ˆ ์ˆ˜์ต๋ฅ ๊ณผ์˜ ์ƒ๊ด€๊ด€๊ณ„. ์–‘,์Œ์œผ๋กœ ๋„ˆ๋ฌด ํฌ๊ฒŒ ๋‚˜์˜ค๋Š” ๊ฒฝ์šฐ๋Š” ํ•ด๋‹น ์ƒํ’ˆ์„ ํ™€๋”ฉํ•˜๋Š” ๊ฒƒ๊ณผ ๋‹ค๋ฆ„ ์—†๊ธฐ์—.

Performance

  • PnL
    • Backtesting์—์„œ ์ „์ฒด์— ๋ฐœ์ƒํ•œ ๋‹ฌ๋Ÿฌ ์ด์•ก. ์ตœ์ข… ํฌ์ง€์…˜์ด ๋‚จ์•„์žˆ๋Š” ๊ฒฝ์šฐ๋Š” ๊ทธ๊ฒƒ์˜ ํ˜„๊ธˆํ™” ๋น„์šฉ์„ ํฌํ•จํ•œ๋‹ค. '์ˆ˜์ต๋ฅ  ๊ธฐ๋ฐ˜ ์‹œ๋ฎฌ๋ ˆ์ด์…˜'์ด ์•„๋‹Œ '๊ธˆ์•ก ๊ธฐ๋ฐ˜ ์‹œ๋ฎฌ๋ ˆ์ด์…˜'์„ ์ง„ํ–‰ํ•ด์•ผ ๋””ํ…Œ์ผํ•˜๊ฒŒ ๊ณ„์‚ฐํ•  ์ˆ˜ ์žˆ์Œ.
  • PnL from Long Positions
    • PnL ๋‹ฌ๋Ÿฌ ์ค‘์—์„œ ์˜ค์ง ๋งค์ˆ˜ ํฌ์ง€์…˜์œผ๋กœ๋งŒ ๋ฐœ์ƒํ•œ ๋น„์ค‘. Long-Short bias๋ฅผ ํ‰๊ฐ€ํ•  ๋•Œ ์œ ์šฉ.
  • Annualized Rate of Return
    • ์ด ์ˆ˜์ต๋ฅ ์˜ ์‹œ๊ฐ„-๊ฐ€์ค‘ ์—ฐํ™˜์‚ฐ ํ‰๊ท  ๋น„์œจ๋กœ ๋ฐฐ๋‹น, ์ฟ ํฐ, ๋น„์šฉ์„ ํฌํ•จ.
  • Hit Ratio
    • ์–‘์˜ ์ˆ˜์ต ๊ฒฐ๊ณผ๋ฅผ ๋‚ธ ๋ฒ ํŒ…์˜ ๋น„์ค‘
  • Average Return from Hits
    • ์ˆ˜์ต์„ ๋‚ธ ๋ฒ ํŒ…์˜ ํ‰๊ท  ์ˆ˜์ต๋ฅ 
  • Average Return from Misses
    • ์†์‹ค์„ ๋‚ธ ๋ฒ ํŒ…์˜ ํ‰๊ท  ์ˆ˜์ต๋ฅ 
  • Time-Weighted Rate of Return
  • Returns Concentration
    • Long์œผ๋กœ ์ธํ•œ ์ˆ˜์ต๋ฅ  ์ง‘์ค‘๋„, Short, ์›”๋ณ„ ์ง‘์ค‘๋„ ๋“ฑ์„ ๊ณ„์‚ฐํ•  ์ˆ˜ ์žˆ์Œ
  • Drawdown and Time under Water
    • ์ „๊ณ ์  ๋Œ€๋น„ ๋‚™ํญ์„ ๋œปํ•˜๋Š” drawdown sequence๋ฅผ ๊ณ„์‚ฐํ•  ์ˆ˜ ์žˆ๊ณ  ์ „๊ณ ์  ํšŒ๋ณต ๊ตฌ๊ฐ„๋งˆ๋‹ค under water ๊ธฐ๊ฐ„์„ ๊ณ„์‚ฐํ•  ์ˆ˜ ์žˆ์Œ

Implementation Shortfall

  • Broker Fees per Turnover
    • ๊ฑฐ๋ž˜ํ•œ ํฌํŠธํด๋ฆฌ์˜ค์— ๋Œ€ํ•ด ๋ธŒ๋กœ์ปค์—๊ฒŒ ์ง€๊ธ‰ํ•˜๋Š” ๋น„์šฉ. ๊ฑฐ๋ž˜์†Œ๋งŒ ์“ฐ๋Š” ๊ฒฝ์šฐ์—” ๊ฑฐ๋ž˜์†Œ ์ˆ˜์ˆ˜๋ฃŒ.
  • Average Slippage per Turnover
    • execution์œผ๋กœ ์ธํ•ด์„œ ์ƒ๊ธฐ๋Š” ์Šฌ๋ฆฌํ”ผ์ง€. ์ฃผ๋ฌธ ์‹œ๊ฐ„ ์ฐจ์— ๋”ฐ๋ฅธ ์Šฌ๋ฆฌํ”ผ์ง€, mid-price์™€ fll-price๊ฐ„์˜ ์†์‹ค์„ ์˜๋ฏธ
  • Dollar Performance per Turnover
    • ๋‹ฌ๋Ÿฌ ๊ธฐ๋ฐ˜ ์„ฑ๊ณผ / ์ด ํฌํŠธํด๋ฆฌ์˜ค ๊ฑฐ๋ž˜์•ก -> ๊ฑฐ๋ž˜ ๊ธˆ์•ก์„ ๊ณ ๋ คํ•œ ์„ฑ๊ณผ ์ง€ํ‘œ
  • Return on Execution Costs
    • ๋‹ฌ๋Ÿฌ ๊ธฐ๋ฐ˜ ์„ฑ๊ณผ / ์ด ์‹คํ–‰ ๋น„์šฉ

Efficiency

  • The Sharpe Ratio
  • The Probabilistic Sharpe Ratio
  • The Deflated Sharpe Ratio
  • Annualized Sharpe Ratio
  • Informatio Ratio

Classification Scores

  • Accuracy
  • Precision
  • Recall
  • F1
  • Negative log-loss