Pytorch - soup1997/Study-Alone GitHub Wiki

Dataset

class CustomDataset(torch.utils.data.Dataset): 
  def __init__(self):
  ๋ฐ์ดํ„ฐ์…‹์˜ ์ „์ฒ˜๋ฆฌ๋ฅผ ํ•ด์ฃผ๋Š” ๋ถ€๋ถ„

  def __len__(self):
  ๋ฐ์ดํ„ฐ์…‹์˜ ๊ธธ์ด. ์ฆ‰, ์ด ์ƒ˜ํ”Œ์˜ ์ˆ˜๋ฅผ ์ ์–ด์ฃผ๋Š” ๋ถ€๋ถ„

  def __getitem__(self, idx): 
  ๋ฐ์ดํ„ฐ์…‹์—์„œ ํŠน์ • 1๊ฐœ์˜ ์ƒ˜ํ”Œ์„ ๊ฐ€์ ธ์˜ค๋Š” ํ•จ์ˆ˜

Transform

์ด๋ฏธ์ง€๋ฅผ ๋ณ€ํ˜•(transform)์„ ์ ์šฉํ•ด์„œ ๋ฐ์ดํ„ฐ๋ฅผ ์กฐ์ž‘ํ•˜๊ณ  ํ•™์Šต์— ์ ํ•ฉํ•˜๊ฒŒ ๋งŒ๋“ ๋‹ค๋Š” ๊ฒƒ์ด ํฐ ๊ฐœ๋…์ด๋‹ค.

import torchvision.transforms as T

# ์ด๋ฏธ์ง€ ์ „์ฒ˜๋ฆฌ ์ž‘์—…์„ ์ •์˜
preprocess = T.Compose([
    T.Resize((256, 256)), # resize to (256, 256)
    T.RandomHorizontalFlip(), # 
    T.ToTensor(), # normalization with range [0, 1]
    T.Normalize((0.5), (0.5)) # ๊ฐ ์ฑ„๋„๋ณ„ ํ‰๊ท ์„ ๋บ€๋’ค ํ‘œ์ค€ํŽธ์ฐจ๋กœ ๋‚˜๋ˆ„์–ด ํ‘œ์ค€ํ™”๋ฅผ ์ง„ํ–‰, standarazation์— ๊ฐ€๊นŒ์›€ 
])

# ์ด๋ฏธ์ง€์— ์ „์ฒ˜๋ฆฌ ์ž‘์—… ์ ์šฉ
image = Image.open('image.jpg')
imge = preprocess(image)

Grdient

LSTM

class Net(nn.Module):
    def __init__(self, input_dim, hidden_dim, seq_len, output_dim, layers):
        super(Net, self).__init__()
        self.hidden_dim = hidden_dim
        self.seq_len = seq_len
        self.output_dim = output_dim
        self.layers = layers

        self.lstm = nn.LSTM(input_size=input_dim, 
                            hidden_size=hidden_dim, 
                            num_layers=layers,
                            batch_first=True)
        
        self.fc = nn.Linear(in_features=hidden_dim, 
                            out_features=output_dim, 
                            bias=True)

    def reset_hidden_state(self, x):
        h_0 = torch.zeros(self.layers, x.size(0), self.hidden_dim)
        c_0 = torch.zeros(self.layers, x.size(0), self.hidden_dim)

        return h_0, c_0

    def forward(self, x):
        h_0, c_0 = self.reset_hidden_state(x)
        output, (h_t, c_t) = self.lstm(x, (h_0, c_0))
        x = self.fc(output[:, -1])
        return x

Parameters

torch.nn.LSTM(input_size, hidden_size, num_layers, bias=True, batch_first=True, dropout, bidirectional)

์ž์„ธํ•œ ์„ค๋ช…

  • input_size:input์˜ feature dimension์„ ๋„ฃ์–ด์ฃผ์–ด์•ผ ํ•œ๋‹ค. time step์ด ์•„๋‹Œ input feature dimension
  • hidden_size: ๋‚ด๋ถ€์—์„œ ์–ด๋–ค feature dimension์œผ๋กœ ๋ฐ”๊ฟ”์ฃผ๊ณ  ์‹ถ์€์ง€๋ฅผ ๋‚˜ํƒ€๋ƒ„
  • num_layers: ์žฌ๊ท€ ์ธต์˜ ๊ฐฏ์ˆ˜

image

  • bias: bias term์„ ๋‘˜ ๊ฒƒ์ธ๊ฐ€ (Default: True)
  • batch_first: batch_first=True๋ผ๋ฉด, (Batch_size, Time_step, Input_feature_dimension) ์ˆœ์„œ (Default: False)
  • dropout: Dropout์˜ ๋น„์œจ, generalization ์ž˜ ์•ˆ๋˜๋ฉด ์กฐ์ •
  • bidirectional: ์–‘๋ฐฉํ–ฅ ์—ฌ๋ถ€ (bidirectional ํ•˜๋ฉด [forward, backword]๊ฐ€ ๋˜์–ด feature dimension 2๋ฐฐ)

Reset State

output, (h_t, c_t) = self.lstm(x, (h_0, c_0))
Input์€ ์ž…๋ ฅ sequence์™€ ์ดˆ๊ธฐ state (LSTM์ด๋ฉด {hidden state, cell state} ๊ทธ ์™ธ๋Š” {hidden state})๋กœ ๊ตฌ์„ฑ
์ดˆ๊ธฐ state๊ฐ€ ์—†๋‹ค๋ฉด ๋„ฃ์–ด์ฃผ์ง€ ์•Š์•„๋„ ๋˜๊ณ  ์ž๋™์œผ๋กœ zero๋กœ ์„ค์ • 
์ž…๋ ฅ sequence์˜ dimension์€ (Batch, Time_step, Feature dimension)์ˆœ (batch_first=True ๊ธฐ์ค€)

LSTM Cell

def LSTMCell(input, hidden, w_ih, w_hh, b_ih=None, b_hh=None):
    if input.is_cuda:
        ...
    hx, cx = hidden
    gates = F.linear(input, w_ih, b_ih) + F.linear(hx, w_hh, b_hh)
    ingate, forgetgate, cellgate, outgate = gates.chunk(4, 1)
    ingate     = F.sigmoid(ingate)
    forgetgate = F.sigmoid(forgetgate)
    cellgate   = F.tanh(cellgate)
    outgate    = F.sigmoid(outgate)
    cy = (forgetgate * cx) + (ingate * cellgate)
    hy = outgate * F.tanh(cy)
    return hy, cy

1_hl9UVtgIcQkDIGD8VFykdw

Equation

$i_t=\sigma(W_{ii}x_t+b_{ii}+W_{hi}h_{t-1}+b_{hi})$
$f_t=\sigma(W_{if}x_t+b_{if}+W_{hf}h_{t-1}+b_{hf}$
$g_t=\tanh(W_{ig}x_t+b_{ig}+W_{hg}h_{t-1}+b_{hg})$
$o_t=\sigma(W_{io}x_t+b_{io}+W_{ho}h_{t-1}+b_{ho})$
$c_t=f_t\odot c_{t-1}+i_t\odot g_t$
$h_t=o_t\odot\tanh(c_t)$

Parameters

LSTM Cell ํŒŒ๋ผ๋ฏธํ„ฐ ์„ค๋ช…

torch.nn.LSTMCell(input_size, hidden_size, bias=True, device=None, dtype=None)

  • input_size: ์ž…๋ ฅ์œผ๋กœ ์ฃผ์–ด์ง€๋Š” ๋ฐ์ดํ„ฐ์˜ feature์˜ ๊ฐฏ์ˆ˜

  • hidden_size: hidden state $h$์˜ ๋ฒกํ„ฐ ํฌ๊ธฐ

โš ๏ธ **GitHub.com Fallback** โš ๏ธ