wshine ai hw8 - TheEvergreenStateCollege/upper-division-cs-23-24 GitHub Wiki
-
d_in : 3
d_out: 2
context_length: 6
dropout: also not used as a parameter to the constructor. but the book had us using a dropout value of 0.
-
b : 2
num_tokens: 6
d_in : 3
-
each of keys, queries, and values are 2x6x2
-
2x6x6
-
2x6x6
2 6 3
keys: torch.Size([2, 6, 2])
queries: torch.Size([2, 6, 2])
values: torch.Size([2, 6, 2])
attn: torch.Size([2, 6, 6])
attn: torch.Size([2, 6, 6])
context_vecs.shape: torch.Size([2, 6, 2])
unsure what to do for this exercise. we are asking to modify the parameters so the shape of the context vector reurned is two-dimensional instead of 4, but the it currently has a shape of 2x6x2.
current ouput with no modifications to parameters:
tensor([[[0.3190, 0.4858],
[0.2943, 0.3897],
[0.2856, 0.3593],
[0.2693, 0.3873],
[0.2639, 0.3928],
[0.2575, 0.4028]],
[[0.3190, 0.4858],
[0.2943, 0.3897],
[0.2856, 0.3593],
[0.2693, 0.3873],
[0.2639, 0.3928],
[0.2575, 0.4028]]], grad_fn=<ViewBackward0>)
context_vecs.shape: torch.Size([2, 6, 2])