Added default configurations and reformated with black.
[mygpt.git] / mygpt.py
2023-01-21 François FleuretAdded default configurations and reformated with black. master
2022-08-27 Francois FleuretThe "mask" array actually specifies what attention...
2022-08-08 Francois FleuretAdded the small weight embedding + id layer norm inits.
2022-08-07 Francois FleuretAdded the small-weight embedding initialization.
2022-07-30 Francois FleuretUpdate.
2022-07-29 Francois FleuretOCDC
2022-07-29 Francois FleuretUpdate.
2022-07-29 Francois FleuretUpdate.
2022-07-28 Francois FleuretOCDC
2022-07-27 Francois FleuretOCDC
2022-07-27 Francois FleuretOCDC
2022-07-26 Francois FleuretFixed the size of w_o.
2022-07-26 Francois FleuretUpdate.
2022-07-26 Francois FleuretRemoved the Linear transformation since there is now...
2022-07-26 Francois FleuretMoved the input/output shift in the forward of the...
2022-07-25 Francois FleuretOCD update
2022-07-25 Francois FleuretInitialize properly w_o.
2022-07-25 Francois FleuretAdded the (missing) W_o
2022-07-01 Francois FleuretAdd cross-attention to QKVAttention.
2022-06-13 Francois FleuretUpdate.
2022-04-29 Francois FleuretUpdate.
2022-04-29 Francois FleuretUpdate.
2022-04-25 Francois FleuretUpdate.
2022-04-24 Francois FleuretUpdate.
2022-04-24 Francois FleuretInitial commit