Added the small weight embedding + id layer norm inits.
authorFrancois Fleuret <francois@fleuret.org>
Mon, 8 Aug 2022 05:13:54 +0000 (07:13 +0200)
committerFrancois Fleuret <francois@fleuret.org>
Mon, 8 Aug 2022 05:13:54 +0000 (07:13 +0200)
commit3b62d298013c7b940aec7cab0f74fb5118493f99
treed451dee9cf50b1a5171dd56dee36f2124b1cc8ee
parentf3a734b6c522b2be0004a1b8bc2fe2eab2a90263
Added the small weight embedding + id layer norm inits.
mygpt.py