RMSNorm and SwiGLU
Implemented RMSNorm — pretty straightforward. Implemented SiLU, GLU, and SwiGLU. I think I did it right? Passed the tests, but not sure if I was supposed to compose the SwiGLU with the GLU more directly than I did.
Implemented RMSNorm — pretty straightforward. Implemented SiLU, GLU, and SwiGLU. I think I did it right? Passed the tests, but not sure if I was supposed to compose the SwiGLU with the GLU more directly than I did.