Museum of Transformers

Ever since “Attention is All you Need”, transformer architectures have been applied to countless NLP problems. As they’ve been refined for accuracy and efficiency

References

• Fedus, William, Barret Zoph, and Noam Shazeer. “Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity.” arXiv preprint arXiv:2101.03961 (2021).

Cited as:

@article{mcateer2021mustrans,
    title = "Museum of Transformers",
    author = "McAteer, Matthew",
    journal = "matthewmcateer.me",
    year = "2021",
    url = "https://matthewmcateer.me/blog/museum-of-transformers/"
}

If you notice mistakes and errors in this post, don’t hesitate to contact me at [contact at matthewmcateer dot me] and I will be very happy to correct them right away! Alternatily, you can follow me on Twitter and reach out to me there.

See you in the next post 😄