neural network – Page 5 – Andrew Fairless, Ph.D.

What I Read: Self-Attention in GPT

By Andrew Fairless on March 4, 2024January 25, 2024

https://twiecki.io/blog/2024/01/04/ An Intuitive Guide to Self-Attention in GPT: The Venetian MasqueradeThomas WieckiJanuary 4, 2024 “In AI, especially with something as intricate as self-attention, it’s easy to get lost in theContinue readingWhat I Read: Self-Attention in GPT

What I Read: Will Scaling Solve Robotics?

By Andrew Fairless on February 15, 2024December 19, 2023

https://nishanthjkumar.com/Will-Scaling-Solve-Robotics-Perspectives-from-CoRL-2023/ Will Scaling Solve Robotics?: Perspectives From Corl 2023Nishanth J. Kumar “…is training a large neural network on a very large dataset a feasible way to solve robotics?”

What I Read: Limits of Transformers on Compositionality

By Andrew Fairless on February 13, 2024December 19, 2023

https://arxiv.org/abs/2305.18654 Faith and Fate: Limits of Transformers on CompositionalityNouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Peter West, Chandra Bhagavatula, Ronan Le Bras, JenaContinue readingWhat I Read: Limits of Transformers on Compositionality

What I Read: Finetuning LLMs Using LoRA

By Andrew Fairless on February 5, 2024December 18, 2023

https://magazine.sebastianraschka.com/p/practical-tips-for-finetuning-llms Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)Sebastian Raschka, PhDNov 19, 2023 “Low-rank adaptation (LoRA) is among the most widely used and effective techniques for efficiently training customContinue readingWhat I Read: Finetuning LLMs Using LoRA

What I Read: Learning JAX

By Andrew Fairless on January 31, 2024November 20, 2023

https://kidger.site/thoughts/torch2jax/ Learning JAX as a PyTorch developerPatrick KidgerPosted on November 9, 2023 “Assuming you already know PyTorch, this is what I’ve found that you need to know to get upContinue readingWhat I Read: Learning JAX

What I Read: Neural algorithmic reasoning

By Andrew Fairless on January 9, 2024November 7, 2023

https://thegradient.pub/neural-algorithmic-reasoning/ Neural algorithmic reasoningPetar Veličković14.Oct.2023 “…we will talk about classical computation… Think shortest path-finding, sorting, clever ways to break problems down… we will also investigate how to capture such computationContinue readingWhat I Read: Neural algorithmic reasoning

What I Read: SAT Solvers

By Andrew Fairless on January 8, 2024November 7, 2023

https://www.borealisai.com/research-blogs/tutorial-9-sat-solvers-i-introduction-and-applications/ Tutorial #9: SAT Solvers I: Introduction and applicationsS. PrinceNov. 10, 2020 “This tutorial concerns the Boolean satisfiability or SAT problem…. we’ll show how to fit both neural networks andContinue readingWhat I Read: SAT Solvers

What I Read: Multimodality

By Andrew Fairless on January 4, 2024November 7, 2023

https://huyenchip.com/2023/10/10/multimodal.html Multimodality and Large Multimodal Models (LMMs)Chip HuyenOct 10, 2023 “For a long time, each ML model operated in one data mode… However, natural intelligence is not limited to justContinue readingWhat I Read: Multimodality

What I Read: Finetuning LLMs with LoRA and QLoRA

By Andrew Fairless on January 3, 2024November 7, 2023

Finetuning LLMs with LoRA and QLoRA: Insights from Hundreds of Experiments Finetuning LLMs with LoRA and QLoRA: Insights from Hundreds of ExperimentsSebastian RaschkaOctober 12, 2023 “LoRA is one of theContinue readingWhat I Read: Finetuning LLMs with LoRA and QLoRA

What I Read: Distributed Training, Finetuning

By Andrew Fairless on December 20, 2023November 7, 2023

https://sumanthrh.com/post/distributed-and-efficient-finetuning/ Everything about Distributed Training and Efficient FinetuningSumanth R HegdeLast updated on Oct 13, 2023 “practical guidelines and gotchas with multi-GPU and multi-node training”

Tag: neural network