EIE: Efficient Inference Engine on Compressed Deep Neural Network Terminology DRAM: Dynamic Random-Access Memory, used for large off-chip storage but consumes high energy per access. EIE a...

Introduction Weights and biases one of the most important factors of neural networks. Think about all the topics (e.g., backpropagation, activation functions, loss functions, optimization, regular...

Introduction Machine learning models aim to discover patterns in data by learning the relationships between inputs and outputs. To measure how well a model performs this task, we calculate the dif...

Introduction This time, we will skim through Yoshua Bengio’s seminal paper A Neural Probabilistic Language Model (2003) which laid the foundation of: Statistical language modeling that addresse...

Introduction Probabilistic language models is trained based on the statistics of the training corpus. However, no matter how large the training corpus is, there is always a possibility that the mod...

Introduction In our previous post, we explored a bigram language model that predicts the next character in a sequence based on probability distributions. At the heart of this model was the negativ...

Introduction Today, we will be building a Bigram Language Model which takes in a text file as training data and generates output text similar to the training data. More specifically, this post is ...

Introduction Broadcasting is a fundamental feature in PyTorch that enables element-wise operations between tensors of different shapes. When performing these operations, PyTorch automatically expa...

Introduction The past two posts have laid the groundwork for understanding the mathematical underpinnings of neural networks. In each post, we briefly covered: Gradient and Derivative: The conce...

Introduction In our previous post, we explored the fundamental concept of derivatives and their application in neural networks. We manually performed backpropagation using the chain rule, adjustin...