This page was last edited on 12 November 2025
Weight initialization is the process of setting the initial values for the weights in a neural network before training begins.
These weights determine how the network processes inputs and learns over time.
In Deep Reinforcement Learning(RL), weight initialization affects how well the agent learns the value function or policy.
Why do we use weight initialization in Deep RL?
If weights are not initialized properly:
- The model might learn too slowly or not at all.
- Gradients can vanish or explode.
- The agent can get stuck in poor local optima.
Good neural network weight initialization helps stabilize learning and allows the agent to converge faster to a good solution.
Equation(s) for weight initialization
Two common initialization methods:
Xavier/Glorot Initialization:
![\[ \hspace{5mm} \fbox{ \begin{array}{c} \vspace{5mm} \\ \displaystyle W \sim \mathcal{U} \left( -\sqrt{\frac{6}{n_{\text{in}} + n_{\text{out}}}}, \, \sqrt{\frac{6}{n_{\text{in}} + n_{\text{out}}}} \right) \\ \vspace{5mm} \end{array} } \hspace{5mm} \]](https://www.reinforcementlearningpath.com/wp-content/ql-cache/quicklatex.com-3f0a7c8be8c1b7964eee2d8f62557572_l3.png "Rendered by QuickLaTeX.com")
He Initialization (for ReLU):
![\[ \hspace{5mm} \fbox{ \begin{array}{c} \vspace{5mm} \\ \displaystyle W \sim \mathcal{N} \left( 0, \sqrt{ \frac{2}{n_{\text{in}}} } \right) \\ \vspace{5mm} \end{array} } \hspace{5mm} \]](https://www.reinforcementlearningpath.com/wp-content/ql-cache/quicklatex.com-14e093e5b1ca31883572616eb78babac_l3.png "Rendered by QuickLaTeX.com")
Where:
- W: weight
- nin: number of input units
- nout: number of output units
- U: uniform distribution
- N: normal distribution
Each method aims to keep the variance of activations stable across layers.
ANALOGY
Imagine we are on a road trip. If our GPS starts 100 km off course, it will struggle to find the right path. That’s like bad weight initialization—our network starts far from the solution.
But good initialization is like starting at the correct location. We still need to drive (learn), but we’re already on the right road.
Implementation steps for weight initialization
- Choose the type of activation function (e.g., ReLU, Tanh).
- Select the right initialization method (e.g., He for ReLU, Xavier for Tanh).
- For each layer:
- Get nin, nout.
- Generate weights using the formula.
- Set biases (often to zero or small constant).
- Apply this before training starts.
Most Deep Learning frameworks (like PyTorch or TensorFlow) do this automatically, but it’s good to understand the logic.
References:
- Glorot, X., & Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks.
- He, K., Zhang, X., Ren, S., & Sun, J. (2015). Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification.
Backpropagation << Previous | Next >> Gradient Descent