Deep reinforcement learning (RL) is a powerful framework for sequential decision making. Despite many high-profile successes in recent years, deep RL is known to be brittle. An emerging research direction into the causes of this brittleness concerns the loss of plasticity of neural networks. However, plasticity loss has so far been understudied in stochastic environments. Our study addresses this issue and provides a principled analysis of the interplay of noise and plasticity loss. We find that plasticity-preserving methods increase learning robustness in stochastic environments, with their advantage growing under higher observation noise, while combining resets with layer normalization is more often detrimental than beneficial. Further, stochasticity can accelerate the loss of plasticity in harder environments, whereas in easier settings noise can act as a regularizer that sustains plasticity. Lastly, our study shows that plasticity loss is most pronounced in early stages of training, and that in the most challenging stochastic settings TD instability emerges as a more critical failure mode which is more likely to be triggered by the presence of noise.