Here is a more direct way of seeing it. Instead of working with the Heaviside function, consider the function
Now, this is a nice, smooth function for finite $\epsilon$. But if you start to make $\epsilon$ smaller (try plotting it for values like $\epsilon=1$ and $\epsilon=0.1$) you'll see that it looks more and more like the Heaviside step function. Taking the derivative with respect to $x$ gives
which, as expected, looks more and more like the delta function for smaller and smaller values of $\epsilon$. In fact, these functions give nice representations of the step and delta functions, respectively, in the $\epsilon \to 0$ limit.
This representation of the step function has a "jump" of 1 at $x=0$, so to get a jump of $\alpha$ you could start with
Taking the derivative with respect to $x$ explains why you get $\alpha\,\delta(x)$.
This post imported from StackExchange Mathematics at 2014-06-09 19:12 (UCT), posted by SE-user Robert McNees