Explain partial derivatives v2

1. $f(x+\alpha u)$ represents the value of the funtion $f$ at the point $x+\alpha u$.

If we wanted to know how much $f$ changes when we move along the direction $u$, starting from the point $x$, we would compute:
$$\lim_{\alpha \rightarrow 0} \dfrac{f(x+\alpha u)-f(x)}{\alpha}$$ Notice that when we compute partial derivatives, we are doing exactly this, only that $u$ in this case would be $(1,\dots,0)$ for the derivative with respect to $x$, or it could be $(0,1,\dots,0)$ for the derivative with respect to $y$, and so on.

2. Notice that $\alpha$ is a scalar that affects all the entries in $x +\alpha u$. For example, in the case of two dimensions, $x +\alpha u = (x_1,x_2)+\alpha(u_1,u_2) = (x_1+\alpha_1 u_1,x_2+\alpha_2 u_2)$

So when we want to understand how $f(x+\alpha u)$ depends on $\alpha$, we must realize that $f$ depends first on its individual entries (so we must compute partial derivatives first), and then compute how does those entries depend on $\alpha$. This is where the expression $$\nabla f (x) \cdot u = u^t \nabla f (x)$$ comes from.

3. The chain rule theorem for functions of several variables states the following:

Let $g: \mathbb{R} \rightarrow \mathbb{R}^n$ and $f: \mathbb{R} ^n \rightarrow \mathbb{R}$ be differentiable functions.

Consider the function $h(\alpha):=f(g(\alpha))$

Then, $h$ is differentiable and we can compute its derivative as $$\dfrac{dh}{d\alpha} = \nabla f(g(\alpha)) \cdot g'(\alpha) $$ This is exactly what is happening above. In our case, $g(\alpha)= x+\alpha u$, for fixed points $x$ and $u$, so that $g'(\alpha) = u$. And this theorem tells us that $f(x+\alpha u)$ is a differentiable function with respect to $\alpha$, and its derivative is given by
$\nabla f(x+\alpha u)\cdot u$.