nonlinear system 

(a) You have
$$x^T( P\cdot \partial_x f(\sigma x)+ (\partial_x f(\sigma x))^T \cdot P)x ≤ x^T(-I)x=-x^Tx$$
from your inequality. Now put an integral of $\int_0^1 d\sigma$ on both sides to recover:
$$x^T P \cdot \left(\int_0^1 \partial_xf(\sigma x)xd\sigma\right) +\left(\int_0^1\partial_x f(\sigma x) xd\sigma\right)^T\cdot Px ≤ -x^Tx\int_0^1d\sigma$$
evaluating the integral with the hint gives
$$x^TPf(x)+f(x)^TPx ≤-x^Tx$$
(b) Since $P$ is positive you have $f(x)^TPf(x)≥0$ and $f(x)^TPf(x)=0$ if and only if $f(x)=0$. Now this is true for $x=0$. However if $x\neq0$ and $f(x)=0$ you get from the inequality of (a) that:
$$0=x^TP0+0Px ≤ - x^Tx$$
this is only possible for $x=0$. So for all other $x$ you get $f(x)\neq0$ and then $V(x)>0$.

The same kind of argument shows radially unbounded. Doing this concretely:

Modify the inequality of (a) by dividing by $x^Tx = \|x\|^2$. Then you get:
$$(\frac{x}{\|x\|})^TP \frac{f(x)}{\|x\|}+(\frac{f(x)}{\|x\|})^TP\frac{x}{\|x\|}≤-1$$
Now the left-hand side has an absolute value smaller than:
$$2\|P\|_\infty \cdot \frac{\|f(x)\|}{\|x\|}\frac{\|x\|}{\|x\|}$$

where $\|P\|_\infty$ is the operator norm of $P$ (ie its larges eigenvalue). This has to be larger than $1$ (since the expression is more negative than $-1$). This is only possible if $\frac{\|f(x)\|}{\|x\|}\not\to0$, ie if $\|f(x)\|\to\infty$ as $\|x\|\to\infty$. This means that $V$ is radially unbounded.

(c) You have
$$\nabla V^T \cdot f= f^T P (\partial_x f) f + f^T(\partial_x f)^T P f= f^T(P\partial_x f + \partial_x f^T P)f≤ -f^Tf$$
(everything evaluated at $x$) whence $\nabla V^T\cdot f <0$ whenever $x\neq0$.

This is the last condition needed to verify global asymptotic stability. See eg these lecture notes for the condition (2nd page): https://inst.eecs.berkeley.edu/~ee222/sp16/LectureNotes/Lecture8.pdf