ODEs - Stability

Define
$$G(y_1, y_2) = (y_2 - y_1 f(y_1, y_2), -y_1 - y_2f(y_1, y_2)).$$
Let us compute the Jacobian of  $G$ at $(0, 0)$. It is
$$\mathbf{J}_G = \begin{bmatrix}-f(0,0) & 1\\ -1 & - f(0,0)\end{bmatrix}.$$
Then $\det \mathbf{J}_G = f(0,0)^2 + 1 > 0$. Thus the system is stable if $\operatorname{tr} \mathbf{J}_G < 0$ and unstable if $\operatorname{tr} \mathbf{J}_G > 0$. We easily compute that $\operatorname{tr} \mathbf{J}_G = -2f(0,0)$.

Then if $f$ is positive in a neighborhood of $(0,0)$, one must have $\operatorname{tr} \mathbf{J}_G < 0$ so the system is stable. If $f$ is negative in a neighborhood of $(0,0)$, then $\operatorname{tr} \mathbf{J}_G > 0$ so the system is unstable.