joint continuous probability function finding covariance

Here we'd like to compute $\mathrm{Cov}(X, Y)$, which we recall is defined as:

$$\mathrm{Cov}(X, Y) = E[XY] - E[X]E[Y]$$

The joint density of this is given by:

$$f(x, y) = \frac{8xy}{3} \qquad 0<x<1 \qquad x<y<2x$$

The key here is to integrate properly over this region, which we can do by first integrating $y$ from $x$ to $2x$ and then from $x$ from 0 to 1. To compute the covariance, we compute the three expectations:
$$E[XY] = \int_0^1 \int_x^{2x} xy  \frac{8xy}{3}\,dy\,dx = \frac{28}{27}$$
$$E[X] = \int_0^1 \int_x^{2x} x \frac{8xy}{3}\,dy\,dx = \frac{4}{5}$$
$$E[Y] = \int_0^1 \int_x^{2x} y \frac{8xy}{3}\,dy\,dx = \frac{56}{45}$$

Therefore our covariance is:

$$\mathrm{Cov}(X, Y) = \frac{28}{27} - \frac{4}{5}\frac{56}{45} = \frac{28}{675}$$

So our final covariance is 28/675 or approximately 0.0415.