The first term (which must equal 27) in the said development is the constant term (i.e., the term(s) without x, or x to the power 0). We get it when x is equal to zero: then all other terms, i.e., higher powers in x, will vanish (i.e., become zero).
In that case we get $ (a+0)^3 \, (1-0)^5 = a^3 $ in the first expression, which must equal 27: the only solution is $ a = 3 $. (If you don't know $ 3^3 = 27$ by heart, you can take the third root using a calculator, or you can factor 27 by trial division: it's not divisible by 2, but indeed by 3 which gives 9, which is again divisible by 3 (only), and you finally have 27 = 3 x 3 x 3.)
To get the terms in higher powers, you could either develop the two powers, and then the product of the two, up to $x^2$ (higher powers won't contribute to the first three terms of the considered development) interest us, so you can just write $+o(x^2)$ for "terms negigible w.r.t. $x^2$" for those terms in each of the steps).
But it might be simpler to take the derivative w.r.t.(= with respect to) $x$ and then set $x=0$. That gives the coefficient of the second term in the development, i.e., $b$, as you can easily see by doing that (derivative and $x=0$) with the polynomial $27+bx+cx^2$.
To do this with the product, you have to use the product or Leibniz rule. Writing $(...)'$ for $\frac d{dx}(...)$, we have:
$ ( A^3 \, B^5)' = 3\,A^2\,A' \times B^5 + A^3\times 5\,B^4\,B' =: f'(x)$
where $A = (a+x) ~~\Rightarrow~~A' = 1$ and $ B = 1 - x/3 ~~\Rightarrow~~ B' = -1/3$.
Setting finally $x=0$ the above becomes
$ f'(x=0) = 3 \, a^2 \, 1 \times 1^5 + a^3\times 5\,(1+0)^4\,(-1/3) = 3\times 9 - 27 \times 5/3 = 9\times (3 - 5) = -18 $.
This is the coefficient of $x$, i.e., $b$ by definition: $b = -18$.
We can do the same to get the coefficient $c$ of $x^2$: taking one more derivative and setting again $x = 0$, we'll get twice that coefficient (because $(x^2)'' = (2x)' = 2$).
Using again the product rule and the fact that $A'$, $B'$ are constant, i.e., $A''=B''=0$, we get
$f''(x) = 3\,(2\,A\,(A')^2)\times B^5 + 3\,A^2\,A'\times 5\,B^4\,B'\times 2 + 3\,A^2\,A'\times 5\times 4\,B^3\,(B')^2.$
(The $\times 2$ is there because when the derivative hits the second part of the first term that gives the same as when it hits the first part of the second term.)
So, (remember $a = 3$ !)
$2\,c = f''(x=0) = 6 \,a\times 1^2\times 1^5 + 30\,a^2\times 1^4\,(-1/3) + 60\,a^2\times 1^3\, (-1/3)^2 = 18 - 90 + 60 = -12$
or $c = -6$.
Note: You can check that this is correct by asking WolframAlpha:
https://www.wolframalpha.com/input?i=series%28%28a+%2Bx%29%5E3*%281-x%2F3%29%5E5%29
It will show you $a^3$ (which must equal 27) for the first term and more complicated stuff for the second and third term; if you replace $a = 3$ then you'll see 27, -18 and -6 for the constant term and coefficients of $x$ and $x^2$.