We start with expanding the binomial:
#int ( e^{2x} + cos(3x) )^2 \ dx =#
#= int e^{4x} \ dx + 2 int e^{2x} cos(3x) \ dx + int cos^2(3x) \ dx =#
#= I_1 + 2 I_2 + I_3#.
Next, we evaluate each of those three integrals. The first of them, #I_1#, is an easy one:
#I_1 = int e^{4x} \ dx = \frac{e^{4x}}{4} + C#.
By means of the double-angle formula for the cosine, #cos(2x) = 2 cos^2(x) - 1#, the third integral, #I_3#, is straightforward as well:
#I_3 = int cos^2(3x) \ dx = \frac{1}{2} int cos(6x) + 1 \ dx = \frac{sin(6x)}{12} + \frac{x}{2} + C#.
The only one that is a bit tricky is integral #I_2#. To solve it, we make use of an iterated application of integration by parts:
#I_2 = int e^{2x} cos(3x) \ dx =#
#= \frac{e^{2x}}{2} cos(3x) + \frac{3}{2} int e^{2x} sin(3x) \ dx =#
#= \frac{e^{2x}}{2} cos(3x) + \frac{3 e^{2x}}{4} sin(3x) - \frac{9}{4} int e^{2x} cos(3x) \ dx =#
#= \frac{e^{2x}}{2} cos(3x) + \frac{3 e^{2x}}{4} sin(3x) - \frac{9}{4} I_2 #.
With this, we arrived at the equation
#I_2 = \frac{e^{2x}}{2} cos(3x) + \frac{3 e^{2x}}{4} sin(3x) - \frac{9}{4} I_2 #,
from which follows that
#I_2 = \frac{4}{13} [ \frac{e^{2x}}{2} cos(3x) + \frac{3 e^{2x}}{4} sin(3x) ] +C = \frac{e^{2x}}{13} [ 2 cos(3x) + 3 sin(3x) ] + C#.
Putting it all together, we finally arrive at the desired result:
#int ( e^{2x} + cos(3x) )^2 \ dx = I_1 + 2 I_2 + I_3 = e^{4x}/4 + \frac{2}{13} e^{2x} [ 2cos(3x) + 3sin(3x)] + sin(6x)/12 + x/2 + C#.
