I = int \ x tan^(-1) x \ dx
we'll use IBP
before we go, note that d/dx (tan^(-1) x) = 1/(1+x^2), a well known result
So
I = int \ d/dx(x^2/2) tan^(-1) x \ dx
and by IBP
I = x^2/2 tan^(-1) x - int \ x^2/2 d/dx( tan^(-1) x) \ dx
= x^2/2 tan^(-1) x - 1/2 int \ x^2 1/(x^2 +1 ) \ dx
= x^2/2 tan^(-1) x - 1/2 int \ (x^2+ 1 - 1)/(x^2 +1 ) \ dx
= x^2/2 tan^(-1) x - 1/2 int \ 1 - 1/(x^2 +1 ) \ dx
= x^2/2 tan^(-1) x - 1/2 int \ 1 - d/dx (tan^(-1) x) \ dx
= x^2/2 tan^(-1) x - 1/2x + 1/2 tan^(-1) x + C
=1/2( x^2 tan^(-1) x + tan^(-1) x - x)+ C