You need to take the derivative of your function and evaluate it at the given point to find the slope of the tangent line to it at that point:
#y=f(x)=x^2cosx+sinx#
#m_1=dy/dx=2xcosx-x^2sinx+cosx#
at #x=-pi/3#:
#m_1=2(-pi/3)cos(-pi/3)-(-pi/3)^2sin(-pi/3)+cos(-pi/3)#
#m_1=2(-pi/3)(1/2)-(pi^2/9)(-sqrt3/2)+1/2#
#m_1=-pi/3+(pi^2sqrt3)/18+1/2#
#m_1=(pi^2sqrt3-6pi+9)/18#
If #m_2#=slope of the line normal to the curve we can say:
#m_1m_2=-1# or #m_2=-1/m_1#. So in this case:
#m_2=-18/(pi^2sqrt3-6pi+9)#
Now we can write the equation of the normal line to the curve using the standard equation of a straight line with the above slope. Let's first figure out the #y# -coordinate of the point at #x=-pi/3#:
#y=(-pi/3)^2cos(-pi/3)+sin(-pi/3)=(pi^2/9)(1/2)-sqrt3/2=pi^2/18-sqrt3/2=(pi^2-9sqrt3)/18#
#y=mx+b#
#y=-18/(pi^2sqrt3-6pi+9)x+b#
Now we have to find the value of #b# by plugging in the coordinates of the tangent / normal point:
#(pi^2-9sqrt3)/18=-18/(pi^2sqrt3-6pi+9)(-pi/3)+b#
#b=(pi^2-9sqrt3)/18-(6pi)/(pi^2sqrt3-6pi+9)=((pi^2-9sqrt3)(pi^2sqrt3-6pi+9)-108pi)/(18(pi^2sqrt3-6pi+9)#
#b=(pi^4sqrt3-6pi^3+9pi^2-27pi^2+54sqrt3pi-81sqrt3-108pi)/(18(pi^2sqrt3-6pi+9)#
#b=(pi^4sqrt3-6pi^3-18pi^2+(54sqrt3-108)pi-81sqrt3)/(18(pi^2sqrt3-6pi+9)#
The equation of normal line is:
#y=-18/(pi^2sqrt3-6pi+9)x+(pi^4sqrt3-6pi^3-18pi^2+(54sqrt3-108)pi-81sqrt3)/(18(pi^2sqrt3-6pi+9)#
#(pi^2sqrt3-6pi+9)y=-18x+(pi^4sqrt3-6pi^3-18pi^2+(54sqrt3-108)pi-81sqrt3)/18#
