The function is
f(x,y)=xy(1-8x-7y)=xy-8x^2y-7xy^2
Caculate the partial derivatives
(delf)/(delx)=y-16xy-7y^2
(delf)/(dely)=x-8x^2-14xy
The critical points are
{(y-16xy-7y^2=0),(x-8x^2-14xy=0):}
<=>, {(y(1-16x-7y)=0),(x(1-8x-14y)=0):}
Therefore, (0,0) is a point
<=>, {((16x+7y)=1),((8x+14y)=1):}
<=>, {((16x+7y)=1),((16x+28y)=2):}
<=>, {(16x+7y=1),(y=1/21):}
<=>, {(x=1/24),(y=1/21):}
The other point is (1/24, 1/21)
Calculate the second derivatives
(del^2f)/(delx^2)=-16y
(del^2f)/(dely^2)=-14x
(del^2f)/(delxdely)=1-16x-14y
(del^2f)/(delydelx)=1-16x-14y
Calculate the Determinant D(x,y) of the hessian Matrix
((-16y,1-16x-14y ),(1-16x-14y,-14y))
D(x,y)=224y^2-(1-16x-14y)^2
Therefore,
D(0,0)=-1
As D(0,0)<0, this is a saddle point.
D(1/24,1/21)=0.51-0.11=0.4
D(1/24,1/21)>0, then (del^2f(1/24,1/21))/(delx^2)=-16/21
(del^2f(1/24,1/21))/(delx^2)<0
This is a local maximum at (1/24,1/21)