I am trying to use fmincon to find maxmimize sqrt(fx^2+fy^2).
I am attaching my code that I have so far that defines all of the things I am trying to do. For my definition, fx is the first row in H and fy is the second row in H.
My constraints would be fy/fx=tan(35) and 0<m<1. I am not sure how to incorporate those into the fmincon function.
Any help would be greatly appreciated.