Add 2xyi = 16i to the first equation and get (x+iy)^2 = 64 + 16i. Compute the root in the ordinary way, separate real and imaginary part, identify x and y and you have the desired result.
That was my idea too, complex numbers make it very simple, as you said, you can add 2ixy to the first line, get the value of (x+iy)^2 and find the square roots of this number then get x and y
Well, way too long. Replace 64 by 8xy in the first equation. Put y = kx. This gives k*k +8k 1 = 0. For real solutions of x,y, k = sqrt(17) - 4. Replace in the second gives x = +or 8.061
Nice, but: when you present a problem like this, you should state right at the beginning that you are looking for real solutions. You spend a lot of time on trivialities (like a^m*b^m=(a*b)^m),...), but no tile at all explaining the motivation for the non-obvious steps, e.g. what is the motivation of squaring both sides of the first equation. If you want your students to learn "entrance exam tricks", then you should point out what the tricks are... Keep up the good work.