Q.Find the value of $k$ for which the equation $(k+1)x^2+4xy+(k-1)y^2=0$ represents a pair of perpendicular straight lines.

Compare with $ax^2+2hxy+by^2=0$ and apply $\tan\theta=\dfrac{2\sqrt{h^2-ab}}{|a+b|}$ to get $\theta=\tan^{-1}\!\left(\dfrac{\sqrt{33}}{5}\right)$.

The given equation $x^2-7xy+4y^2=0$ is a homogeneous second-degree equation representing a pair of straight lines through the origin. Writing it as $ax^2+2hxy+by^2=0$:

$a=1,\qquad 2h=-7\Rightarrow h=-\tfrac{7}{2},\qquad b=4.$

The angle $\theta$ between the pair of lines is

$\tan\theta=\frac{2\sqrt{h^2-ab}}{|a+b|}.$

Compute $h^2-ab=\dfrac{49}{4}-4=\dfrac{49-16}{4}=\dfrac{33}{4}$, so $\sqrt{h^2-ab}=\dfrac{\sqrt{33}}{2}$.

…

Using $\tan\theta = \dfrac{2\sqrt{h^2 - ab}}{a+b}$ with $a=1,\,h=-\tfrac72,\,b=4$ gives $\tan\theta = \dfrac{\sqrt{33}}{5}$, so the answer is option (a).

For a pair of straight lines through the origin $ax^2 + 2hxy + by^2 = 0$, the angle $\theta$ between them is

$\tan\theta = \left|\frac{2\sqrt{h^2 - ab}}{a + b}\right|.$

Compare $x^2 - 7xy + 4y^2 = 0$ with $ax^2 + 2hxy + by^2 = 0$:

$a = 1,\quad 2h = -7 \Rightarrow h = -\tfrac{7}{2},\quad b = 4.$

Compute $h^2 - ab$:

$h^2 - ab = \frac{49}{4} - (1)(4) = \frac{49}{4} - \frac{16}{4} = \frac{33}{4}.$

So …