Q.Differentiate w.r.t. $x$: $\tan^{-1}\left(\dfrac{a\cos x - b\sin x}{b\cos x + a\sin x}\right),\ -\dfrac{\pi}{2} < x < \dfrac{\pi}{2}$ and $\dfrac{a}{b}\tan x > -1$.

Concept: Use the identity $\tan^{-1} u - \tan^{-1} v = \tan^{-1}\left(\frac{u-v}{1+uv}\right)$ to simplify the argument.

Step 1: Divide numerator and denominator of the argument by $b\cos x$ (valid since $\cos x > 0$ in the given interval):

Step 2: Recognise this as $\tan\left(\tan^{-1}\frac{a}{b} - x\right)$, because

Here $\tan\alpha = \frac{a}{b}$ and $\tan\beta = x$, and the condition $\frac{a}{b}\tan x > -1$ ensures the denominator is positive, keeping the angle in the principal branch.

Step 3: Hence the given expression simplifies to: …

We are asked to differentiate

with respect to $x$, under the conditions $-\pi/2 < x < \pi/2$ and $\frac{a}{b}\tan x > -1$.

The direct quotient rule inside an inverse tangent would be messy. Instead, notice the structure: numerator and denominator are linear combinations of $\cos x$ and $\sin x$. This strongly suggests the tangent subtraction formula:

If we can rewrite the fraction inside the $\tan^{-1}$ as $\tan(\text{something} - x)$, the whole expression collapses to a simple linear function.

1. Rewrite the fraction using $\tan$ of a difference

   Divide numerator and denominator by $\cos x$ (valid since $\cos x > 0$ on $(-\pi/2, \pi/2)$):

Now factor $b$ out of the denominator (assuming $b \neq 0$; if $b=0$ the problem trivialises, but the given condition $\frac{a}{b}\tan x > -1$ implies $b \neq 0$):

Write $\frac{a}{b} = \tan \theta$ for some $\theta$ (since $a/b$ is a real constant, we can always set $\theta = \tan^{-1}(a/b)$). Then

This is exactly $\tan(\theta - x)$.

2. Simplify the inverse tangent

   Hence

   $y = \tan^{-1}\bigl(\tan(\theta - x)\bigr) = \theta - x,$ …

Method: Rewrite the Argument as $\tan(\theta - x)$ Using the Tangent Subtraction Formula

Whenever the argument of $\tan^{-1}$ is a ratio of linear combinations of $\sin x$ and $\cos x$ — like $\dfrac{a\cos x-b\sin x}{b\cos x+a\sin x}$ — check whether dividing through by $\cos x$ reveals a tangent-subtraction (or addition) pattern before attempting the quotient rule directly.

Steps

Step 1: Divide numerator and denominator by $\cos x$ (or $b\cos x$)

valid wherever $\cos x\neq 0$, as guaranteed by the given domain.

Step 2: Recognise the constant ratio as the tangent of a fixed angle

Write $\dfrac ab = \tan\theta$ for the constant $\theta=\tan^{-1}(a/b)$. The expression from Step 1 then matches the subtraction formula

Step 3: Apply $\tan^{-1}(\tan\alpha)=\alpha$, using the given condition to confirm the branch …

Common Mistakes

Mistake 1: Ignoring the given domain condition that guarantees the principal branch

Why it's wrong: without confirming that condition, $\tan^{-1}(\tan(\theta-x))$ could in principle differ from $\theta-x$ by $\pm\pi$ — a student who skips this check is only right by luck, and the same shortcut can fail on a differently-worded problem. Correct approach: always translate the problem's stated domain condition into "does the simplified angle stay in the principal branch?" before finalising the simplification.

Mistake 2: Attempting direct quotient-rule and chain-rule differentiation of the raw fraction …