Q.Differentiate w.r.t. $x$: $\tan^{-1}\left(\dfrac{3a^2 x - x^3}{a^3 - 3ax^2}\right),\ -\dfrac{1}{\sqrt{3}} < \dfrac{x}{a} < \dfrac{1}{\sqrt{3}}$.

Using half-angle identities, $\dfrac{\cos x}{1-\sin x}=\tan\!\left(\tfrac{\pi}{4}+\tfrac{x}{2}\right)$, so its arctangent is $\tfrac{\pi}{4}+\tfrac{x}{2}$.

Main part (a). Convert to half-angles. Using $\cos x=\cos^{2}\tfrac{x}{2}-\sin^{2}\tfrac{x}{2}$ and $\sin x=2\sin\tfrac{x}{2}\cos\tfrac{x}{2}$, and $1=\cos^{2}\tfrac x2+\sin^{2}\tfrac x2$:

$\cos x=\left(\cos\tfrac{x}{2}-\sin\tfrac{x}{2}\right)\left(\cos\tfrac{x}{2}+\sin\tfrac{x}{2}\right),$

$1-\sin x=\cos^{2}\tfrac{x}{2}+\sin^{2}\tfrac{x}{2}-2\sin\tfrac{x}{2}\cos\tfrac{x}{2}=\left(\cos\tfrac{x}{2}-\sin\tfrac{x}{2}\right)^{2}.$

Therefore

$\frac{\cos x}{1-\sin x}=\frac{\cos\tfrac{x}{2}+\sin\tfrac{x}{2}}{\cos\tfrac{x}{2}-\sin\tfrac{x}{2}}=\frac{1+\tan\tfrac{x}{2}}{1-\tan\tfrac{x}{2}}=\tan\left(\frac{\pi}{4}+\frac{x}{2}\right),$

dividing numerator and denominator by $\cos\tfrac x2$ and using $\tan\tfrac{\pi}{4}=1$.

For $-\tfrac{3\pi}{2}<x<\tfrac{\pi}{2}$, $\tfrac{\pi}{4}+\tfrac{x}{2}$ lies in the principal range of $\tan^{-1}$, so

$\tan^{-1}\frac{\cos x}{1-\sin x}=\frac{\pi}{4}+\frac{x}{2}.$

…

Substituting $x=\tan\theta$ simplifies the expression to $\dfrac12\tan^{-1}x$. (OR part: $\cos^{-1}(1/2)=\pi/3$, and $\tan^{-1}(2\sin(\pi/3))=\tan^{-1}\sqrt3=\pi/3$.)

Main question: Simplify $\tan^{-1}\left(\dfrac{\sqrt{1+x^2}-1}{x}\right)$.

Let $x=\tan\theta$ where $\theta\in\left(-\dfrac{\pi}{2},\dfrac{\pi}{2}\right)\setminus\{0\}$. Then $\sqrt{1+x^2}=\sqrt{1+\tan^2\theta}=\sec\theta$ (positive on this interval).

$\frac{\sqrt{1+x^2}-1}{x}=\frac{\sec\theta-1}{\tan\theta}=\frac{\frac{1}{\cos\theta}-1}{\frac{\sin\theta}{\cos\theta}}=\frac{1-\cos\theta}{\sin\theta}$

Using the half-angle identities $1-\cos\theta=2\sin^2(\theta/2)$ and $\sin\theta=2\sin(\theta/2)\cos(\theta/2)$:

$\frac{1-\cos\theta}{\sin\theta}=\frac{2\sin^2(\theta/2)}{2\sin(\theta/2)\cos(\theta/2)}=\tan\left(\frac{\theta}{2}\right)$

…

Substitute $x=\tan\theta$ so both sides reduce to $2\theta$; separately, $\sin^{-1}(-1/2)=-\pi/6$ gives the numeric value $1$.

Proof of $2\tan^{-1}x = \sin^{-1}\dfrac{2x}{1+x^2}$ for $x\in[-1,1]$:

Let $x = \tan\theta$, so $\theta = \tan^{-1}x$. Since $x \in [-1,1]$, we have $\theta \in \left[-\dfrac{\pi}{4}, \dfrac{\pi}{4}\right]$.

Then

$\sin^{-1}\frac{2x}{1+x^2} = \sin^{-1}\frac{2\tan\theta}{1+\tan^2\theta} = \sin^{-1}(\sin 2\theta).$

Since $\theta \in \left[-\dfrac{\pi}{4},\dfrac{\pi}{4}\right]$, we have $2\theta \in \left[-\dfrac{\pi}{2},\dfrac{\pi}{2}\right]$, which is exactly the principal value range of $\sin^{-1}$. So $\sin^{-1}(\sin 2\theta) = 2\theta = 2\tan^{-1}x$. This proves

$2\tan^{-1}x = \sin^{-1}\frac{2x}{1+x^2}.$

Value of $\sin\left(\dfrac{\pi}{3} - \sin^{-1}\left(-\dfrac12\right)\right)$:

$\sin^{-1}\left(-\dfrac12\right) = -\dfrac{\pi}{6}$ (since $\sin(-\pi/6) = -1/2$ and $-\pi/6 \in [-\pi/2,\pi/2]$).

So the expression becomes $\sin\left(\dfrac{\pi}{3} - \left(-\dfrac{\pi}{6}\right)\right) = \sin\left(\dfrac{\pi}{3}+\dfrac{\pi}{6}\right) = \sin\dfrac{\pi}{2} = 1$.

OR: Show $\sin^{-1}\dfrac{12}{13} + \cos^{-1}\dfrac{4}{5} + \tan^{-1}\dfrac{63}{16} = \pi$.

Let $A = \sin^{-1}\dfrac{12}{13}$, so $\sin A = \dfrac{12}{13}$, $\cos A = \dfrac{5}{13}$ (positive, $A$ acute), so $\tan A = \dfrac{12}{5}$.

Let $B = \cos^{-1}\dfrac{4}{5}$, so $\cos B = \dfrac45$, $\sin B = \dfrac35$, so $\tan B = \dfrac34$.

Using the tangent addition formula: …

Find $f(x)=ax+b$ from the given mapping conditions, then verify $\cot(\cot^{-1}7+\cot^{-1}8+\cot^{-1}18)=f(2)=3$ using the cotangent addition formula.

Main question.

Step 1 — Find $f$. $f(x)=ax+b$, $a>0$, maps $[-1,1]$ onto $[0,2]$. Since $a>0$, $f$ is increasing, so $f(-1)=0$ and $f(1)=2$:

$-a+b=0,\qquad a+b=2.$

Adding: $2b=2\Rightarrow b=1$; then $a=1$. So $f(x) = x+1$, and $f(2) = 3$.

Step 2 — Simplify the cotangent sum. Use $\cot(A+B)=\dfrac{\cot A\cot B - 1}{\cot A+\cot B}$.

Let $A=\cot^{-1}7$, $B=\cot^{-1}8$: $\cot A=7,\cot B=8$.

$\cot(A+B) = \frac{7\times8-1}{7+8} = \frac{55}{15} = \frac{11}{3}.$

Now add $C=\cot^{-1}18$ ($\cot C=18$):

$\cot(A+B+C) = \frac{\cot(A+B)\cdot\cot C - 1}{\cot(A+B)+\cot C} = \frac{\frac{11}{3}(18)-1}{\frac{11}{3}+18} = \frac{66-1}{\frac{11+54}{3}} = \frac{65}{\frac{65}{3}} = 3.$

So $\cot(\cot^{-1}7+\cot^{-1}8+\cot^{-1}18) = 3 = f(2)$, as required.

OR — Find $\cos^{-1}x+\cos^{-1}\left\{\tfrac12(x+\sqrt3\sqrt{1-x^2})\right\}$ for $\tfrac12\le x\le1$.

Let $x=\cos\theta$, with $\theta=\cos^{-1}x\in[0,\pi/3]$ (since $x\in[\tfrac12,1]$).

…

Substitute $x=\cos 2\theta$; then $\sqrt{1\pm x}$ become $\sqrt2\cos\theta$ and $\sqrt2\sin\theta$, and the ratio becomes $\tan(\tfrac\pi4-\theta)$.

Let $x = \cos 2\theta$, so that $\theta = \tfrac{1}{2}\cos^{-1}x$. Using $1+\cos 2\theta = 2\cos^2\theta$ and $1-\cos 2\theta = 2\sin^2\theta$:

$\sqrt{1+x} = \sqrt{2\cos^2\theta} = \sqrt{2}\,\cos\theta$ and $\sqrt{1-x} = \sqrt{2\sin^2\theta} = \sqrt{2}\,\sin\theta$ (both non-negative for the given range of $x$).

Substitute into the argument:

$\dfrac{\sqrt{1+x}-\sqrt{1-x}}{\sqrt{1+x}+\sqrt{1-x}} = \dfrac{\sqrt2(\cos\theta-\sin\theta)}{\sqrt2(\cos\theta+\sin\theta)} = \dfrac{\cos\theta-\sin\theta}{\cos\theta+\sin\theta}$.

Divide numerator and denominator by $\cos\theta$:

$= \dfrac{1-\tan\theta}{1+\tan\theta} = \tan\!\left(\dfrac{\pi}{4} - \theta\right)$.

Therefore the left side $= \tan^{-1}\!\left[\tan\!\left(\dfrac{\pi}{4}-\theta\right)\right] = \dfrac{\pi}{4} - \theta = \dfrac{\pi}{4} - \dfrac{1}{2}\cos^{-1}x$.

…