Q.If $\cot^{-1}2$ and $\cot^{-1}3$ are two angles of a triangle, then the third angle is

These are the working-formula properties (Properties VI–X) for combining or composing inverse trig functions, together with the reference-triangle technique for a raw composite like $\tan(\sin^{-1}x)$.

Composing a trig function with an unrelated inverse trig function (reference triangle). To evaluate $f(g^{-1}(x))$ where $f\ne g$ (e.g. $\cot(\sin^{-1}x)$), let $\theta=g^{-1}(x)$, build a right triangle encoding $\theta$ from the definition of $g^{-1}$ (e.g. $\sin\theta=x$ gives opposite $=x$, hypotenuse $=1$, so adjacent $=\sqrt{1-x^2}$ by Pythagoras — taking the adjacent side non-negative since $\theta\in\left[-\tfrac{\pi}2,\tfrac{\pi}2\right]$ keeps cosine $\ge0$ there), then read $f(\theta)$ straight off the triangle. This proves, e.g., $\tan(\sin^{-1}x)=\dfrac{x}{\sqrt{1-x^2}}$, $-1<x<1$.

Property VI (addition/subtraction formulas).

with analogous subtraction forms for sine and cosine. Extending the tangent addition formula to three terms gives $\tan^{-1}x+\tan^{-1}y+\tan^{-1}z=\tan^{-1}\left[\dfrac{x+y+z-xyz}{1-xy-yz-zx}\right]$; setting the left side equal to $\pi$ and taking the tangent of both sides (which is $0$) proves the classical identity $x+y+z=xyz$ whenever $\tan^{-1}x+\tan^{-1}y+\tan^{-1}z=\pi$.

Property VII (double-angle-style formulas, from setting $y=x$ in Property VI).

Property VIII. $\sin^{-1}(2x\sqrt{1-x^2})=2\sin^{-1}x$ for $|x|\le\tfrac1{\sqrt2}$, and $\sin^{-1}(2x\sqrt{1-x^2})=2\cos^{-1}x$ for $\tfrac1{\sqrt2}\le x\le1$ (the SAME left side splits into two different right sides depending on which half of $[-1,1]$, i.e. which principal-range piece, $x$ falls in). …