Q.$\cos(\sec^{-1}x + \csc^{-1}x) =$

The inverse cosine function requires its argument to lie in $[-1, 1]$. Solving $-1 \le 2x - 5 \le 1$ gives the domain $[2, 3]$.

Since $|\csc\theta|\ge 1$ always, its inverse function is defined only for inputs with absolute value at least $1$.

The cosecant function $\csc\theta = \dfrac{1}{\sin\theta}$ satisfies $|\csc\theta| \ge 1$ for all $\theta$ (except where $\sin\theta=0$), since $|\sin\theta|\le1$.

…

Unlike an odd function, $\sec^{-1}$ is NOT odd on its restricted range; the correct identity is $\sec^{-1}(-x)=\pi-\sec^{-1}x$.

The principal value branch of $\sec^{-1}$ is $[0,\pi]-\{\pi/2\}$. Let $\sec^{-1}x=\theta$, so $\sec\theta=x$ with $\theta \in [0,\pi]$.

…

$\text{cosec}^{-1}$ is defined only where $|x|\ge1$, since $\text{cosec}\,\theta$ never takes values strictly between $-1$ and $1$.

$\text{cosec}\,\theta = \dfrac{1}{\sin\theta}$, and since $-1\le\sin\theta\le1$ (with $\sin\theta\neq0$), $\text{cosec}\,\theta$ can never lie strictly between $-1$ and $1$.

…

Reciprocal identity: $\operatorname{cosec}^{-1}x=\sin^{-1}\dfrac{1}{x}$ for $|x|\ge1$.

If $\theta=\operatorname{cosec}^{-1}x$ then $\operatorname{cosec}\theta = x$, i.e. $\sin\theta = \dfrac{1}{x}$, so $\theta = \sin^{-1}\dfrac{1}{x}$.

…

sec⁻¹(2/√3) = π/6, since sec(π/6) = 2/√3 and π/6 lies in the principal range.

The principal value branch of sec⁻¹ is [0, π] − {π/2}.

We need θ in this range such that sec θ = 2/√3, i.e. cos θ = √3/2.

…

The principal value branches of the inverse trig functions are fixed by convention (NCERT).

By definition, the principal value branches of the inverse trigonometric functions are:

$\sin^{-1}: \left[-\dfrac{\pi}{2},\dfrac{\pi}{2}\right],\quad \cos^{-1}:[0,\pi],\quad \tan^{-1}:\left(-\dfrac{\pi}{2},\dfrac{\pi}{2}\right)$

$\csc^{-1}:\left[-\dfrac{\pi}{2},\dfrac{\pi}{2}\right]-\{0\},\quad \sec^{-1}:[0,\pi]-\left\{\dfrac{\pi}{2}\right\},\quad \cot^{-1}:(0,\pi)$

…

$\sec^{-1}x+\csc^{-1}x=\tfrac{\pi}{2}$, and $\sin\tfrac{\pi}{2}=1$.

For all $|x|\ge 1$ the complementary identity holds: $\sec^{-1}x+\csc^{-1}x=\dfrac{\pi}{2}$.

…

$\cos(\sec^{-1}x + \csc^{-1}x) = \cos\dfrac{\pi}{2} = 0$.

A standard complementary identity of inverse trig functions states, for $|x|\ge 1$:

$\sec^{-1}x + \csc^{-1}x = \dfrac{\pi}{2}.$

…

Range $=[0,\pi]-\left\{\dfrac{\pi}{2}\right\}$.

Concept. For each inverse trig function we fix a principal branch so that it becomes a genuine (single-valued) function.

…

$\sec^{-1}\!\big(\tfrac{2}{\sqrt3}\big)=\dfrac{\pi}{6}$ — option (iv).

Concept. The principal value branch of $\sec^{-1}$ is $[0,\pi]\setminus\{\tfrac{\pi}{2}\}$. We seek the unique $\theta$ in this range with $\sec\theta=\tfrac{2}{\sqrt3}$.

Why. $\sec\theta=\dfrac{1}{\cos\theta}$, so $\sec\theta=\tfrac{2}{\sqrt3}\iff\cos\theta=\tfrac{\sqrt3}{2}$.

Steps.

$\cos\theta=\frac{\sqrt3}{2}\implies \theta=\frac{\pi}{6}.$ …