Q.Find the angle between two vectors $\vec{a}$ and $\vec{b}$ with magnitudes $\sqrt{3}$ and $2$, respectively having $\vec{a} \cdot \vec{b}=\sqrt{6}$.

Showing the 12 most recent of 46 on this concept.

1. AP EAPCET 2021Set eng-2021-08-23-FN1 markMCQ
   Q.The angle between the planes $2x - y + z = 6$ and $x + y + 2z = 3$ is ______ (A) $\dfrac{\pi}{3}$ (B) $\cos^{-1}\left(\dfrac{1}{6}\right)$ (C) $\dfrac{\pi}{4}$ (D) $\dfrac{\pi}{6}$

   ›Reveal solutionSolution

   Tests finding the angle between two planes via the angle between their normal vectors.

   Concept and Intuition

   The angle between two planes equals the angle between their normal vectors (up to supplementary ambiguity, resolved by taking the acute angle). If a plane is $Ax+By+Cz=D$, its normal vector is $(A,B,C)$, and the angle between two normals is found using the dot-product formula.

   Step-by-Step Solution

   1. Plane 1: $2x-y+z=6$, normal $n_1=(2,-1,1)$.
   2. Plane 2: $x+y+2z=3$, normal $n_2=(1,1,2)$.
   3. $n_1\cdot n_2 = 2(1)+(-1)(1)+1(2)=2-1+2=3$.
   4. $|n_1|=\sqrt{4+1+1}=\sqrt6$, $|n_2|=\sqrt{1+1+4}=\sqrt6$.
   5. $\cos\theta=\dfrac{3}{\sqrt6\cdot\sqrt6}=\dfrac{3}{6}=\dfrac12$.
   6. So $\theta=\cos^{-1}\left(\dfrac12\right)=\dfrac{\pi}{3}$.

   Common Mistakes

   • Forgetting to normalize (divide by the magnitudes), landing on the wrong cosine value.
   • Not recognizing $\cos^{-1}(1/2)=\pi/3$ is a standard angle, and instead leaving the answer in an unsimplified inverse-cosine form (which happens to also appear as a distractor option (B) computed for different vectors).

   ✓Final answer

   The correct option is (A) — $\dfrac{\pi}{3}$.

   ANSWER: A

2. AP EAPCET 2023Set eng-2023-05-19-FN1 markMCQ
   Q.Let $\vec{a},\vec{b},\vec{c}$ be three vectors such that $\vec{a}$ is perpendicular to $\vec{b}$ and $\vec{b}$ is perpendicular to $\vec{c}$. If $|\vec{a}|=2, |\vec{b}|=3, |\vec{c}|=5$ and $|\vec{a}+\vec{b}+\vec{c}|=4\sqrt3$, then the angle between $\vec{a}$ and $\vec{c}$ is (A) $\cos^{-1}\left(\dfrac{2}{5}\right)$ (B) $\dfrac{\pi}{3}$ (C) $\cos^{-1}\left(\dfrac{2}{3}\right)$ (D) $\dfrac{\pi}{6}$

   ›Reveal solutionSolution

   Expand $|\vec a+\vec b+\vec c|^2$; the perpendicularity conditions kill two of the three cross terms, leaving $\vec a\cdot\vec c$ to solve for. Answer: (B).

   Concept and Intuition

   Squaring a vector sum brings out all pairwise dot products; when some pairs are given as perpendicular, those dot-product terms vanish, isolating the one unknown dot product — here $\vec a\cdot\vec c$, which directly gives the angle between $\vec a$ and $\vec c$.

   Step-by-Step Solution

   1. $|\vec a+\vec b+\vec c|^2=|\vec a|^2+|\vec b|^2+|\vec c|^2+2\vec a\cdot\vec b+2\vec b\cdot\vec c+2\vec a\cdot\vec c$.
   2. Since $\vec a\perp\vec b$, $\vec a\cdot\vec b=0$; since $\vec b\perp\vec c$, $\vec b\cdot\vec c=0$.
   3. So $|\vec a+\vec b+\vec c|^2=4+9+25+2\vec a\cdot\vec c=38+2\vec a\cdot\vec c$.
   4. Given $|\vec a+\vec b+\vec c|=4\sqrt3\Rightarrow|\vec a+\vec b+\vec c|^2=48$. So $38+2\vec a\cdot\vec c=48\Rightarrow\vec a\cdot\vec c=5$.
   5. $\cos\theta=\dfrac{\vec a\cdot\vec c}{|\vec a||\vec c|}=\dfrac{5}{2\times5}=\dfrac12\Rightarrow\theta=\dfrac{\pi}{3}$.

   Common Mistakes

   • Forgetting to double the cross terms when expanding $|\vec a+\vec b+\vec c|^2$.
   • Assuming $\vec a\perp\vec c$ as well just because both are perpendicular to $\vec b$ — that's not implied in 3D.

   ✓Final answer

   The correct option is (B) — $\dfrac{\pi}{3}$.

   ANSWER: B

3. AP EAPCET 2025Set eng-2025-05-21-AN1 markMCQ
   Q.$\bar{a},\bar{b},\bar{c}$ are three vectors such that $|\bar{a}|=2$, $|\bar{b}|=3$, $|\bar{c}|=5$, $|\bar{a}+\bar{b}+\bar{c}|=\sqrt{69}$. If $(\bar{a},\bar{b})=(\bar{b},\bar{c})=\dfrac{\pi}{3}$ then $(\bar{c},\bar{a})=$ (A) $\dfrac{\pi}{6}$ (B) $\dfrac{\pi}{4}$ (C) $\dfrac{\pi}{3}$ (D) $\dfrac{\pi}{2}$

   ›Reveal solutionSolution

   Expanding $|\bar a+\bar b+\bar c|^2$ and using the two known angles isolates $\bar c.\bar a$, giving the third angle as $\frac{\pi}{3}$.

   Concept and Intuition

   The squared magnitude of a vector sum expands into the sum of squared magnitudes plus twice the pairwise dot products. With two of the three pairwise angles already known, this single scalar equation is enough to solve for the third dot product — and hence the third angle.

   Step-by-Step Solution

   1. Expand: $|\bar a+\bar b+\bar c|^2=|\bar a|^2+|\bar b|^2+|\bar c|^2+2(\bar a.\bar b+\bar b.\bar c+\bar c.\bar a)$.
   2. Substitute known magnitudes: $69 = 4+9+25+2(\bar a.\bar b+\bar b.\bar c+\bar c.\bar a) = 38+2(\ldots)$.
   3. So $\bar a.\bar b+\bar b.\bar c+\bar c.\bar a=\dfrac{69-38}{2}=\dfrac{31}{2}=15.5$.
   4. Compute $\bar a.\bar b=|\bar a||\bar b|\cos\dfrac{\pi}{3}=2\cdot3\cdot\dfrac12=3$.
   5. Compute $\bar b.\bar c=|\bar b||\bar c|\cos\dfrac{\pi}{3}=3\cdot5\cdot\dfrac12=7.5$.
   6. So $\bar c.\bar a=15.5-3-7.5=5$.
   7. Also $\bar c.\bar a=|\bar c||\bar a|\cos\theta=5\cdot2\cdot\cos\theta=10\cos\theta$, so $10\cos\theta=5 \Rightarrow \cos\theta=\dfrac12$.
   8. Therefore $\theta=\dfrac{\pi}{3}$.

   Common Mistakes

   • Forgetting the factor of 2 in the cross-term expansion of $|\bar a+\bar b+\bar c|^2$.
   • Mixing up which pair of vectors' angle is asked (here $(\bar c,\bar a)$, not $(\bar a, \bar c)$ reversed with a different sign — dot product is symmetric so this doesn't actually matter, but it's easy to substitute the wrong known angle by mistake).

   ✓Final answer

   The correct option is (C) — $\dfrac{\pi}{3}$.

   ANSWER: C

4. AP EAPCET 2024Set eng-2024-05-18-FN1 markMCQ
   Q.Let $\bar{a}, \bar{b}$ be two unit vector. If $\bar{c} = \bar{a} + 2\bar{b}$ and $\bar{d} = 5\bar{a} - 4\bar{b}$ are perpendicular to each other, then the angle between $\bar{a}$ and $\bar{b}$ is (A) $\dfrac{\pi}{6}$ (B) $\dfrac{\pi}{4}$ (C) $\dfrac{\pi}{3}$ (D) $\dfrac{\pi}{8}$

   ›Reveal solutionSolution

   Expand the perpendicularity condition $\bar c\cdot\bar d=0$ to isolate $\bar a\cdot\bar b$.

   Concept and Intuition

   Two vectors are perpendicular exactly when their dot product is zero. Expanding the dot product of linear combinations of unit vectors reduces everything to the single unknown $\bar a\cdot\bar b=\cos\theta$.

   Step-by-Step Solution

   1. $\bar c\cdot\bar d = (\bar a+2\bar b)\cdot(5\bar a-4\bar b) = 5(\bar a\cdot\bar a) -4(\bar a\cdot\bar b) + 10(\bar b\cdot\bar a) -8(\bar b\cdot\bar b)$.
   2. Since $|\bar a|=|\bar b|=1$: $=5(1) + 6(\bar a\cdot\bar b) - 8(1) = 6(\bar a\cdot\bar b) - 3$.
   3. Set to zero: $6(\bar a\cdot\bar b)=3\Rightarrow \bar a\cdot\bar b = \dfrac12$.
   4. Since both are unit vectors, $\bar a\cdot\bar b=\cos\theta = \dfrac12 \Rightarrow \theta = \dfrac{\pi}{3}$.

   Common Mistakes

   • Sign errors when combining the $-4$ and $+10$ cross terms (they add, not cancel).
   • Forgetting $|\bar a|=|\bar b|=1$ so $\bar a\cdot\bar a=\bar b\cdot\bar b=1$.

   ✓Final answer

   The correct option is (C) — $\dfrac{\pi}{3}$.

   ANSWER: C

5. AP EAPCET 2021Set eng-2021-08-24-AN1 markMCQ
   Q.If $\vec{a}$ is a vector of magnitude 7, $\vec{b}$ is a vector of magnitude 8, then the maximum value of $|\vec{a}.\vec{b}|$ is (A) $5$ and $(\vec{a}.\vec{b}) = \frac{\pi}{6}$ (B) $28$ and $(\vec{a}.\vec{b}) = \frac{\pi}{3}$ (C) $56$ and $(\vec{a}.\vec{b}) = \frac{\pi}{2}$ (D) $56$ and $(\vec{a}.\vec{b}) = \pi$

   ›Reveal solutionSolution

   The dot product's magnitude is maximized when the vectors are parallel or antiparallel; the maximum value is $56$, achieved at $\theta=\pi$ (or $\theta=0$).

   Concept and Intuition

   $\vec a\cdot\vec b=|\vec a||\vec b|\cos\theta$, and $|\cos\theta|\le1$ always, with equality only when $\theta=0$ or $\theta=\pi$. So $|\vec a\cdot\vec b|$ is maximized exactly at the parallel/antiparallel configurations.

   Step-by-Step Solution

   1. $|\vec a\cdot\vec b| = |\vec a||\vec b||\cos\theta| = 56|\cos\theta|$.
   2. This is maximized when $|\cos\theta|=1$, i.e. $\theta=0$ or $\theta=\pi$, giving maximum value $56$.
   3. Among the options, only the one with value $56$ and a valid maximizing angle ($\theta=\pi$) is consistent (the option with $\theta=\pi/2$ would give $\cos\theta=0$, the minimum, not the maximum).

   Common Mistakes

   • Picking the option with the right magnitude ($56$) but a wrong angle (like $\pi/2$, which actually gives zero).

   ✓Final answer

   The correct option is (D) — $56$ and $(\vec{a}.\vec{b}) = \pi$.

   ANSWER: D

6. AP EAPCET 2021Set eng-2021-08-19-AN1 markMCQ
   Q.If $\vec{a}$ and $\vec{b}$ are two vectors such that $\dfrac{\vec{a}\cdot\vec{b}}{|\vec{a}||\vec{b}|} < 0$ and $|\vec{a}\cdot\vec{b}| = |\vec{a}\times\vec{b}|$ then the angle between the vectors $\vec{a}$ and $\vec{b}$ is ________ (A) $\dfrac{\pi}{4}$ (B) $\mathrm{Sec}^{-1}(-\sqrt{2})$ (C) $\mathrm{Tan}^{-1}\left(\dfrac{-1}{2}\right)$ (D) $\mathrm{Sin}^{-1}\left(\dfrac{1}{2}\right)$

   ›Reveal solutionSolution

   The two conditions together force $\theta=135^\circ$, which is precisely $\sec^{-1}(-\sqrt2)$.

   Concept and Intuition

   $\dfrac{\vec a\cdot\vec b}{|\vec a||\vec b|}=\cos\theta$, so a negative value means the angle is obtuse. The magnitude condition compares the dot and cross product magnitudes, which are $|\vec a||\vec b|\,|\cos\theta|$ and $|\vec a||\vec b|\,|\sin\theta|$ respectively.

   Step-by-Step Solution

   1. $\dfrac{\vec a\cdot\vec b}{|\vec a||\vec b|}<0\Rightarrow\cos\theta<0\Rightarrow\theta$ is obtuse (between $90^\circ$ and $180^\circ$).
   2. $|\vec a\cdot\vec b|=|\vec a\times\vec b|\Rightarrow|\vec a||\vec b||\cos\theta|=|\vec a||\vec b||\sin\theta|\Rightarrow|\cos\theta|=|\sin\theta|\Rightarrow\tan\theta=\pm1$.
   3. Combined with $\theta$ obtuse, the only solution in $(90^\circ,180^\circ)$ is $\theta=135^\circ$.
   4. Checking option (B): $\sec^{-1}(-\sqrt2)$ means $\cos\theta=-\dfrac1{\sqrt2}\Rightarrow\theta=135^\circ$ — matches exactly.
   5. Options (A), (C), (D) give acute or non-matching angles ($45^\circ$, $\tan^{-1}(-1/2)$, $30^\circ$ or $150^\circ$ respectively — none is exactly $135^\circ$).

   Common Mistakes

   • Picking the acute solution $\theta=45^\circ$ from $\tan\theta=\pm1$ without checking the sign condition $\cos\theta<0$, which rules it out.

   ✓Final answer

   The correct option is (B) — $\mathrm{Sec}^{-1}(-\sqrt{2})$.

   ANSWER: B

7. AP EAPCET 2024Set eng-2024-05-23-FN1 markMCQ
   Q.If $\bar{a} = -4\bar{i} + 2\bar{j} + 4\bar{k}$, $\bar{b} = \sqrt{2}\bar{i} - \sqrt{2}\bar{j}$ are two vectors then angle between the vectors $2\bar{a}$ and $\dfrac{\bar{b}}{2}$ is (A) $30^\circ$ (B) $135^\circ$ (C) $90^\circ$ (D) $0^\circ$

   ›Reveal solutionSolution

   The angle between $2\bar a$ and $\bar b/2$ equals the angle between $\bar a$ and $\bar b$ (scalar multiples by positive numbers don't change direction); computing that angle gives $135^\circ$.

   Concept and Intuition

   Multiplying a vector by a positive scalar only changes its magnitude, not its direction. So $\theta(2\bar a,\ \bar b/2)=\theta(\bar a,\ \bar b)$, and we can use the original vectors directly in the cosine formula.

   Step-by-Step Solution

   1. $\bar a\cdot\bar b=(-4)(\sqrt2)+(2)(-\sqrt2)+(4)(0)=-4\sqrt2-2\sqrt2+0=-6\sqrt2$.
   2. $|\bar a|=\sqrt{(-4)^2+2^2+4^2}=\sqrt{16+4+16}=\sqrt{36}=6$.
   3. $|\bar b|=\sqrt{(\sqrt2)^2+(-\sqrt2)^2+0^2}=\sqrt{2+2}=\sqrt4=2$.
   4. $\cos\theta=\dfrac{\bar a\cdot\bar b}{|\bar a||\bar b|}=\dfrac{-6\sqrt2}{6\times2}=\dfrac{-6\sqrt2}{12}=-\dfrac{\sqrt2}{2}=-\dfrac1{\sqrt2}$.
   5. $\theta=\cos^{-1}\left(-\dfrac1{\sqrt2}\right)=135^\circ$.

   Common Mistakes

   • Wasting time actually computing $2\bar a$ and $\bar b/2$ component-wise instead of recognizing the angle is scale-invariant.
   • Sign slip in the dot product from the negative components.

   ✓Final answer

   The correct option is (B) — $135^\circ$.

   ANSWER: B

8. AP EAPCET 2023Set eng-2023-05-15-FN1 markMCQ
   Q.Two vectors of same magnitude act at a point. Twice the product of the magnitudes of two vectors is equal to the square of the magnitude of their resultant. The angle between the two vectors is (A) $60°$ (B) $30°$ (C) $90°$ (D) $120°$

   ›Reveal solutionSolution

   Setting $2A^2$ (twice the product of equal magnitudes) equal to $R^2$ for two equal vectors directly forces $\cos\theta=0$, so $\theta=90°$.

   Concept and Intuition

   For two vectors of equal magnitude $A$ at angle $\theta$, the parallelogram law gives $R^2=2A^2(1+\cos\theta)$. The given condition ("twice the product of magnitudes equals square of resultant") is simply an equation relating $A$ and $R$ that pins down $\cos\theta$.

   Step-by-Step Solution

   1. Let both vectors have magnitude $A$. Resultant: $R^2 = A^2+A^2+2A\cdot A\cos\theta = 2A^2(1+\cos\theta)$.
   2. Given condition: $2(A)(A) = R^2$, i.e. $2A^2 = R^2$.
   3. Substitute: $2A^2 = 2A^2(1+\cos\theta)$.
   4. Divide both sides by $2A^2$ (nonzero): $1 = 1+\cos\theta \Rightarrow \cos\theta = 0$.
   5. $\theta = 90°$.

   Common Mistakes

   • Misreading "twice the product of the magnitudes" as $2A$ instead of $2A^2$ (product of two equal magnitudes is $A\times A=A^2$).
   • Sign error in the resultant formula (using $-2A^2\cos\theta$ instead of $+2A^2\cos\theta$).

   ✓Final answer

   The correct option is (C) — $90°$.

   ANSWER: C

9. AP EAPCET 2026Set eng-2026-05-18-FN1 markMCQ
   Q.The position vectors of the vertices A and B of a triangle $ABC$ are $\bar{i}+3\bar{j}+4\bar{k}$ and $2\bar{i}+\bar{j}+2\bar{k}$ respectively. If $|\overline{AC}| = 5$ and angle $A = \pi/3$, then $|\overline{BC}| =$ (A) $\sqrt{26}$ (B) $3\sqrt{19}$ (C) $3\sqrt{26}$ (D) $\sqrt{19}$

   ›Reveal solutionSolution

   Find $|AB|$ from the position vectors, then apply the law of cosines at the known angle $A$.

   Concept and Intuition

   Once we know two sides meeting at a vertex ($AB$ and $AC$) and the included angle there ($A$), the third side $BC$ is fixed by the law of cosines — position vectors are just a way of encoding the side length $AB$.

   Step-by-Step Solution

   1. $A=(1,3,4)$, $B=(2,1,2)$, so $\overline{AB}=B-A=(1,-2,-2)$, giving $|AB|=\sqrt{1^2+(-2)^2+(-2)^2}=\sqrt9=3$.
   2. We are given $|AC|=5$ and $\angle A = \pi/3$ (the angle between $AB$ and $AC$ at vertex $A$).
   3. By the law of cosines in $\triangle ABC$: $BC^2 = AB^2+AC^2-2\cdot AB\cdot AC\cos A$.
   4. $=3^2+5^2-2(3)(5)\cos(\pi/3) = 9+25-30\left(\dfrac12\right)=34-15=19$.
   5. $BC=\sqrt{19}$.

   Common Mistakes

   • Computing $|AB|$ from the wrong difference (e.g. $A-B$ instead of $B-A$ — though the magnitude is the same, sign slips elsewhere are common).
   • Using $\cos(\pi/3)=1$ or a wrong standard value instead of $1/2$.

   ✓Final answer

   The correct option is (D) — $\sqrt{19}$.

   ANSWER: D

10. AP EAPCET 2024Set eng-2024-05-18-FN1 markMCQ
    Q.If $\bar{f}, \bar{g}, \bar{h}$ be mutually orthogonal vectors of equal magnitudes, then the angle between the vectors $\bar{f} + \bar{g} + \bar{h}$ and $\bar{h}$ is (A) $\cos^{-1}\left(\dfrac{\sqrt{3}}{4}\right)$ (B) $\cos^{-1}\left(\dfrac{1}{\sqrt{3}}\right)$ (C) $\pi - \cos^{-1}\left(\dfrac{1}{\sqrt{3}}\right)$ (D) $\pi - \cos^{-1}\left(\dfrac{\sqrt{3}}{4}\right)$

    ›Reveal solutionSolution

    Uses orthogonality to kill cross dot products; answer is $\cos^{-1}(1/\sqrt3)$.

    Concept and Intuition

    When three vectors are mutually perpendicular and of equal magnitude, their sum is the space-diagonal of a cube built on them. The angle any diagonal makes with an edge is a classic $\cos^{-1}(1/\sqrt3)$ result.

    Step-by-Step Solution

    1. Let $|\bar f|=|\bar g|=|\bar h|=a$, and $\bar f\cdot\bar g=\bar g\cdot\bar h=\bar h\cdot\bar f=0$.
    2. $(\bar f+\bar g+\bar h)\cdot\bar h = \bar f\cdot\bar h + \bar g\cdot\bar h + \bar h\cdot\bar h = 0+0+a^2 = a^2$.
    3. $|\bar f+\bar g+\bar h|^2 = |\bar f|^2+|\bar g|^2+|\bar h|^2 + 2(\bar f\cdot\bar g+\bar g\cdot\bar h+\bar h\cdot\bar f) = 3a^2$, so $|\bar f+\bar g+\bar h|=a\sqrt3$.
    4. $\cos\theta = \dfrac{(\bar f+\bar g+\bar h)\cdot\bar h}{|\bar f+\bar g+\bar h|\,|\bar h|} = \dfrac{a^2}{a\sqrt3\cdot a}=\dfrac{1}{\sqrt3}$.
    5. So $\theta = \cos^{-1}(1/\sqrt3)$.

    Common Mistakes

    • Forgetting the cross terms vanish because the vectors are orthogonal, not just non-collinear.
    • Mixing up $\cos^{-1}(\sqrt3/4)$ with $\cos^{-1}(1/\sqrt3)$ — the magnitude of the sum vector is $a\sqrt3$, not $2a$.

    ✓Final answer

    The correct option is (B) — $\cos^{-1}\left(\dfrac{1}{\sqrt3}\right)$.

    ANSWER: B

11. AP EAPCET 2021Set eng-2021-08-24-FN1 markMCQ
    Q.Find the angle between the vectors $\vec{A} = 2\hat{i} + 4\hat{j} + 4\hat{k}$ and $\vec{B} = 4\hat{i} + 2\hat{j} - 4\hat{k}$. (A) $0^\circ$ (B) $45^\circ$ (C) $60^\circ$ (D) $90^\circ$

    ›Reveal solutionSolution

    The dot product of the two vectors is exactly zero, so the angle between them is $90^\circ$.

    Concept and Intuition

    The angle between two vectors is found from $\cos\theta=\dfrac{\vec A\cdot\vec B}{|\vec A||\vec B|}$; a zero dot product directly signals perpendicularity without needing the magnitudes.

    Step-by-Step Solution

    1. $\vec A\cdot\vec B=(2)(4)+(4)(2)+(4)(-4)=8+8-16=0$.
    2. Since $|\vec A|,|\vec B|\ne0$, $\cos\theta=0\Rightarrow\theta=90^\circ$.

    Common Mistakes

    • Sign slip while multiplying the $k$-components ($4\times(-4)=-16$, not $+16$).

    ✓Final answer

    The correct option is (D) — $90^\circ$.

    ANSWER: D

12. AP EAPCET 2024Set eng-2024-05-21-AN1 markMCQ
    Q.If $\bar{a}, \bar{b}, \bar{c}$ are 3 vectors such that $|\bar{a}|=5, |\bar{b}|=8, |\bar{c}|=11$ and $\bar{a}+\bar{b}+\bar{c}=\bar{0}$ then the angle between the vectors $\bar{a}$ and $\bar{b}$ is (A) $\cos^{-1}\dfrac{2}{5}$ (B) $\cos^{-1}\dfrac{10}{11}$ (C) $\cos^{-1}\dfrac{41}{55}$ (D) $\dfrac{\pi}{3}$

    ›Reveal solutionSolution

    From $\bar c=-(\bar a+\bar b)$, $|\bar c|^2=|\bar a|^2+|\bar b|^2+2\,\bar a\!\cdot\!\bar b$ gives $\cos\theta=\tfrac25$.

    Since $\bar a+\bar b+\bar c=\bar 0$, we have $\bar c=-(\bar a+\bar b)$, so

    $|\bar c|^2=|\bar a+\bar b|^2=|\bar a|^2+|\bar b|^2+2|\bar a||\bar b|\cos\theta,$

    where $\theta$ is the angle between $\bar a$ and $\bar b$.

    Substituting $|\bar a|=5,\ |\bar b|=8,\ |\bar c|=11$:

    $121 = 25 + 64 + 2(5)(8)\cos\theta = 89 + 80\cos\theta.$

    $80\cos\theta = 32 \ \Rightarrow\ \cos\theta=\frac{32}{80}=\frac{2}{5}.$

    Hence $\theta=\cos^{-1}\dfrac{2}{5}$.

    ✓Final answer

    The angle between $\bar a$ and $\bar b$ is $\cos^{-1}\dfrac{2}{5}$ — option (A).