The vectors $\hat{i}+2\hat{j}+3\hat{k},\lambda \hat{i}+4\hat{j}+7\hat{k}$ and $-3\hat{i}-2\hat{j}-5\hat{k}$ are coplanar, if $\lambda $ is equal to?
(a) 6
(b) 5
(c) 4
(d) 3

Hint: Compare the given three vectors with the general form given as ${{a}_{1}}\hat{i}+{{b}_{1}}\hat{j}+{{c}_{1}}\hat{k},{{a}_{2}}\hat{i}+{{b}_{2}}\hat{j}+{{c}_{2}}\hat{k}$ and ${{a}_{3}}\hat{i}+{{b}_{3}}\hat{j}+{{c}_{3}}\hat{k}$ respectively. Now, for the condition of coplanar vectors use the determinant formula given as $\left| \begin{matrix}
   {{a}_{1}} & {{b}_{1}} & {{c}_{1}} \\
   {{a}_{2}} & {{b}_{2}} & {{c}_{2}} \\
   {{a}_{3}} & {{b}_{3}} & {{c}_{3}} \\
\end{matrix} \right|=0$, substitute the values of the coefficients considered and expand it to form an equation in $\lambda $ as a variable. Solve for the value of $\lambda $ to get the answer.

Complete step by step answer:
Here we have been provided with three vectors $\hat{i}+2\hat{j}+3\hat{k},\lambda \hat{i}+4\hat{j}+7\hat{k}$ and $-3\hat{i}-2\hat{j}-5\hat{k}$ which are said to be coplanar. We have been asked to find the value of $\lambda $ for the condition to be satisfied.
Now, comparing the given vectors with their general forms given as ${{a}_{1}}\hat{i}+{{b}_{1}}\hat{j}+{{c}_{1}}\hat{k},{{a}_{2}}\hat{i}+{{b}_{2}}\hat{j}+{{c}_{2}}\hat{k}$ and ${{a}_{3}}\hat{i}+{{b}_{3}}\hat{j}+{{c}_{3}}\hat{k}$ respectively we have the following values of coefficients of the unit vectors \[\hat{i},\hat{j}\] and $\hat{k}$.
\[\Rightarrow \left( {{a}_{1}},{{b}_{1}},{{c}_{1}} \right)=\left( 1,2,3 \right),\left( {{a}_{2}},{{b}_{2}},{{c}_{2}} \right)=\left( \lambda ,4,7 \right),\left( {{a}_{3}},{{b}_{3}},{{c}_{3}} \right)=\left( -3,-2,-5 \right)\]
The condition for three vectors to be coplanar is given by the determinant formula given as $\left| \begin{matrix}
   {{a}_{1}} & {{b}_{1}} & {{c}_{1}} \\
   {{a}_{2}} & {{b}_{2}} & {{c}_{2}} \\
   {{a}_{3}} & {{b}_{3}} & {{c}_{3}} \\
\end{matrix} \right|=0$, so substituting the values of coefficients we get,
$\Rightarrow \left| \begin{matrix}
   1 & 2 & 3 \\
   \lambda & 4 & 7 \\
   -3 & -2 & -5 \\
\end{matrix} \right|=0$
Performing the elementary row operation ${{R}_{1}}\to {{R}_{1}}+{{R}_{3}}$ we get,
$\Rightarrow \left| \begin{matrix}
   -2 & 0 & -2 \\
   \lambda & 4 & 7 \\
   -3 & -2 & -5 \\
\end{matrix} \right|=0$
Now, performing the elementary column operation given as ${{C}_{3}}\to {{C}_{3}}-{{C}_{1}}$ we get,
$\Rightarrow \left| \begin{matrix}
   -2 & 0 & 0 \\
   \lambda & 4 & 7-\lambda \\
   -3 & -2 & -2 \\
\end{matrix} \right|=0$
Expanding the determinant along the first row we get,
$\begin{align}
  & \Rightarrow -2\left[ 4\times \left( -2 \right)-\left( -2 \right)\times \left( 7-\lambda \right) \right]-0\left[ \lambda \times \left( -2 \right)-\left( -3 \right)\times \left( 7-\lambda \right) \right]+0\left[ \lambda \times \left( -2 \right)-\left( -3 \right)\times 4 \right]=0 \\
 & \Rightarrow -2\left[ -8+14-2\lambda \right]=0 \\
 & \Rightarrow 6-2\lambda =0 \\
\end{align}$
Solving the above linear equation for the value of $\lambda $ we get,
$\begin{align}
  & \Rightarrow 2\lambda =6 \\
 & \therefore \lambda =3 \\
\end{align}$

So, the correct answer is “Option d”.

Note: Note that the above formula of the determinant arises from the fact that ‘if three vectors are coplanar then the dot product of one vector with the cross product of other two vectors must be zero’, in other words the scalar triple product of three vectors $\overrightarrow{a}$, $\overrightarrow{b}$ and $\overrightarrow{c}$ given by the relation $\left( \overrightarrow{a}\times \overrightarrow{b} \right).\overrightarrow{c}=0$. Here the dot (.) symbol is the dot product and the cross $\left( \times \right)$ symbol is the cross product. The above formula we have applied is nothing but the simplified form of the mentioned condition.