If ${{\left( {{\cot }^{-1}}x \right)}^{2}}-7\left( {{\cot }^{-1}}x \right)+10>0$, then x lies in the interval
(a) $\left( \cot 5,\cot 2 \right)$
(b) $\left( -\infty ,\cot 5 \right)\cup \left( \cot 2,\infty \right)$
(c) $\left( -\infty ,\cot 5 \right)$
(d) $\left( \cot 2,\infty \right)$

Hint:Assume that ${{\cot }^{-1}}x=t$. Write quadratic inequality based on the given assumption and factorize the quadratic expression by splitting the middle term. Solve the inequality to calculate the value of ‘x’ which satisfies the given equation.

Complete step-by-step answer:
We have to calculate the value of ‘x’ which satisfies the inequality ${{\left( {{\cot }^{-1}}x \right)}^{2}}-7\left( {{\cot }^{-1}}x \right)+10>0$.
We will assume that ${{\cot }^{-1}}x=t$.
Thus, we have ${{t}^{2}}-7t+10>0$. We will now factorize this inequality by splitting the middle term.
We can rewrite the above equation as ${{t}^{2}}-2t-5t+10>0$.
Taking out the common terms, we have $t\left( t-2 \right)-5\left( t-2 \right)>0$.
Thus, we have $\left( t-2 \right)\left( t-5 \right)>0$.
So, we can have $t-2>0,t-5>0$ or $t-2<0,t-5<0$.
Thus, the solution of the above inequality is $t<2,t>5$.
Substituting ${{\cot }^{-1}}x=t$ in the above equation, we have ${{\cot }^{-1}}x<2,{{\cot }^{-1}}x>5$.
Taking inverse on both sides, we have $\cot \left( {{\cot }^{-1}}x \right)>\cot 2,\cot \left( {{\cot }^{-1}}x \right)<\cot 5$. The sign of the above inequality reverses as cotangent is a decreasing function over $\left[ 0,\pi \right]$.
We know that $\cot \left( {{\cot }^{-1}}x \right)=x$.
Thus, we have $x>\cot 2,x<\cot 5$. So, we have $x\in \left( -\infty ,\cot 5 \right)\cup \left( \cot 2,\infty \right)$.
Hence, the values of ‘x’ which satisfy the given inequality ${{\left( {{\cot }^{-1}}x \right)}^{2}}-7\left( {{\cot }^{-1}}x \right)+10>0$ are $x\in \left( -\infty ,\cot 5 \right)\cup \left( \cot 2,\infty \right)$, which is option (b).

Note: We can’t solve this equation without factorizing the quadratic inequality and then calculating the value of ‘x’ which satisfies the given equation. We can factorize the given equation by completing the square method or calculating the discriminant method.