The whole idea of compactifying at a CY manifold to obtain 4-dimensional physics dates back to the 80's and the reason why people started thinking about it is, as Urs says, that CYs only conserve part of the supersymmetry. Consider strings in the total 10-dimensional space-time $M$ which can be "decomposed" as $\mathcal{M}=M_4 \times X_6$. This space-time is always locally 10 dimensional and thus there exists (locally) a set of 10d supercharges, transforming in the spinor representation of $SO(10)$. The supersymmetry we want to achieve, in 4d, contains supercharges which are well defined globally in the $X_6$ part of the manifold $\mathcal{M}$. Now, the compact space $X_6$ is curved and as a result if we parallel transport a supercharge around $X_6$ it does not have to return to identity. Therefore, supercharges within $X_6$ are related by parallel transport with the $SO(6)$ spin connection corresponding to the metric of $X_6$. Now, let us consider a "test" local supercharge at a point $p \in X_6 \subset \mathcal{M}$. We can parallel transport this supercharge around a closed loop $C$ and come back to $p$. This action will result in a rotation of the supercharge by a non-trivial element $R \in SO(6)$. The important thing here is that such supercharges that get rotated after a parallel transport do not lead to globally well-defined supercharges. What does this mean for $X_6$? It means that upon string compactification on $X_6$ this compact space will preserve some of the supersymmetry if there exist 6d non-trivial spinors $\zeta(y^i)$, where $y$ is a coordinate system in $X_6$ and $i=4, \ldots 9$. These are the Killing spinors and are covariantly constant in the $X_6$, i.e. $\nabla_{X_6}\zeta$ =0. Now, here is where the holonomy group of $X_6$ plays a special role. Generally, 6d manifolds have the holonomy group $SO(6)$ and all components of the 10d spinor should get transformed under this. This means that there are no Killing-spinors and as result all supersymmetries are broken in a general manifold of such a holonomy group. But we need to consider manifolds that preserve some susy, that is manifolds with special holonomy groups which should be subgroups of the bigger $SO(6)$ group. What will work for us are manifolds with $SU(2)$ holonomy such that indeed they preserve some supersymmetry. Such manifolds are the Calabi-Yau manifolds. This gives us the connection between Killing spinors and CYs.

This gives the basic idea. Now, as Urs says, there are nice reference with more details in the nLab entry.