This is more of a comment.

The first obvious problem is that the partition function can sometimes be infinite. For example, $Z(T^3)$ is the dimension of the Hilbert space attached to $T^2$, which is infinite-dimensional for a non-compact group $G$.

The second problem is the choice of the representation. Skein relations in Chern-Simons arise due to finite-dimensionality of the Hilbert space attached to $S^2$ with 4 marked points (2 positively-oriented and 2 negatively-oriented). If the Hilbert space is, say, $n$-dimensional, the partition functions evaluated on $n+1$ different crossings should be linearly dependent, this is the skein relation. An easy computation shows that $n$ is the number of irreducible representations occurring in $V^{\otimes 2}$, where $V$ is the representation attached to the knot. So, on the one hand you want $V$ to be finite-dimensional (in which case there is a skein relation). On the other hand, finite-dimensional representations give the same answers as the compact form, at least on the perturbative level (this claim appears in Gukov's and Witten's papers), so people are not so interested in them.

This post has been migrated from (A51.SE)