I believe the answer to original question is probably yes, but unfortunately, I can't say that definitively. I can help answer Peter's extended question, however.

In math/0001038, by Nebe, Rains, and Sloane, they show that the Clifford group is a maximal finite subgroup of U(2^n). Solovay has also shown this in unpublished work that "uses essentially the classification of finite simple groups." The Nebe et al. paper also shows that the qudit Clifford group is a maximal finite subgroup for prime p, also using the classification of finite groups. This means that the Clifford group plus any gate is an infinite group, which makes one of the assumptions of the original question redundant.

Now, both Rains and Solovay told me that the next step, showing that an infinite group containing the Clifford group is universal, is relatively straightforward. However, I don't know how that step actually works. And more importantly for the original question, I don't know if they were only considering the qubit case or also the qudit case.

Actually, I might add that I don't understand the Nebe, Rains, and Sloane proof either, but would like to.

This post has been migrated from (A51.SE)