The Coleman-Oort conjecture predicts that up to isomorphism there are at most finitely many CM Jacobians for smooth projective curves of genus g, when g is large enough. We show that this is true when suitable conditions are put on the CM type, and discuss its connection with the Ekedahl-Serre problem. This is a joint work with X. Lu and K. Zuo