The scaling dimension matrix is the (position-independent) matrix whose eigenvalues are the scaling dimensions. In the unitary case, the dilation operator is diagonalizable. If this is the case, one may choose a basis of scale covariant fields that change by an infinitesimal constant factor (the scaling dimension of the field) when an infinitesimal scaling is applied. In this basis, the scaling matrix is diagonal. If one starts directly from the assumption of scale covariant fields one gets the scaling dimensions directly, without an intervening matrix.

In general the dilation operator may or may not be the case; the author states on p.15 explicitly that s/he wants to include this case. If the normal form of the matrix has a nontrivial Jordan block, there is no basis of scale covariant fields, and one has a so-called logarithmic CFT. This is explained on p.22. (See also pp.61-62.)

Added: Since the commutator relation stated by the OP holds for every local field $O(x)$, the scale matrix must figure on the right hand side. If it is restricted to scale covariant fields, one can replace the matrix by the scaling dimension of $O(x)$.