I think he's spot on about this, and about the danger of treating digital as a silver bullet.
Government is in the business of achieving great outcomes for society. Any government who fails to keep this central to their thinking is likely to find itself at the receiving end of significant pressure, ranging from social media complaints all the way up to violent revolution (depending on how far they've strayed).
Digital has a major role in achieving these great outcomes, however it isn't the only approach, nor always the best.
In my view digital should be considered a adjective, not a noun.
The goal is never to 'go digital' - that's just as ridiculous as suggesting that the goal is to 'go telephone' or 'go print'.
Digital, as an area, encompasses a range of tools and techniques that can help an organisation to achieve its goals more effectively or efficiently, but it should not replace those goals - government must be driven by social and citizen needs.
So where does this leave the notion of places like the 'Digital Transformation Office' - it certainly doesn't invalidate them. The goal is improving governance, improving citizen services, reducing costs, increasing compliance, improving outcomes. This is achieved through transforming what already exists, with a key toolkit being digital.
Provided the people leading and working in places like the Digital Transformation Office are clear on what their end goal is (which I believe they are), this can produce great outcomes for citizens, the country, politicians and government agencies themselves.
It's only when 'digital' becomes a noun - the goal, rather than part of the process - that the value is distorted and often lost.