XML or SGML or ...

DocBook was historically an SGML DTD.

DocBook V4.x is supported in XML and SGML DTDs (with unofficial RELAX NG and W3C XML Schema implementations).

DocBook V5.x is expected to be a RELAX NG grammar (with W3C XML Schema and DTD implementations).