Gryffinclaws like you are rare in the wild, but you’re easy to recognize as a Gryffinclaw because you’re bold, outgoing, and a natural leader. Your ambition and your drive to change the world are powerful forces inside you, but they’re tempered and softened by your knowledge and your understanding, as well as your keen wit and sense of humor. If they haven’t already, somebody should probably put you in charge of something big.
Actually, they have Arithmetic - it mentions Hermione taking it in a couple of the books - it just isn't mandatory that the students take it, it's more of an elective than a core subject. There IS math, though.