In theory, you can change the system language through System Preferences on macOS.

I did find a couple of machines that came with the system language in a different one from what I wanted, and even when I changed the primary language back to English (and rebooted when prompted), the original language persisted.

After doing a bit of Google searching, I came across this gem, which points to the command-line tool that really makes the system language change stick: