Audio signal path is positive(signal) > driver (which translates signal to ears& brain as sound) > ground. Ground cable impedance comes after driver and isn`t affecting signal path that ends up in our ears. Source is "pushing" signal one way and "pulling" it other way (ground). If for rough example signal would travel from both, ground and positive, to driver, then it would make more sense that ground is also affecting sound re-produced by driver.
Or am I thinking totally wrong?
taking your thinking to the extreme (in absurdum): then you wouldn't need a cable after the driver: that would equal a cable with infinite resistance, but since resistance after the driver doesn't matter you would be ok anyway...
If there's resistance between the driver and your source's ground, then the signal needs to be "pushed" all the way, even from the driver to the ground.
i'm an engineering student, so some things might be basic knowledge for me but not for others. I hope i don't come across rude or so!