I think most people are confused because of the popularity of Integrated Amps (Pre-Amp and (Power) Amplifier in one box), which have all the controls of a pre-amp but are often referred to simply as Amplifiers.
Growing up with a dad who was into Hi-Fi in the 70's and 80's, this is how I understood things:
(Power) Amplifier: One set of pre-amp inputs, speaker outputs, designed solely to amplify the input signal and send it to the speakers. No volume control. (Arcam P49, Schiit Vidar)
Pre-Amplifier: Multiple inputs (turntable, cassette, CD, AM/FM Tuner, etc.) Designed to select and modify the input signal with tone and volume controls, often included a built-in phono pre-amp and headphone amp. (McIntosh C22, Schiit Freya)
Integrated Amplifier: Pre-Amp and (Power) Amplifier in one box (NAD C356, Schitt Ragnarok)
Receiver: AM/FM Tuner, Pre-Amp, (Power) Amplifier in one box (McIntosh MAC6700, NAD 7140)
With respect to Headphone Amplifiers, I have never come across an external Headphone Amplifier that does NOT have a volume control. This may also be where the confusion comes from as Headphone Amplifiers and (Power) Amplifiers are designed to do the exact same thing (amplify an input signal) but are constructed differently. Because, and I am purely guessing here, Headphone Amps sprang from the digital/personal audio explosion where one input source was the norm and the input signals are not particularly strong, Headphone amps evolved with a volume control.