The symbol ‘&’ comes from a combination of the Latin letters ‘e’ and ‘t’, because in Latin, ‘et’ means ‘and’ (according to this Dictionary.com blog).
In other words, since the Romans used it, ‘&’ is a really, really old symbol (about 2,000 years).
A website about web design corroborates this information and adds:
The first ampersands looked very much like the separate E and T combined, but as type developed over the next few centuries, it eventually became more stylized and less representative of its origins.
So, why do English-speakers refer to ‘&’, as ‘ampersand’? Crazy story: the symbol actually used to be the 27th of the English alphabet (source).
When ‘&’ was the 27th symbol and kids had to recite the entire alphabet, at the end they would finish with “…X,Y,Z, and, per se and”, because this made it clear that ‘&’ was the last symbol of the list of symbols they were reciting (source 1, source 2). Eventually, because this is how language happens, “and per se and” started to be pronounced as one word, the word we know today, ‘ampersand’. The design website linked to above describes it this way:
The word was created as a slurred form of “and, per se and”, which was what the alphabet ended with when recited in English-speaking schools.
Disclaimer: The above post is an old one, which I am reposting on this new website. It has been edited.