(Taken from the Tokyo Classified web site. It's about the best description of Thee Michelle Gun Elephant in English that's out there.)
There's no rule that says the English and romaji names that Japanese bands use have to make sense: Witness Puffy, Glay, and Do As Infinity. Stringing together three unrelated nouns and one article, garage-punk band Thee Michelle Gun Elephant could win a prize for the most confounding use of English. Fortunately, rather than demonstrating their lack of grammar-ability, the name is actually a kind of homage to two rock bands that Thee Michelle Gun Elephant counts as its heroes.
Known as TMGE to fans, the group's unusual name came from a mispronunciation of The Damned's 1979 album Machine Gun Etiquette. After buying a ticket to see The Damned, an early band member is said to have proudly uttered, "This is a Michelle Gun Elephant ticket." The name stuck; the band member didn't. Thee is a tribute to Billy Childish's offbeat band Thee Headcoats, also one of TMGE's many influences.