Born William Andrew Jones in the Wollaston neighborhood of Quincy, Massachusetts, DeWolfe was the son of a Welsh bookbinder who encouraged him to become a Baptist minister. Instead, "Billy" developed an interest in the theatre. He found work as an usher before becoming a dancer with the Jimmy O'Connor Band. It was at this point that he changed his last name to De Wolfe, which was the last name of the manager of the Massachusetts theatre where he worked. He went on to tour Europe with a dance team, appearing in a London revue called "Revels in Rhythm" and "danced before royalty on nine continents." In 1942, at the age of 35, he enlisted and served in the United States Navy as a musician until he was discharged for "medical" reasons in July 1944.