According to legend, Hung Ga was named after Hung Hei-Gun, who learned martial arts from Jee Sin, a Chan (Zen) master at the Southern Shaolin Temple. Jee Sin (ak Gee Sum Sim See) was also the master of four other students, namely Choy Gau Lee, Mok Da Si, Lau Sam-Ngan and Li Yao San. These five martial artists later became the founders of the five major family styles of Southern Chinese martial arts: (Hung Ga, Choy Gar, Mok Gar, Li Gar and Lau Gar).

The temple wher they trained had become a refuge for opponents of the Qing Dynasty, who used it as a base for their activities, and was soon destroyed by Qing forces. Hung, a tea merchant by trade, eventually left his home in Fujian for Guangdong, bringing the art with him.

Because the history of the Chinese martial arts was historically transmitted orally rather than by text, much of the early history of Hung Ga will probably never be either clarified or corroborated by written documentation.

The character "hung" (洪) was used in the reign name of the emperor who overthrew the Mongol Yuan Dynasty to establish the Han Chinese Ming Dynasty, opponents of the Manchu Qing Dynasty made frequent use of the character in their imagery. (Ironically, Luk Ah-Choi was the son of a Manchu stationed in Guangdong.)

Hung Hei-Gun is itself an assumed name intended to honor that first Ming Emperor. Anti-Qing rebels named the most far reaching of the secret societies they formed the "Hung Mun" (洪门).

The Hung Mun claimed to be founded by survivors of the destruction of the Shaolin Temple, and the martial arts its members practiced came to be called "Hung Ga" and "Hung Kuen."