About 2200 years ago, Dongargarh was a fairly renounced territory named Kamavati ruled by Maharaja Kamsen also called Kamakhya Nagri. When his Queen gave birth to a son they named him Madansen. Since Raja Veersen considered it a blessing of Lord Shiv & Parvati, he constructed the temple of Shri Bamleshwari Mandir at Dongargarh. Now Raja Madansen had a son named Kamsen because of whom the area became famous and came to be known as Kamakhaya Nagari.

A big battle took place between Raja Kamsen and Raja Vikramaditya, reason being the love between Maadhavnal & Kaamkandla, which led to the complete destruction of KAMAVATI. Legend has it that after the death of Maadhavnal & Kaamkandala, Raja Vikramaditya felt guilty as he thought that he was responsible for Kaamkandala committing suicide. He underwent deep anguish and started constant worship, resulting in Maa Bagulamukhi Devi appearing before him.

Raja Vikramaditya requested the goddess to make both Maadhavnal and Kaamkandala alive again and also asked Maa Bagulamukhi Devi to stay at the temple. Since then it is believed that Maa Bagulamukhi Devi is present here. As time passed by the name got transformed from Maa Bagulamukhi to Maa Bamlai to Shri Bamleshwari Mandir as it is known today.