The British National Party has said next year's Red White and Blue Festival (RWB) could be moved from Derbyshire. The party said it was concerned about the disruption caused to local people and the licensing restrictions imposed by the council.
Last weekend 19 people were arrested during anti-BNP protests near Denby in a policing operation estimated to have cost £500,000.
A BNP spokesman said no decision had been made on any alternative sites. The event faced restrictions on music, alcohol sales and the number of caravans it could host.
The centre of Denby, where most of the protests took place, was heavily policed and some residents needed an escort to leave the town.
Deputy leader of the BNP Simon Darby said: "There is a school of thought that says we have had RWB in Wales, in the Midlands, in the North, in the East Midlands and we have never had it, for instance, in Scotland, the South West or the South. So I think there will be a lot of competition, there are several sites which want us."
Protesters said they would follow the festival no matter where it went and added they hoped councils would carefully consider any application from organisers.