Scott Barbour/TAS/Getty Images

Some of pop star Taylor Swift's Rhode Island neighbors are angry about construction work on her property, which they claim is restricting access to a public beach.

Swift, 24, paid $17 million for the 11,000 square-foot historic mansion located in the exclusive Watch Hill neighborhood in Westerly. The mansion boasts 16 rooms on more than five acres.

WATCH: Taylor Swift Meets Prince William

The "Red" singer is spending money on improvements to the property, including on a seawall that some locals have claimed will restrict access to a public beach.

Swift maintains that she has all the required permits for the job and is restoring the wall after it was damaged by Superstorm Sandy, but others say the large boulders being placed by construction crews are restricting use of the area, which is popular with fishermen and surfers.

"I don't think any celebrity should have the right to do that with such a historic area," Sabrina Plew, a local resident, told "Good Morning America."

Taylor Swift Leaves Huge Tip

Added resident Jim Toscano: "If it does impact it negatively, that's something that is just going to stick and we won't be able to surf there."

The Historical Preservation and Heritage Commission for Rhode Island told ABC News it had "no issues" with the project and the group that regulates coastal development has said the work is up to snuff.