A user was removing nasty tags etc from the html filtering dialog. They would then click on html filtering link again and the nasty tags I removed appear again. It turns out the user needs to push two buttons. The remove button and then the save button way down the bottom of the page. This is potentially confusing. (it confused us for awhile).
I can think of two fixes:
get rid of save and cancel buttons.
put an info dialog saying "changes currently not saved. Press "save" to save them now. (this system works well on another service we use).