You and Anna are both correct.
In the future we handle categories a little different, properbly in 0.8
So whe you create a category, you can name it whatever you want. It will then be saved as 2 different versions, one for the html output and 1 which is url save ( ' also not good for url, especially with clean_urls ).
In addition to that we make sure to make ' " and such generally programming language chars, db safe. Currently they are not escaped \' i think, which make it valid to insert / output and dont throw php errors
Sometimes its hard to enhance an existing script
For now, pls use a-z 0-9