There are some nicer parts of Aylesbury. If you walk up from the A41 up Church street, toward temple square and then down to St Mary's (I think - the big church) and then around Parson's fee and pebble lane then it's all very nice. There's also some lovely spots down by the canal. Do that little circuit on a sunny Sunday and it's very nice. The Town Centre was actually quite nice until around 20 years ago. I don't really know what happened to it - but it's not the relatively serene and pleasant place it was. There seem to be less local businesses and more low-end national retailers.
