I know what you are talking about and it still doesn't work even when the brake is set. Since I am the only one that drives the car I use seat position 2 to park. It angles the mirrors down so I can see the curb I am backing up next to. That usually works fine, but sometimes if the seat does move back into the position that I set to get out then it won't move back into seat position 1 or 2 unless I move the seat forward with the control closer to position 1. Then if I press the button it will move the seat to the correct position for button 1. Hope that didn't totally confuse everyone.Do you have an auto or 6spd manual?...because if the parking brake isn't up on the manual the seat won't recall.
Could it be you don't use the parking brake everytime you park the car?