I do something similar to @Mud, but switch to previous view buffer,
#, instead of the previous buffer in buffer list. Here is a binding key in my
nnoremap <silent> <leader>q :lclose<bar>b#<bar>bd #<CR>
Close Location windows, if exist, switch to the previous view buffer, and then close the last switched buffer.