I know the original question is ancient here, but I want to post in case someone finds this information useful. I had exactly the same problem, and the following has basically cured me completely.
I do these exercises at least once a day that look ridiculous, but don't take very long and are highly effective. Basically, get down on all fours (on your hands and knees) and flex your back gently backwards and forwards at least ten times. The more the better. Arch your back as far as you comfortably can, then gently arch it in the other direction. So one way you're making your back as round as you can, the other way you're sticking your butt out and arching as far as possible.
It's like fixing a stiff door by moving it backwards and forwards till it "unsticks". Somehow it really works.
If your lower back is inflamed from sitting too long, your butt will also hurt; a cushion helps a lot here! But fixing your back helps more.
Of course back pain can have a variety of causes, but I find that just flexing my back like this regularly sorts it out. Before I discovered this, I spent years in daily pain from having to sit 8 hours a day doing my job.
Also be sure not to slouch in your chair. Sit up straight-ish; don't slouch in a way that puts pressure on your back.
On a separate, but related note, I also used to have terrible posture but I didn't find exercises any help, or even seeing a physiotherapist. Alexander Technique sorted out my posture. The downside to that is that lessons are expensive, you can't easily learn it yourself, and the quality of teachers varies tremendously. A lot of them seem to be just teaching their own thing really. Fortunately I found a good one who enabled me to stand up straight for the first time since I was about six!
I threw my back out just over two years ago. It was NOT fun and something I didn't ever want to experience again: having to crawl on your stomach across the floor at about the speed of a snail while you're clenching your teeth in agony at every movement.
I decided to try standing. Since I spent so much of my time sitting in front of a computer I decided it was probably the best place to try and shake things up.
I bought some cinder blocks and propped my desk up. I bought a good quality gel mat (from Gel Pro) as well as a draft chair.
Standing for 8 hours a day at work was not going to be a cure all; staying in one position too long is where the problem can arise. You need to mix things up.
With a standing desk and a draft chair I'm able to change my position. I can stand when I want to as well as sit.
I probably spent about 75% of my day standing, as I find it more productive. I can concentrate longer, my posture is better. Bonus is that you end up burning around an extra 500 calories a day too while upright. But the chair is there when I need to take a break or vary things.
Take regular breaks, make sure your seat is adjusted optimally. Get an exercise ball and roll around while you are at the computer. stretch your legs and hips and back, just take alittle break once in a while and stretch. Also you can adjust the height of your keyboard, that might make some difference.
Change your chair to those with 5 wheels and adjustable to the correct height. Lean backwards on the chair as you type. This will support your back and prevent it from slouching.