Soreness in the muscle during exercise is known as 'muscle failure', and occurs when metabolic waste and other compounds build up in the muscle cell and hinder the ability of the muscle to contract. Muscle failure goes away relatively quickly.
Prolonged muscle soreness is called 'Delayed Onset Muscle Soreness', or DOMS for short. It is caused by similar mechanisms as above, but is more delayed in its clearance and return to norm. This is the soreness that somebody would feel after a hard run or leg session and be unable to easily walk up stairs the next day.
There are various things that contribute towards DOMS. Including:
Lactic Acid is commonly thought to induce muscle failure or soreness, but has not been shown to be the causative factor. It is highly correlated though, as lactic acid is produced when the muscle is sore; however its more of a fuel source than a soreness causing agent.
There are various mechanisms of reducing DOMS.
Some light exercise, or moving the affected joints and muscles, can also alleviate DOMS. This may be due to merely moving the affected muscles, as 'whole-body vibration therapy' has recently been suggested to do similar.
Cryotherapy, or ice water submerging, has beneficial anecdotes by various athletes but has not been shown to be of much benefit in randomized controlled trials. That being said, cryotherapy may hold some potential in cases of muscular trauma such as hamstring tears or intense muscular strains. Cryotherapy's effectiveness may be closely tied in to the degree of muscular damage, and serve as a bridge between such anecdotes in high level athletes and a lack of results in novice trainees in intervention studies.
Proper pre-workout nutrition can also play a role in preventing DOMS, as BCAA supplementation has been shown to be beneficial (and can be consumed through Whey Protein or protein-containing foods)
Stretching (static) before or after exercise is not significantly effective in reducing DOMS from exercise.