Energy Transport in Stars

Energy is transported by three mechanisms

Radiation - photons carry energy away from the star's center
Convection - cells of hot gas move up, cells of cool gas move down
Conduction - collisions between electrons can move energy outwards

In most stars, conduction is not important.

So we have a situation where a lots of energy is being created in the middle of a star. And it's coming out. Why?

Radiation

Photons carry energy away. But photons also can be absorbed (and re-emitted) by atoms. So photons "random walk" their way out of a star, constantly being absorbed and re-emitted.
The efficiency of this process is clearly linked to how much absorbing is going on. If a photon is frequently absorbed, the efficiency of radiative transport is low -- the photon can't travel very quickly. The absorbing power of material is called its opacity.

If the opacity is high, or if the density is high, or if the flux of photons is high, radiative transport alone is not sufficient to get the energy out fast enough. What is meant by "fast enough?"

Under these situations, convection takes over.

Convection

Hot gas rises, cool gas falls. Bulk motion inside the star moves energy outwards. If a parcel of gas is heated (ie by energy flux from deeper inside the star) to temperatures which are larger than the surrounding gas, it will rise. As it rises, it expands and cools (remember the equation of state?). Once it thermalizes, it will stop rising.
A side effect: mixing!

When does convection win? When does radiation win?

Let's look at two examples.

1. Opacity

Opacity comes from photons being absorbed to ionize or excite atoms. Stars like the Sun (low mass stars) have temperatures in their outer envelopes which are low enough that hydrogen is not ionized. So higher energy photons from the interior of the star are easily absorbed by the hydrogen -- the outer portions of low mass stars have high opacity, and are thus convective. In high mass stars, the temperature is high enough that the hydrogen stays ionized, so radiation is not so easily absorbed -- high mass stars have radiative envelopes.

2. Energy Production

Stars have temperature gradients -- the deeper in you go, the hotter they are. If the energy production mechanism inside a star is very sensitive to temperature, a strong radiative flux is set up. A piece of the star has a lot of energy coming through it from below, and not much energy leaving from above. It can't move this energy fast enough by radiation, so convection kicks in.
Low mass stars operate via the p-p chain, which has a relatively weak temperature dependance (E ~ T⁴). Radiative transport can handle the energy flux, so low mass stars have radiative cores. High mass stars operate via the CNO cycle which has a much stronger temperature dependence (E ~ T²⁰). This is too strong for radiative transport, so high mass stars have convective cores.

So a fundamental difference in the structure of low mass and high mass stars:

Low mass stars: radiative cores, convective envelopes
High mass stars: convective cores, radiative envelopes