I looked into it this morning. The fix is to change:
if (normal_random(0, spellBlock.chance) == 0 && (master || health > mType->info.runAwayHealth || normal_random(1, 3) == 1))
to:
if (uniform_random(0, spellBlock.chance) == 0 && (master || health > mType->info.runAwayHealth || uniform_random(1, 3) == 1))
in monsters.cpp
normal_random uses a normal distribution for generating random numbers, this means that numbers closer to the average are generated most often. Numbers at the extrema, such as 0(in this example), are generated less often. This is useful for getting more consistent damage numbers for instance. Uniform random will give you the expected behavior.
There's other instances in the code where normal_random is also used incorrectly, I believe. For instance, with spears break chance. You might have noticed spears are very durable by default.
I also believe this might be more correct:
if (uniform_random(1, spellBlock.chance) == 1 && (master || health > mType->info.runAwayHealth || uniform_random(1, 3) == 1))
Unless spellBlock.chance is decremented at some point, 0 would make a spell that's say a '7' chance in the files, a 1 in 8 chance (0-7).