That means that the maximum is somewhere in between 9,990,000,000 and 9,999,999,900.
Set the monster Phyrus's experience to 9990000000 and see if he gives you that much experience.
IF he does, then it means you have went past the max value with 9999999900 so you will have to make it less until it works.
The current limitation is 2,147,483,647 experience, correct?
You probably need to dig into the source code and change the data type for monster experience.