When you reach the highest number, you have to add a digit. In decimal, this is 9. In binary, it's 1.
Digits in binary are also referred to as bits. From this table you can see that you can represent the number 7 in binary with 3 bits: 111 but that it takes 5 bits to represent the number 16 in binary: 10000.
When storing sound as numbers, the bigger the range of numbers you can use, the lower the distortion (THD: total harmonic distortion) and greater the dynamic range.
CDs store audio data using 16 bit numbers. This means that the highest number is 65535 (1111111111111111 binary). This is generally considered adequate for finished product but is not acceptable when recording the tracks since you'll end up adjusting the volume of tracks and usually compressing them which will sometimes take small values bigger. When this happens the sound gets gritty since you are not adding new data.
Wikipedia has more on audio bit depth.
Refer to the Sound page for more info.
Background image by Pauline Moss. at http://paulinemoss.deviantart.com/art/Stars-307119758