My intuition is that you could take one tooth and find one isosceles triangle with two other equidistant teeth and find the diameter by bisecting one angle. You find that isosceles triangle with a measuring tool and just choosing two teeth that give you the same distance from the tooth in question. I figure most parts would be made using nominal diameters so if you choose the teeth which give you the narrowest angle you should be close enough with that technique. Then again, I've never worked in a machine shop so I have no idea how these things actually work in practice.
