Actually, I use the metric system too! I just live in the US now so I'm just putting up with the measurement system here (that my woodshop uses) =)
So for a tight friction fit, would you say it would be best to just measure the outer diameter of the driver, cut out holes that are exactly that size, and sand it until the drivers fit? I've been drilling the center of my cup on a drill press using two circular drill tips: 1 7/8 inches 1/4 inch in, and 1 3/4 inches the rest of the way through. I can't get any more granularity than that using the drill tips available, and these measurements currently don't give me a tight friction fit. The CNC machine will probably offer more precision, so I'm planning to use the exact measured OD of the drivers for these holes when I make them on the CNC machine.
You don't necessary need a friction fit, as you could just find some material to interface between the drivers and the cups. Especially since you've already finished a pair. For your new designs tho, I would definitely go for what you mentioned: approx. 46mm ID of the cups and carefully sand the inside until you feel that the driver can be pressure fit comfortably (remember that the wood will shrink and expand depending on temperature and humidity if the wood is unfinished. This can translate into cracking cups if they're too tight). I usually also round the sharp edges of the glue residue by carefully sanding the drivers by hand.
Notice how the driver OD is approx. 46mm and +-6,5mm high. The next portion has an OD of approx. 44mm and +-3mm high. This usually means that you need at least a 10mm deep slot, of course depending on the ID of the cups.