Given minimal cheaper tools to do the job, I would get a drill (corded or cordless, depends a bit on what else you might use a drill for, really), and at least one bit 1/16" or slightly more, smaller than your smallest hole. To clarify I don't mean a 1/16" bit, I mean just smaller than the target hole. If the budget stretches to more bits, great, or you may need at least two or three depending on what sized step bit(s) you have.
Next is a small sized (Nicholson et al. make more than one size, IIRC, it's the smallest version), fine toothed chainsaw file, available at many hardware stores it is round and slightly smaller diameter than a 1/8" phone jack hole. To compliment this your drill bit needs to be at least that big, the diameter of the file so the holes you drill undersized can be enlarged with it. It has fairly fine teeth but will cut fast enough on Al and plastic, the extra 1-10 minutes you might file is no big deal compared to an extra bit of time hunting around for other ways to do it. For one case that is, I wouldn't want to do a dozen like this.
The idea is that the less control you have over drilling, the more you want to undersize the hole and manually enlarge it. Especially so on plastic end panels, a hand drill with a bit the exact size of the hole you want may have a disappointing result that tears or flares the edges. Not necessarily, but I'm being conservative.