The shock indicates there's definitely a problem somewhere. The basic strategy to tracking this down is divide and conquer--plugging and unplugging things until you can isolate the problem to one or more components and/or connectors.
First, make sure you're measuring correctly. You need to measure at the outlet itself with the meter set to the ac voltage setting that allows more than the voltage you expect: 110, 220, whatever it is by you. (e.g. On my meter it would be the 300vac setting.) Make sure you're not trying to measure DC voltage.
As you connect and remove components, the test should always be the same. Always measure the same way at the same outlet and touch the computer to check for shock. Be careful when you touch the computer. Typically is these situations, the current is too small to hurt you, and any damage will be collateral from you jumping and dropping stuff. Just tap it while you have good balance and don't hold anything or touch anything else.
The obvious suspect is the computer itself, so unplug it (and all peripherals connected to it) and check the outlet voltage again. Compare the voltage with and without the computer plugged in. If the voltage is normal with the computer unplugged, the problem is in or connected to the computer. Plug the computer back in but not the peripherals, and disconnect the computer from any powered components (printers, speakers, LAN cables, etc.), and measure again. (You may need to disconnect the monitor and measure, since that may be the faulty component.) If the voltage is normal without anything connected to the computer, the problem is in some other component connected to the computer, or in how those components are connected.
At this point, it makes sense to turn to a general approach. The approach is still to isolate the faulty component, but keeping in mind that any wire could be responsible: audio or digital cables, LAN cables, etc. It may also be a ground loop involving more than one component connected through the power line. The basic idea is to keep the computer plugged in and do the measurement/shock test while you plug/unplug other things. If the problem is in the computer only when it's connected to something, the following will still help you isolate the component, but you'll only check the peripherals, not everything on the power line (or at least start with those). The difference is only in which components you unplug and connect.
So the procedure is as follows. Unplug devices from the power one at a time (and disconnect any other wires it has connecting to other devices) and measure until you get a normal voltage again. Do it in thie following order. Start with anything connected to the computer. Then do those components mentioned above. Finally, start unplugging everything else. If you still can't restore the voltage to normal, disconnect everything--including refrigerators, lights, etc. If that doesn't help, get a new meter or ask someone knowledgable to check it with their meter. At that point, call an electrician.
The main suspects for your symptoms are old television sets, old or cheap stereo equipment, radios, powered speakers, and other electronics, especially those with a two prong unpolarized plug--unpolarized plugs have both prongs the same--polarized plugs have one prong wider or longer than the other. If you have anything like that anywhere on the electrical line (not just the outlet, but the line to the main junction box, i.e. all outlets on the same fuse or breaker), turn the plug around and measure again and touch the computer to see if the shock goes away. Do them one at a time. Of course, do the disconnect test first, and if you find the problem goes away when you unplug one of those unpolarized jobs, turn the plug around and test again. Unfortunately, it's not rare for polarized equipment to be wired incorrectly, in which case you need to rewire it or throw it out.
A little background is needed here. There are different types of cases, but in typical consumer audio equipment, a metal case is often connected to ground. The problem is there are different grounds. Equipment with a three prong power plug usually has the case connected to earth ground--the third prong on the plug. Equipment with a two prong plug is usually, but not always, floating. This means the case is connected to the transformer, but the side that's isolated from the power line. Some two prong audio equipment is connected directly to the power return line. The problem is that with an unpolarized plug, there's no way to know which way to plug it in, so the chassis might be hot or neutral. Connect two different pieces of equipment connected like this but plugged in differently, and you develop a voltage across the interconnect. It's literaly shocking! Fortunately, most of this equipment is designed so that the low impedance path for current does the right thing, so there's very little current flowing across the interconnects, and the shock has very little current. Of course, this small current is enough to overwhelm the sound. There's also quite a lot of equipment with polarized plugs that are wired wrong, and end up with a hot chassis. Televisions are a disaster in terms of grounding, but fortunately, the chassis aren't grounded, so you're only in real danger if you open the case, but it can caue all of these problems if you connect one to your stereo. There's also a lot of equipment that uses cheap shortcuts in the power supply that result in failures with similar symptoms. The worst of these is the thermistor (might be an MOV device) that are notorious for being unreliable. This means any piece of cheap equipment (like powered computer speakers or that cheap receiver) can easily fail like this.
One overlooked thing that can cause this is interconnects. If you connect the right (wrong?) types of equipment, especially if one component is one of those unpolarized jobs mentioned above, it can directly connect a hot chassis to a neutral or grounded chassis. Just turning the plug around can sometimes solve this kind of problem. This polarization issue is the main suspect because it will change the line voltage and cause a shock exactly as you describe. The procedure above will work to locate this, but there's one thing more to try. This applies to any devices, because even the best eqipment might develop a fault. There's a chance you have a more complicated scenario involving interconnects, but it's unlikely unless you did something weird like use an unconventional cable with a special grounding scheme--it happens around here because people mix balanced or professional audio equipment with consumer equipment. The other case where you run into this is when equipment develops a fault.
Disconnect any interconnect cables between the computer or stereo and other powered devices and check again to see if you get shocked or if the voltage goes back to normal. After you've gone through the whole unplugging routine, plug in all the peripherals to the power line, but disconnect any interconnects. Start connecting them one at a time and measure again.
You may have more than one device on the line that causes this kind of problem. It's always more difficult to track down multiple problems. Once you find a component that you can connect/disconnect plug/unplug, work backwards unplugging & disconnecting other things to make sure the problem isn't caused by some combination of two connected components. One or more may be faulty.
Make sure you understand the purpose of the procedures before starting or you risk wasting a lot of time. Ask questions before you start.
Once you isolate the problem, you can work on fixing it. That could be even more interesting.
Is this what you have? Type C (European 2-pin)
http://www.kropla.com/!c.htm
If so, there's no question of earth ground. That's unpolarized, so you might have to play around with which direction you plug things in. Do you have any equipment with a three prong power plug? If so, how do you connect it?
Finally, if you live where I think you live, you may really just have a low voltage coming from the power company. But 60 is ridiculously low, since that starts at 220vac. The only way to correct it is with a special type of UPS, battery system, or voltage regulator.
