Noise increases precision! A proven engineering principle. EXPLAINED - GregLinthicum/From-Logistic-Regression-to-Long-short-term-memory-RNN GitHub Wiki

Adding the noise solves serious technical limitation: a low precision

In the early days of digital converters and electrical metrology the engineers were really severely limited by number of bits their converter had.

This is why they came with a technical solution that sounds counterintuitive when you first time hear about it.

The principle is: you add noise to the signal in order to make the reading of the system more precise than your converter alone provides.

The trick is that you add more than just a noise you add time and repetitions.

Lets explain using 2 bit converter that covers range of 0 volt to 4 volt. Converter changes to higher state when the predefined physical level of the voltage is reached.

There converter faces the following limitations:

  • cannot express any values smaller than zero
  • cannot express any values bigger than 3
  • cannot express anything else but 0 or 1 or 2 or 3

No Noise

Now let us add a white noise.
White noise is a noise that has the same power at every frequency. For engineering purposes we are happy with a noise that has frequencies higher than the signal under measurement, not much higher than sampling frequency we use BUT does have a perfectly random distribution of the amplitude that is somehow related to the gap between levels that our converter can express ( here assume for simplicity oscillates between -1 and 1 volt around the signal that is being measured.

With Noise

Now we simply add a sampling frequency to the picture.

With Sampling points in Yellow

A system (voltmeter) readout will contain N samples. Here engineering makes friends with math to compute actual precision but intuitively we can say; the more samples in a readout the more precise it can be. Too long readout will be a subject of signal to be measured changing value. Too short might capture a set of samples that is too small to be representative.

In general the instrument will calculate an average of samples within readout. Here for example, let us assume the readouts of 10 samples and 2 sample wasted for internal works of the system between readouts. Then we get:

(2+2+3+2+2+2+3+2+2+2)/10= 2.2 followed by

(2+2+3+2+2+2+2+2+2+2)/10= 2.1 followed by

(2+3+3+2+3+2+3+2+2+3)/10= 2.5

As you see the quality of my noise was not perfect, we would expect 2.7 at the last readout. But as poor as my drawing is it actually illustrates well kinds of problems we face and kind of details we need to take care of in the production solution.


Main Index