Advice for Mr. Nikhil Bikhchandani of Verily

A Feynman moment - remember in Surely, he says he's reviewing some textbooks where he sees a problem posed as : There are four stars, Star A is at temperature 5000K, Star B at blah, etc.. What is the total temperature of the stars? And he would explode!

Anyhow, this excellent if Ambienic mensch of Udacity sprach also :

If we plot the FFT of just one portion of the signal, we see that the main frequency component is pretty small.

Yikes - that doesn't sound good do it? Use small and large to talk about amplitude dude!

For frequency, use high/low. Please!

But, a nice concept - the short time Fourier transform.

When there are frequency components changing with time, it doesn't make sense to analyze the entire thing in one shot..

He gives an example of a signal whose instantaneous frequency changes linearly from 0 to 10 Hz (f0 to f1).

But, one could easily concoct a signal with that and also a component whose frequency linearly goes from 10 to 0. Why not?:)

And, yes, please, Mr. N, stop using np.pi and please do import pi from numpy as PI :)

fs = 200
T = 120
f0, f1 = 0, 10
ts = np.arange(0, T, 1/fs)
c1 = (f1 - f0) / T
c2 = f1 - (f1-f0)/T
sig = np.sin(2 * PI * (c / 2 * np.square(ts) + f0 * ts)) + np.sin(2 * PI * (-c / 2 * np.square(ts) + f1 * ts))

plt.clf()
plt.title('Spectrogram')
plt.specgram(sig, Fs=fs, NFFT=1000, noverlap=75, xextent=((0, T)))
plt.ylim((0, 20))
plt.xlabel('Time (sec)')
plt.ylabel('Frequency (Hz)')

A design pattern you'd get out of this.

You have a waveform and now you have an algorithm that identifies interesting points on this waveform. How can you tell how well your algorithm has done?

Visually, it's super easy to identify the points on the plot where your algo should have put a marker.

So, how do you QC your algorithm visually, to pass judgement?

matplotlib's pyplot.plot can plot just a sequence (list), in which case the x-axis will be the index of the element, or, you can give it x,y in which case, x and y are lists, or you can give it the NumPy ndarray, in which case, the [0] is the list of x-vals and [1] is the list of y-vals.

Point? Your algorithm (i.e. whatever function your using to process the aforementioned y-val list) will return a bunch of x-vals that specify where it thinks the events happened.

You now want to mark on the original plot, these events. Meaning? You got to concoct a new x,y pair for pyplot.plot that can do this. How? Easy, thanks to python. If you take a list and then just it

listVar[ list_of_indices], that's all it takes to pull the new list out of the existing one.

So, then, you just plot 

plt.plot( algo_returned_interesting_x_val_list, signal_yval_list[ algo_returned_interesting_x_val_list] )

And you're done :)

For example, say you have an ECG (not really an ECG, but who cares :) :

Now, your algorithm reports what it things are the peaks, which you'll then use to determine the heart rate :

This is enough to tell you that your algorithm isn't good enough and you need something better..

And you need to tweak :

  OMG!!! I get to a place where this guy computes the signal to N ratio the FFT and I'm going WT*. He doesn't square the amplitudes to get the power. What is wrong with this guy? Say he's right, shouldn't he at least address such a fundamental question? OMG! Shame on your Carnegie Mellon!

Please Sir. In future, when thou doest

%matplotlib osx

do

%matplotlib osx    # windows and linux users use qt4

You may have mentioned qt4 somewhere earlier, but we're not going to remember everything. Up *your* game!

And yes, please import only what you actually use :) Imposter syndrome is bad enough. The more you import, the less up to the task I feel :) Which? The Data Exploration walkthrough in the Activity Classifier section.