Killer Jupyter Notebook Tips

A keeper

https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/

 

https://towardsdatascience.com/jupyter-notebook-hints-1f26b08429ad

How to Clear Cell Output : 

ESC - R

will do it. ESC goes into command mode.

R then takes you into Raw mode, which clears the output.

And there you have it :)

Igor Tulchinsky's Four-Month Data Science Course

1	Program flow, data structures
2	Data structures, algorithms, classes
3	Data formats
4	Multi-dimensional arrays and vectorization in NumPy
5	DataFrame, Series, data ingestion and transformation with pandas
6	Data aggregation in pandas
7	SQL and Object-Relational Mapping
8	Data munging
9	Introduction to Machine Learning
10	Regression and classification
11	Metrics and overfitting
12	Model selection
13	Principal Component Analysis and Dimensionality Reduction, feature engineering
14	Statistical methods and nonparametric analysis, probability distributions
15	Ensemble methods
16	Support Vector Machine and Natural Language Processing

https://wqu.org/programs/data-science

This by the way is the man who claims to find peace disturbing - "I'm serious. I'm so used to fast pace of things and ideas and emails and events happening that, if you take it away from me, I'll have a withdrawal."

How to Column Grep a CSV File (Perl)

You know what I mean.. only show rows where entries in a particular column match a certain criterion.

You only want to see the rows, where field number 10 (index 11 for hackers :) matches your regex. How?

Good news is, phew, perl trumps awk hands down AFAIK.

$ > perl -F, -ane 'print if $F[ordinal_field_num_minus_1] =~ /regex/;' filename

Does it.

So, if you're interested in the first column (i.e., ordinal number is 1, then use $F[0]. YGTI :)

Wha?

-a is the autosplit option - it auto splits each line read in into the @F array, but it uses ' ' (space) as the character to split on. You change that using -F (same as awk). Note that you can do -F'regex' if you want, for -F'[:,]'

If you need to get up to speed on -a, -n, -e, -F and the very useful -l, read $ > man perlrun

pandas : drop rows with null only from specific columns

 aka : drop rows for certain columns if null

Tough one to find - and of course, as usual, DuckDuckGone is totally useless..

Thank you Google : 

df.dropna( axis=0, subset=column_list )

M$ Excel : How to Capture the Max of Only the Displayed Values of a Column in a Table

MAX is not your friend. SUBTOTAL is. How should you know that? Don't ask me.

So, you have your beautiful data and, you've done the Format as Table trick.

Now, you can quickly do filtering on it - to see how good/bad things are under different conditions.

Thing is, can you have a cell somewhere up top that shows you the MAX of your table, and uses ONLY THE DISPLAYED VALUES. Else, what's the point. Right?

Thanks to these (https://exceljet.net/excel-functions/excel-subtotal-function) guys, here's what you do :

You start typing : =subtotal

You'll get prompted by Excel with SUBTOTAL. Hit Tab key to select that and now, you have to choose the function. If you refer above page, you'll find that the version of the function whose number has "10" in front (i.e., 101 vs 1) ignores hidden values. So, for MAX, it's 104. Thankfully, Excel gives you a gorgeous drop-down menu that makes life super easy.

So, if you've named your table (good idea:), you'll end up with something like

=SUBTOTAL( 104, myTable[@col_name])

And it'll work great!!