pandas with hundreds of millions of rows

The problem We want to find out which are the top #5 American airports with the largest average (mean) delay on domestic flights. Data We will be using the Data Expo 2009: Airline on time data dataset from the Harvard Dataverse. The data consists...

comments

Python operators and how they affect pandas

Python is today among the most popular programming languages. And in my opinion, there is one main reason for it, readability. A clear example of how Python was designed to be readable is next example: if 'melon' not in ('apple', 'coconut'):...

comments

An update on the pandas documentation

Some context This post is mainly a technical post on what's the status of the pandas documentation. But let me provide a bit of context on where this comes from. It's a personal opinion, but I think pandas is one of the clearest examples of how...

comments

New pandas workflow

Some exciting news. After some years of organizing sprints, and maintaining open source, I've been thinking on a more efficient workflow for projects with high volume of activity, like pandas. An exaggerated example would be that I want to create...

comments

Dataframe summit @ EuroSciPy write up

Last week took place in Bilbao, Spain, EuroSciPy 2019. This year we introduced the maintainers track a room dedicated to discussions among maintainers. The idea is similar to the birds of a feather or unconference sessions of other conferences....

comments

pandas: The two cultures

Leo Breiman was a distinguished statistician at UC Berkeley, known among other things for his major contributions to CART (decision trees), and ensemble techniques, mainly bootstrap aggregation. Combining both, he was able to define one of the...

comments

#pandasSprint write-up

The past 10th of March took place #pandasSprint. To the best of my knowledge, an unprecedented kind of event, where around 500 people worked together in improving the documentation of the popular pandas library. As one of the people involved in...

comments