ImportPython Newsletter - Issue 188
From ImportPython Blog

by Ankur Gupta

Worthy Read

The clean architecture is the opposite of spaghetti code, where everything is interlaced and there are no single elements that can be easily detached from the rest and replaced without the whole system collapsing. The main point of the clean architecture is to make clear "what is where and why", and this should be your first concern while you design and implement a software system, whatever architecture or development methodology you want to follow.

Welcome to the Python Graph Gallery. This website displays hundreds of charts, always providing the reproducible python code! It aims to showcase the awesome dataviz possibilities of python and to help you benefit it. Feel free to propose a chart or report a bug. Any feedback is highly welcome. Get in touch with the gallery by following it on Twitter, Facebook, or by subscribing to the blog. Note that this online course  is another good resource to learn dataviz with python.

Join me along with Steve Dower (a core dev working at Microsoft), who just published an amazing retrospective of Python at Microsoft entitled: Python at Microsoft: flying under the radar.

See how Python compared with other languages.

Facebook prophet is by far my favorite python package. It allows for quick and easy forecasting of many time series with a novel bayesian model, that estimates various parameters using a general additive model. More on Facebook prophet can be found right here. Forecasting is important many business situations, particularly supply chain management and demand planning.

If you’ve ever written a program and printed out a value to see what’s going on during execution, then you understand at some level why logging is so valuable. Knowing what’s happening in your code at a point in time is enormously useful from both technical and business perspectives. This knowledge lets developers and product managers make smart choices about what systems to fix or modify and lets them see what actions users take when they use your software.

FTDI chips are frequently used as USB-to-serial adaptors, but the newer devices have the ability to drive more complex protocols such as SPI and I2C.I like to use Python when first experimenting with new PC hardware, and there are some Python libraries for interfacing to FTDI chips, but I couldn’t find any real projects or complete worked examples.

Advent of Code is an Advent calendar of small programming puzzles for a variety of skill sets and skill levels that can be solved in any programming language you like. People use them as a speed contest, interview prep, company training, university coursework, practice problems, or to challenge each other.

And me, well... I just blurted it out: I hate Python. I hate it with a passion. If I have the choice between using some pre-existing Python code or rewriting it in C, I'd rather rewrite it in C. Curator's Note - Someone give him a Django website to rewrite in C. ;) ... Nevertheless it's good to hear critique and ponder over it.

A new thread was started on the Python committers Discourse instance to discuss the pros and cons of various voting systems. Instant-runoff voting fell out of favor; there were concerns that it didn't truly represent the will of the electorate, as seen in a Burlington, Vermont mayoral election in 2009, for example. The fact that it was put in place by fiat under a self-imposed deadline based on in-person conversations at the core developer sprint, rather than being hashed out on the Discourse instance or the python-committers mailing list may have also been a factor.

Like many PyData developers, I’m loosely aware that GPUs are sometimes fast, but don’t deal with them often enough to have strong feeling about them.To get a more visceral feel for the performance differences, I logged into a GPU machine, opened up CuPy (a Numpy-like GPU library developed mostly by Chainer in Japan) and cuDF (a Pandas-like library in development at NVIDIA) and did a couple of small speed comparisons:

Perhaps you’ve faced the fortunate challenge of scaling up a Python application to accommodate a steadily increasing user base. Though most cloud hosting providers make it easier than ever to throw more hardware at a problem, there comes a point when the cost outweighs convenience.Around the time scaling horizontally starts looking less attractive, developers turn to performance tuning to make applications more efficient. In the Python community there are a number of tools to help in this arena; from the built-in timeit module to profiling tools like cProfile, there are quick ways to test the difference between a particular line of code and any of its alternatives.Although profiling tools help you see important information about which calls in your application are time consuming, it’s difficult to exercise an application during local development the same way your users exercise it in real life. The solution to bridging this gap? Profile in production!PyflamePyflame is a profiling tool

Python IO streams: BytesIO and StringIO in practice.

I hope you had an exciting intro to Numpy because Pandas builds on it. What is Pandas? Forget about the song ‘Panda’ by American hiphop artiste Desiigner or the animal (KungFu Panda enthusiasts), this is a python library that offers powerful and flexible data structures that make data manipulation and analysis easy. It stands for “Python Data Analysis Library” and according to Wikipedia, the name Pandas is derived from the term “panel data”, an econometrics term for multidimensional structured data sets.

A collection of Tips & Tricks to better your code.

I’ve recently launched Homemade Machine Learning repository that contains examples of popular machine learning algorithms and approaches (like linear/logistic regressions, K-Means clustering, neural networks) implemented in Python with mathematics behind them being explained. Each algorithm has interactive Jupyter Notebook demo that allows you to play with training data, algorithms configurations and immediately see the results, charts, and predictions right in your browser.

It is hassle-free and saves time.

Recently facebook opensourced Pytext, for rapid prototyping and deploying deep-learning based models. I took it for a spin, and documented few examples on it. especially RNN and LSTMs.

Sampling, Chunking, Optimising Pandas dtypes, Parallelising Pandas with Dask.

This is a list of the best libraries and packages that changed our lives this year, compiled from my weekly digests.


Super-SloMo - 1053 Stars, 122 Fork
PyTorch implementation of Super SloMo by Jiang et al.

uncaptcha2 - 890 Stars, 75 Fork
defeating the latest version of ReCaptcha with 91% accuracy

PixelGame - 31 Stars, 4 Fork
A free game

bandersnatch-graph - 28 Stars, 3 Fork
Graphing all possibilities in the Netflix Black Mirror episode, "Bandersnatch"

sceutils - 28 Stars, 2 Fork
SCE decryption utilities

pdf-split-merge - 23 Stars, 0 Fork
simple pdf file split and merge tool

profanity-check - 23 Stars, 0 Fork
A fast, robust Python library to check for offensive language in strings.

educational_python_scripts - 13 Stars, 6 Fork
Some small sctipts I use while teaching python

Tacotron-Wavenet-Vocoder - 12 Stars, 5 Fork
Tacotron, Wavenet-Vocoder, Korean TTS

multiprocess unsupervised chinese_detect_words ngram_combination

Code example used in the tutorial "How to Implement Grouped Model Choice Field"

Project-domCAP - 5 Stars, 2 Fork
Random Activity generator for Gmails

Ankur has coded and deployed numerous Python software over the last 10 years, at three venture funded startup and a fortune 10 company. He currently heads Numerate Labs. ImportPython is his side project with Python being his go to programming language.

Blog Categories