Many developers think that having a critical bug in their code is the worst thing that can happen. Well, there is something much worse than that: Having a critical bug in your code and not knowing about it!
To make sure I get notified about critical bugs as soon as possible, I started looking for ways to find anomalies in my data. I quickly found that information about these subjects tend to get very complicated, and involve a lot of ad-hoc tools and dependencies.
I’m not a statistician and not a data scientist, I’m just a developer. Before I introduce dependencies into my system I make sure I really can’t do without them. So, using some high school level statistics and a fair knowledge of SQL, I implemented a simple anomaly detection system that works.
When I started my career in development, my first job was a DBA. Back then, before AWS RDS, Azure, Google Cloud and the rest of them cloud services, there were two types of DBAs:
The Infrastructure DBA was in charge of setting up the database, configuring the storage and taking care of backups and replication. After setting up the database, the infrastructure DBA would pop up from time to time and do some “instance tuning”, things like sizing caches.
The Application DBA got a clean database from the infrastructure DBA, and was in charge of schema design: creating tables, indexes, constraints, and tuning SQL. The application DBA was also the one who implemented ETL processes and data migrations. …
One of my favorite job interview questions is this:
Write a function that returns tomorrow’s date
This looks innocent enough for someone to suggest this as a solution:
This will work, but there is a followup question:
How would you test this function?
Before you move on…. take a second to think about your answer.
Originally published at https://hakibenita.com on June 1, 2020.
In my latest article for RealPython I share three ways to tackle one of the most challenging tasks involving Django migrations: moving a model from one Django app to another.
The article covers some exotic migration operations and many of the built-in migration CLI commands such
sqlsequencereset. In the article I also demonstrate important migrations concepts such as reversible migrations, migration plans and introspection.
Originally published at https://hakibenita.com on May 6, 2020.
Following my previous article on how to build an Interactive Voice Response (IVR) system with Twilio, Python and Django, in this follow-up tutorial I show how to write automated tests for this system.
It can be very challenging to test a system that rely heavily on a third party service such as Twilio. In this article, I show how to organize your code in a way that would isolate your bushiness logic and make it easier to test separately. The article demonstrate useful testing patterns using Django’s
unittest.mock, Pytest fixtures, build-in
django-pytest and many more.
The source code for this article and the previous one can be found here.
Originally published at https://hakibenita.com on May 1, 2020.
As developers, we rely on static analysis tools to check, lint and transform our code. We use these tools to help us be more productive and produce better code. However, when we write content using markdown the tools at our disposal are scarce.
In this article we describe how we developed a Markdown extension to address challenges in managing content using Markdown in Django sites.
Like every website, we have different types of (mostly) static content in places like our home page, FAQ section and “About” page. …
Last year my team and I worked on a very challenging IVR system. After almost a year in production and thousands of processed transactions, I teamed up with the great people over at the Twilio blog to write an introductory tutorial for developing IVR systems using Django and Twilio IVR.
Aside from “making your server talk” and diving into the cool speech features, I found the most challenging part working on IVR is designing the views. Unlike APIs and Forms, IVR is very limited in the type of input it takes (DTMF tones, transcribed speech), and the amount of data it can communicate and process is limited.
Originally published at https://hakibenita.com on February 12, 2020.
Aggregation is a source of confusion in any type of ORM and Django is no different. The documentation provides a variety of examples and cheat-sheets that demonstrate how to group and aggregate data using the ORM, but I decided to approach this from a different angle.
In this article I put QuerySets and SQL side by side. If SQL is where you are most comfortable, this is the Django GROUP BY cheat-sheet for you.
To demonstrate different GROUP BY queries, I will use models from Django’s built-in
On the 20th of March I’ll be giving an online session inspired by this article. The session will feature SQL productivity and performance tips, and a walk through of common mistakes in SQL.
For more details and registration go to SQL Next Steps: Optimization
Most programming languages are designed for professional developers with knowledge of algorithms and data structure. SQL is different.
SQL is used by analysts, data scientists, product managers, designers and many others. …
SQL injection are constantly ranked among the most common attacks against systems. For this reason, ORM’s offer many ways of dealing with injections. A common solution is bind variables, a placeholder in the query that is sanitized by the ORM for safe execution in the database.
However, while binding values is very common, I often find myself needing to use table and column names as variables as well. A stroll through
psycopg2's documentation led me to the discovery of
psycopg2.sql.Literal, two low-level functions for safely binding any type of variable in a query.
This discovery sparked my fourth article for RealPython, “Preventing SQL Injection Attacks With Python”. If you’re not sure what SQL injection is, this article will walk you through everything you need to know. If you are an ORM veteran, check your knowledge and get yourself familiar with the low level