Working with APIs the Pythonic Way

For a better reading experience, check out this article on my website.

Image for post
Image for post
It used to be harder

Communication with external services is an integral part of any modern system. Whether it’s a payment service, authentication, analytics or an internal one — systems need to talk to each other.

In this short article we are going to implement a module for communicating with a made-up payment gateway, step by step.

The External Service

Let’s start by defining an imaginary payment service.

To charge a credit card we need a credit card token, an amount to charge (in cents) and some unique ID provided by the client (us):

If the charge was successful we get a 200 OK status with the data from our request, an expiration time for the charge and a transaction ID:

If the charge was not successful we get a 400 status with an error code and an informative message:

There are two error codes we want to handle — 1 = refused, and 2 = stolen.

Naive Implementation

As usual, to get the ball rolling we start with a naive implementation and build from there:

90% of developer will stop here, so what is the problem?

Handling Errors

There are two types of errors we need to handle:

  • HTTP errors such as connection errors, timeout or connection refused.
  • Remote payment errors such as refusal or stolen card.

Our decision to use requests is an internal implementation detail. The consumer of our module shouldn’t have to be aware of that.

To provide a complete API our module must communicate errors.

Let’s start by defining custom error classes:

I previously wrote about the benefits of using a base error class.

Let’s add exception handling and logging to our function:

Great! Our function no longer raises requests exceptions. Important errors such as stolen card or refusal are raised as custom exceptions.

Defining the Response

Our function returns a dict. A dict is a great and flexible data structure, but when you have a defined set of fields you are better off using a more targeted data type.

In every OOP class you learn that everything is an object. While it is true in Java land, Python has a lightweight solution that works better in our case — namedtuple.

A namedtuple is just like it sounds, a tuple where the fields have names. You use it like a class and it consumes less space (even compared to a class with slots).

Let’s define a namedtuple for the charge response:

If the charge was successful, we create a ChargeResponse object:

Our function now returns a ChargeResponse object. Additional processing such as casting and validations can be added easily.

In the case of our imaginary payment gateway, we convert the expiration date to a datetime object. The consumer doesn’t have to guess the date format used by the remote service (when it comes to date formats I am sure we all encountered a fair share of horrors).

By using a custom “class” as the return value we reduce the dependency in the payment vendor‘s serialization format. If the response was an XML, would we still return a dict? That’s just awkward.

Using a session

To skim some extra milliseconds from API calls we can use a session. Requests session uses a connection pool internally. Requests to the same host can benefit from that. We also take the opportunity to add useful configuration such as blocking cookies:

More Actions

Any external service, and a payment service in particular, has more than one action.

The first section of our function takes care of authorization, the request and HTTP errors. The second part handle protocol errors and serialization specific to the charge action.

The first part is relevant to all actions while the second part is specific only to the charge.

Let’s split the function so we can reuse the first part:

This is the entire code.

There is a clear separation between “transport”, serialization, authentication and request processing. We also have a well defined interface to our top level function charge.

To add a new action we define a new return type, call make_payment_request and handle the response the same way:



The challenge with external APIs is that you can’t (or at least, shouldn’t) make calls to them in automated tests.

I want to focus on testing code that uses our payments module rather than testing the actual module.

Luckily, our module has a simple interface so it’s easy to mock.

Let’s test a made up function called charge_user_for_product:

Pretty straight forward — no need to mock the API response. The tests are contained to data structures we defined ourselves and have full control of.

NOTE: Another approach to test a service is to provide two implementations: the real one, and a fake one. Then for tests, inject the fake one.

This is of course, how dependency injection works. Django doesn’t do DI but it utilizes the same concept with “backends” (email, cache, template, etc). For example you can test emails in django by using a test backend, test caching by using in-memory backend, etc.

This also has other advantages in that you can have multiple “real” backends.

Whether you choose to mock the service calls as illustrated above or inject a “fake” service, you must define a proper interface.


We have an external service we want to use in our app. We want to implement a module to communicate with that external service and make it robust, resilient and reusable.

We worked the following steps:

  1. Naive implementation — Fetch using requests and return a json response.
  2. Handled errors — Defined custom errors to catch both transport and remote application errors. The consumer is indifferent to the transport (HTTP, RPC, Web Socket) and implementation details (requests).
  3. Formalize the return value — Used a namedtuple to return a class-like type that represents a response from the remote service. The consumer is now indifferent to the serialization format as well.
  4. Added a session — Skimmed off a few milliseconds from the request and added a place for global connection configuration.
  5. Split request from action — The request part is reusable and new actions can be added more easily.
  6. Test — Mocked calls to our module and replaced them with our own custom exceptions and

Written by

Full Stack Developer, Team Leader, Independent. More from me at

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store