Breakups and Reconciliations, Solving N+1 Across Database Boundaries

In a single database setup, when we need that a Django Database object references another tables, to diminish the amount of atomic searches (N+1) we would use prefetch_related() in case of ManyToMany relationships and select_related() in OneToOne or ManyToOne relationships. But what happens when your data lives in two different databases?

Imagine you’re building a monitoring system. Your time-series events live in a specialized database optimized for write-heavy workloads (think TimescaleDB, InfluxDB, or even a separate PostgreSQL instance). Meanwhile, your reference data—users, devices, locations—lives in your primary application database, and you need to display those events with human-readable context.

Tags: python, Django, Optimization

The only useful Nepo babies, Django class-based views

While I am not a big fan of the use of OOP in all programming problems and paradigms, there are places where I can see their usefulness. One of those places is when creating a simple REST API application. I have known about Django class-based views from a long time, but the thing is, I have always seen them as an after though, because they never are what I want them to be, and before you think about Djando Rest-Framework, I want to tell you that it doesn’t solve either some of the issues that you would normally encounter, like query optimizations. This guide is intended for people who want to avoid infinite dependencies and enjoy designing their own solutions, tailored to their needs.

Tags: python, Django

Python Parallelism for Point Cloud Processing

LAS and its compressed counterpart LAZ are popular file formats for storing Point Cloud information, typically generated by LiDAR technology. LiDAR, or Light Detection and Ranging, is a remote sensing technology used to measure distances and create highly accurate 3D maps of objects and landscapes. The Point Cloud information stored mainly consists of X, Y, and Z coordinates, intensity, color, feature classification, GPS time, and other custom fields provided by the scanner. LAS files comprise millions of points that accurately describe the sensed environment or object, making their analysis a challenging task.

Tags: python, LAS, LAZ, PointCloud, LiDAR

Metaprogram your problems away

Metaprogramming is a useful tool when you want to incorporate general behavior into your program without having to add extensive boilerplate code throughout it. This practice is typically employed by individuals who create frameworks or development tools. The idea behind metaprogramming is to provide you with an initial insight into how such tasks are accomplished, hopefully inspiring ideas for future design projects.

To make sense of all the concepts discussed here, you should be using Python version 3.6 or later and have a basic understanding of decorators, functions, and classes. In the case we’re studying, we will log the names of functions called within an existing Python program.

Tags: python, metaprogramming

File Transfer Between Cloud Services (AWS to Azure)

Sometimes is a requirement to transfer files from one cloud service to another. At least that was the challenge that I encountered while developing a file classification web service. Mid development cycle management changed cloud provider because of some juicy Azure discounts and the all so great office 365 integration that everyone loves.

Because of this, the compromise reached to be able to continue at the same development speed and still lower the cloud services bill, was to continue to work on AWS in the web service, but the files, after finished the process, must be transferred to Azure.

Tags: python, AWS, Azure