This article dives into the complexity of Python dependency management and how projects can easily catch "fire" as they grow and evolve. It shares numerous techniques, tools, and best practices to improve "fire safety," analyzes the pros and cons of different tools, and provides usage recommendations for various scenarios.
TL;DR
Python dependency management gets called a dumpster fire in a fine-grained look at inter-project bloat, PyPI rolls out a new generation of attestations, and PyPIM executes code straight in RAM.
Articles & Tutorials
× 12A paper from the Association for Computing Machinery (ACM) that analyzes the source code of 1,302 popular Python open-source projects and their 3,232 transitive dependencies. It builds a fine-grained project dependency graph, performs reachability analysis, and discovers that over 50% of dependencies are bloated.
The author shares a Django project template based on uv and just, covering recommended directory structures, configuration management, environment variables and secrets management, web UI, deployment, and maintenance.
Django's default Paginator relies on COUNT(*) queries to calculate page numbers, which can be time-consuming. This article explains how to implement custom paginators to improve pagination API query performance. The example demonstrated improves from 855ms to 12ms.
When building AI applications, text needs to be tokenized. The author discovered that tokenizer selection and optimization are crucial for RAG effectiveness. The article discusses different tokenizer types (word-based, character-based, and subword) and their impact on model performance.
PEP-8 recommends checking if a list is empty using "if seq" rather than "if len(seq)." Some argue the first method is ambiguous. This article refutes that viewpoint, advocating for meaningful variable names, type hints, clear function names, and docstrings to improve code readability and reduce ambiguity.
PyPI introduces a security feature from PEP-740 specification: digital attestations. It improves upon traditional PGP signatures by enhancing key availability, index verifiability, cryptographic strength, and provenance attributes.
Israeli researchers developed a new software "platform" that converts easy-to-read Python instructions into low-level machine code and executes it directly in RAM without going through the CPU. This new Processing-In-Memory (PIM) architecture significantly improves code performance.
The Pipeline pattern, also known as the Chain of Command pattern. This article presents an example using functools.reduce, partial functions, and the BeautifulSoup library to parse HTML content, explaining the usage and benefits of this pattern.
Blue-green deployment is a common strategy to prevent service downtime during deployments. This article explores what blue-green deployment is, why modifying databases during blue-green deployment can be tricky, and how to handle common database changes using a Django project as an example.
Workbench is a remote development workspace launched by Netflix that recently experienced slow and unresponsive JupyterLab UI issues. This article reviews the detailed troubleshooting process, from UI debugging all the way down to the Linux kernel.
How to use machine learning models in Go projects? This article covers several approaches, including using existing LLM APIs, running open-source models locally, and integrating trained models through a Python sidecar pattern. The article focuses on explaining and demonstrating the last approach.
Projects & Resources
× 12Transform inaccessible nested dictionaries into accessible attributes, allowing you to use dot notation to access dictionary elements, making code more concise and readable. (2.6K stars)
Supports element interaction, structured data extraction, authentication, file upload/download, and browsing without being blocked.
Features colorful terminal printing, text styling (background colors, bold, underline, etc.), custom styles, border and table creation, and intelligent defaults.
Read epub ebooks in the terminal using Vim-like commands, with features to chat with current text, generate summaries, and text-to-speech functionality.
A new Python toolbox for interpretable machine learning model development and validation. Through low-code interfaces and high-code APIs, it supports an increasing number of inherently interpretable ML models. (1.2K stars)
Extract high-quality content from complex and diverse PDF documents, integrating leading document parsing models, high-quality parsing across different documents, modular design, and providing diverse and comprehensive PDF evaluation benchmarks. (5.6K stars)
A one-stop video translation and localization dubbing tool that generates Netflix-level high-quality subtitles with one click, eliminating stiff machine translations and multi-line subtitles, while adding high-quality dubbing to let knowledge cross language barriers worldwide. (6.3K stars)
An async-first pure Python rendering engine with React-like context support, built-in baseline HTML tags, Markdown support, JSON-based internationalization, and easy customization of everything.
Automatically scrapes data hourly from multiple platforms' hot rankings, archived daily, with data from October 25, 2023 to present.
Implementation and pre-trained models for the Watermark Anything paper, supporting embedding multiple localized watermarks in images.
Very advanced video generation models, completely open-source, with high-fidelity motion and strong prompt adherence. (2K stars)
Not only recognizes faces but also analyzes person attributes (age, gender, emotion, race). Face recognition accuracy reaches 97.53%. (14.7K stars)
Discussions & Questions
× 2What "defaults" are currently being promoted in Python teams?
Why do some highly-rated beginner books not teach these things? Is it problematic not to learn them?