Python's official 7th annual developer survey results are out! Over 25,000 Python developers from nearly 200 countries and regions participated in the survey. Let's see what new trends emerged this year, what changed, and what stayed the same?
TL;DR
The Python Developers Survey 2023 results land, chDB makes Pandas queries 87x faster, and HivisionIDPhotos turns out AI ID photos in seconds.
Articles & Tutorials
× 12When developing Python projects with Docker, is it still necessary to use virtual environments? The author's recommendation is yes, and the article shares several reasons: predictability and familiarity of directory structure, unified standards for easier communication, reduced code import lookups, etc.
The author is a computational biologist who shares their experience using Python for data analysis, including virtual environments, dependency management, module management, programming practice recommendations, and more.
How to process a file containing one billion weather data records in the shortest time? The author took on this challenge with Python, optimizing from 8 minutes to just over half a minute in their local environment, sharing some Python optimization tricks.
Using an example of preparing coffee and tea, this introduces the Template Method design pattern for creating flexible and reusable code structures that are easier to maintain and extend.
"Hidden gems" in Python's standard library - some are actually quite commonly used, like collection and contextlib, but others are truly rare like fractions, webbrowser, and zipapp.
There are two common caching strategies: time-based and least recently used. The article introduces both caching approaches and implements a caching algorithm from scratch in Python that combines both strategies.
A deep dive into the design and implementation of the CPython virtual machine, including bytecode format, the interpreter's bytecode evaluation loop, and how stack frames and computed jumps are used to optimize performance. Finally, it analyzes the execution process of a simple Python program, showing how the VM manages and executes compiled bytecode.
chDB is an embedded version of ClickHouse that runs in-process. Its version 2.0 is 87x faster than 1.0 when querying Pandas DataFrames. The article explains how this was achieved: reducing serialization/deserialization steps, directly reading Python objects, batch data copying, and rewriting Python string encoding/decoding logic in C++.
Beginners who install Anaconda on macOS might not be able to find a way to run the official Python. The article introduces a simple solution: moving the .zshrc file out of the home directory to escape Anaconda's control over the Python environment.
tracemalloc is a Python standard library that can track memory allocation. The article shows how to use it to implement a simple memory estimation function, compares it with sys.getsizeof(), and examines the memory size of importing numpy. (Also: What should you pay attention to when Python calculates memory?)
What habits do efficient, excellent software engineers have? The author summarizes some important points, such as focusing beyond just the code itself, pursuing efficiency, enjoying fixing things, systematic thinking, knowledge transfer, simplifying complex problems, etc.
Projects & Resources
× 12Interactively visualize decision trees in Jupyter Notebooks, Jupyter Lab, Google Colab, and other notebooks that support HTML rendering.
A versatile toolbox supporting screen recording, audio recording, broadcasting, OCR, webcam, and other features.
Python training course from JPMorgan Chase for business analysts and traders. (5.7K stars)
For interacting with iPhone and other devices, supporting device discovery, TCP port forwarding, syslog viewing, app management, file management, firmware updates, notification monitoring and triggering, screenshots, backups, etc. (1.3K stars)
Play music using Xiaomi AI speakers, with music downloaded via yt-dlp. Supports multiple voice commands, multiple speaker devices, and various music formats. (1.4K stars)
Customizable pomodoro timer with strict distraction prevention (disables key combinations that might interfere with focus, prevents mouse from moving to screen edges), forced fullscreen, and user-friendly interface.
Designed for preemptive concurrency and parallelism, using thread pools for concurrency and extending to multiprocessing for parallelism. Thread pools support automatic idle shrinking.
Simple, customizable status page generator that can monitor the status of various services like HTTP endpoints, ping hosts, check ports, etc.
Build knowledge graphs that represent complex relationships between entities that change over time. Supports extracting unstructured and structured data, and can query the generated graphs through time, full-text, semantic, and graph algorithms.
Powerful music and audio analysis tool. (7K stars)
Aimed at automatically gathering information, scanning vulnerabilities, and generating reports, including services, bugs, vulnerabilities, misconfigurations, etc. Uses TCP SYN, ACK, ICMP, and other protocols to detect and bypass firewall/IDS/IPS devices. (3.3K stars)
Intelligent ID photo creation tool that can perform lightweight background removal and generate ID photos of different sizes. (7.2K stars)