Python Trending Weekly
ISSUE 65

CSV Is Kinda Bad

2024-08-17 Season 3

TL;DR

Why CSVs fall short and DSVs work better, PEP-750's tag strings for writing DSLs, and 350 million tokens parsed to reveal what Hacker News loves and hates.

Articles 12 Projects 12
🦄

Articles & Tutorials

× 12
01 CSVs Are Kinda Bad. DSVs Are Kinda Good matthodges.com

As we all know, CSV files use commas to separate content, but this doesn't work well for situations involving commas, separators, and line breaks. The article suggests using ASCII's universal separators as an alternative, which can handle complex data and avoid formatting errors.

02 Crawling Pages with Infinite Scroll using Scrapy and Playwright xiegerts.com

Introduces how to use the scrapy-playwright plugin to scrape content from infinite scroll pages, including configuration, handling CORS issues, principles of scroll scraping, and detailed implementation code.

03 Testing with Python (part 9): the extra mile bitecode.dev

Introduces how to calculate test coverage for your codebase using pytest-cov, and use tools like nox for combination testing across different versions of Python and libraries. Testing both the code itself and its integration with the entire Python ecosystem.

04 fastcore: An Underrated Python Library fastpages.fast.ai

The author of Fastcore introduces the main features of this library, which is primarily used to extend Python's capabilities, including: making **kwargs transparent, avoiding boilerplate code when setting instance attributes, avoiding subclass boilerplate, type dispatch, better functools.partial, more useful __repr__, and more.

05 State of Knuckledragger, a Semi-Automated Python Proof Assistant philipzucker.com

The author developed Knuckledragger, a Z3-based semi-automated Python verification assistant. The article introduces its basic design principles, core principles, theory and applications, main supported features, and more.

06 Optimizing PyTorch Docker images: how to cut size by 60% mveg.es

When training neural networks on clusters, Docker containers are commonly used to package all dependencies. Generic configuration images can reach 7.6GB in size. The article introduces several optimization configurations that can save 60% of space.

07 Approximating sum types in Python with Pydantic blog.yossarian.net

Sum Types are a concept in type theory that combines different types into a single type. Python doesn't support this type natively. The article introduces using Pydantic's unions and TypeAdapter to implement a similar type and complete JSON serialization and deserialization.

08 PEP 750 – Tag Strings For Writing Domain-Specific Languages peps.python.org

This is a newly proposed PEP (in draft status) that can be seen as an extension of f-strings, similar to JavaScript's tagged template literals. It uses custom functions instead of the "f" prefix to achieve richer functionality, such as security checks, lazy evaluation, web templates, etc.

09 7 Things I've learned building a modern TUI Framework textualize.io

The textual framework has become quite popular, with nearly 25K GitHub stars. This article was written by the framework's author in 2022, sharing 7 insights: terminals are fast, DictViews are awesome, lru_cache is fast, and more.

10 Debugging running Python scripts with PDB via GDB disconnect3d.pl

How do you debug running Python scripts on Linux? The article introduces using the system's native debugger GDB's attach command, along with Python's built-in debugger pdb to debug Python scripts.

11 Efficiently Manage Memory Usage in Pandas with Large Datasets geekpython.in

Copy-on-Write is an optimization technique that improves memory usage efficiency. Pandas 2.0 already supports it, but hasn't fully implemented it. The article introduces how to enable CoW, what benefits it brings to Pandas, and the differences between enabling and not enabling CoW.

12 350M Tokens Don't Lie: Love And Hate In Hacker News outerbounds.com

Analyzed all Hacker News posts from January 2020 to June 2023 with at least 5 comments, using LLama3 70B to perform statistics on this content from different dimensions. The framework used is Netflix's open-source metaflow. The article introduces the workflows and prompts used.

Python ranks quite high on HN's popularity charts
🐿️

Projects & Resources

× 12
01 emval: a blazingly fast python email validator written in rust github.com

A high-performance email validation library developed in Rust, 100-1000 times faster than traditional tools (like python-email-validator, verify-email, and pyIsEmail).

02 fastcore: Python supercharged for the fastai library github.com

Adds excellent features from other languages to Python, such as Julia's multiple dispatch, Ruby's mixins, and Haskell's currying and binding; also adds many useful functionalities.

03 scrapy-playwright: Playwright integration for Scrapy github.com

Integrates the advantages of Scrapy and Playwright, follows Scrapy's regular workflow, and handles JavaScript-related content well. Supports coroutines and asyncio.

04 deltadb: a lightweight, fast, and scalable database built on polars and deltalake github.com

A lightweight, scalable database developed in Python that provides basic database operations and version control, with high-performance support from polars and deltalake.

05 AgentK: An autoagentic AGI that is self-evolving and modular github.com

An AGI composed of multiple collaborating agents with self-evolution capabilities, able to build new agents as needed to complete specific tasks.

06 llm_aided_ocr: Enhance Tesseract OCR output for scanned PDFs by applying LLM corrections github.com

Utilizes natural language and LLMs to improve OCR accuracy for scanned PDF files, converting results into well-formatted and readable documents. (star 1.2K)

07 translation-agent: Agentic translation using reflection workflow github.com

Andrew Ng's open-source workflow for machine translation. The basic process involves having an LLM translate first, then having it propose modifications, and finally making changes based on the suggestions. (star 4.4K)

08 PY4E - Python for Everybody py4e.com

This is an online website for learning Python, containing a series of courses, videos, PPT materials, and reference materials. It's also published on platforms like Coursera and edX, where you can earn certificates by taking courses.

09 unstract: No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents github.com

Utilizes LLMs to launch APIs that convert complex documents into structured JSON, or launch ETL pipelines that read complex files from cloud file/object storage systems and write structured data to data warehouses and databases.

10 tokencost: Easy token price estimates for 400+ LLMs. TokenOps github.com

This library provides token cost calculation capabilities for 400+ mainstream LLMs, tracks the latest price changes, and effectively manages costs for LLM integration. (star 1.3K)

11 PyOptInterface: Efficient modeling interface for mathematical optimization in Python github.com

Mathematical optimization modeling with performance exceeding Pyomo and JuMP.jl, suitable for modeling and solving large-scale mathematical optimization problems (linear programming, quadratic programming, mixed-integer linear programming, etc.).

12 metasequoia-sql: Performance-focused SQL syntax parser and analyzer github.com

Suitable for SQL formatting, execution, and analysis scenarios, committed to building the highest-performance Python SQL parser, including lexical tree parsing, syntax tree parsing, and syntax tree analysis functions.

ESC
↑↓ select · ↵ open175 issues