As we all know, CSV files use commas to separate content, but this doesn't work well for situations involving commas, separators, and line breaks. The article suggests using ASCII's universal separators as an alternative, which can handle complex data and avoid formatting errors.
TL;DR
Why CSVs fall short and DSVs work better, PEP-750's tag strings for writing DSLs, and 350 million tokens parsed to reveal what Hacker News loves and hates.
Articles & Tutorials
× 12Introduces how to use the scrapy-playwright plugin to scrape content from infinite scroll pages, including configuration, handling CORS issues, principles of scroll scraping, and detailed implementation code.
Introduces how to calculate test coverage for your codebase using pytest-cov, and use tools like nox for combination testing across different versions of Python and libraries. Testing both the code itself and its integration with the entire Python ecosystem.
The author of Fastcore introduces the main features of this library, which is primarily used to extend Python's capabilities, including: making **kwargs transparent, avoiding boilerplate code when setting instance attributes, avoiding subclass boilerplate, type dispatch, better functools.partial, more useful __repr__, and more.
The author developed Knuckledragger, a Z3-based semi-automated Python verification assistant. The article introduces its basic design principles, core principles, theory and applications, main supported features, and more.
When training neural networks on clusters, Docker containers are commonly used to package all dependencies. Generic configuration images can reach 7.6GB in size. The article introduces several optimization configurations that can save 60% of space.
Sum Types are a concept in type theory that combines different types into a single type. Python doesn't support this type natively. The article introduces using Pydantic's unions and TypeAdapter to implement a similar type and complete JSON serialization and deserialization.
This is a newly proposed PEP (in draft status) that can be seen as an extension of f-strings, similar to JavaScript's tagged template literals. It uses custom functions instead of the "f" prefix to achieve richer functionality, such as security checks, lazy evaluation, web templates, etc.
The textual framework has become quite popular, with nearly 25K GitHub stars. This article was written by the framework's author in 2022, sharing 7 insights: terminals are fast, DictViews are awesome, lru_cache is fast, and more.
How do you debug running Python scripts on Linux? The article introduces using the system's native debugger GDB's attach command, along with Python's built-in debugger pdb to debug Python scripts.
Copy-on-Write is an optimization technique that improves memory usage efficiency. Pandas 2.0 already supports it, but hasn't fully implemented it. The article introduces how to enable CoW, what benefits it brings to Pandas, and the differences between enabling and not enabling CoW.
Analyzed all Hacker News posts from January 2020 to June 2023 with at least 5 comments, using LLama3 70B to perform statistics on this content from different dimensions. The framework used is Netflix's open-source metaflow. The article introduces the workflows and prompts used.
Projects & Resources
× 12A high-performance email validation library developed in Rust, 100-1000 times faster than traditional tools (like python-email-validator, verify-email, and pyIsEmail).
Adds excellent features from other languages to Python, such as Julia's multiple dispatch, Ruby's mixins, and Haskell's currying and binding; also adds many useful functionalities.
Integrates the advantages of Scrapy and Playwright, follows Scrapy's regular workflow, and handles JavaScript-related content well. Supports coroutines and asyncio.
A lightweight, scalable database developed in Python that provides basic database operations and version control, with high-performance support from polars and deltalake.
An AGI composed of multiple collaborating agents with self-evolution capabilities, able to build new agents as needed to complete specific tasks.
Utilizes natural language and LLMs to improve OCR accuracy for scanned PDF files, converting results into well-formatted and readable documents. (star 1.2K)
Andrew Ng's open-source workflow for machine translation. The basic process involves having an LLM translate first, then having it propose modifications, and finally making changes based on the suggestions. (star 4.4K)
This is an online website for learning Python, containing a series of courses, videos, PPT materials, and reference materials. It's also published on platforms like Coursera and edX, where you can earn certificates by taking courses.
Utilizes LLMs to launch APIs that convert complex documents into structured JSON, or launch ETL pipelines that read complex files from cloud file/object storage systems and write structured data to data warehouses and databases.
This library provides token cost calculation capabilities for 400+ mainstream LLMs, tracks the latest price changes, and effectively manages costs for LLM integration. (star 1.3K)
Mathematical optimization modeling with performance exceeding Pyomo and JuMP.jl, suitable for modeling and solving large-scale mathematical optimization problems (linear programming, quadratic programming, mixed-integer linear programming, etc.).
Suitable for SQL formatting, execution, and analysis scenarios, committed to building the highest-performance Python SQL parser, including lexical tree parsing, syntax tree parsing, and syntax tree analysis functions.