BeautifulSoup

What it does

BeautifulSoup reads HTML and lets you find and extract specific parts of it — headlines, links, prices, whatever you need. It's the simplest way to scrape a basic website.

When to use it

Use this for simple scraping of static HTML pages — news sites, documentation, product listings.

Real example

You want to extract all article headlines and URLs from a news site. Prompt: 'Use BeautifulSoup to fetch https://news-site.com with requests, parse the HTML, find all <article> elements, and extract the h2 text and href from each. Print as a list of dicts.'

Good to know

Only works on static HTML. If the page loads content with JavaScript, you need Playwright instead.

Alternatives

Install

Use cases

Language

Category

Scraping & automation