Web Scraping with Python (res)

Web Scraping with Python (res)

0%

1. HTML Files and DevTools

Here we will explore the structure of the HTML file, how to load it and work with the data in the file.

Intro to Web Scraping

HTML File Structure

Hierarchy in the HTML

2. Beautiful Soup

Learn how to extract the data more comfortably with the BeautifulSoup library.

Work with the Library

Find All!

3. CSS Selectors/XPaths

Explore how XPath and CSS Selectors can be used to locate web elements without id, class, or name.

Attributes in XPath

CSS Selectors in BeautifulSoup

XPath vs CSS Selector

4. Tables

Here we will learn how to convert tables into the DataFrame for the following analysis.

Intro to Tables

Ways to Scrape Table

Simple Solution for Scraping

Conteúdo do Curso

Web Scraping with Python (res)

Web Scraping with Python (res)

1. HTML Files and DevTools

Intro to Web Scraping First Steps HTML File Structure Hierarchy in the HTML String Methods Attributes DevTools

2. Beautiful Soup

Work with the Library Work with Soup Find/Find_all Find All!

3. CSS Selectors/XPaths

Intro to XPath Attributes in XPath Selectors CSS Selectors CSS Selectors in BeautifulSoup XPath vs CSS Selector

4. Tables

Intro to Tables Ways to Scrape Table Simple Solution for Scraping Challenge

Find All!

We can also use the function .find_all() to find not only one but multiple tags using a list. For instance:


              1
            
print(soup.find_all(["title","h1"]))

The code above returns the list of all title and h1 tags in our HTML file.

We can also use functions as an argument to find:


              123
            
def find_all_condition(tag):
&nbsp;&nbsp;&nbsp;&nbsp;return tag.string == "City: New York"
print(soup.find_all(find_all_condition))

The code returns all tags which contain only the text "City: New York".

If you feel uncomfortable with functions in Python, repeat them with the course.

The BeautifulSoup library can optimize work and save a lot of time extracting the data from web pages. However, sometimes even such a magic tool can't detect the needed information. In such cases, parse the needed information with the function .find() and regular expressions!

Tudo estava claro?

Obrigado pelo seu feedback!

Seção 2. Capítulo 4

some-alt