Conteúdo do Curso

Web Scraping with Python (res)

1. HTML Files and DevTools

Intro to Web Scraping First Steps HTML File Structure Hierarchy in the HTML String Methods Attributes DevTools

2. Beautiful Soup

Work with the Library Work with Soup Find/Find_all Find All!

3. CSS Selectors/XPaths

Intro to XPath Attributes in XPath Selectors CSS Selectors CSS Selectors in BeautifulSoup XPath vs CSS Selector

4. Tables

Intro to Tables Ways to Scrape Table Simple Solution for Scraping Challenge

Work with the Library

Regular expressions are great for matches but a bit inconvenient. Python provides us with an instrumental library for web scrapping - BeautifulSoup!

BeautifulSoup makes it easy to go through HTML files and extract the parts we are interested in. To import the library, use:

To create the first object and start iterating with the website, use the following code:

We assign the Beautiful Object to the variable soup with two parameters. The first one is the HTML file we want to parse. The second argument tells which parser to use. "html.parser" corresponds to Python's built-in HTML parser.

BeautifulSoup is highly comfortable to work with since you don't need to write regexes or additional conditions to extract the data from tags.

For instance, let's get the first tag of the type title from the website:


              1
            
print(soup.title)

BeautifulSoup can also help to convert websites into DataFrames (using pandas), which are easier to manipulate. We will learn how to do this in the following chapters.

Tarefa

Print the first h1 of the page tag using BeautifulSoup:

Import the needed library.
Create the BeautifulSoup object and assign it to the variable soup.
Print the first h1 tag using the variable soup.

Tarefa

Print the first h1 of the page tag using BeautifulSoup:

Import the needed library.
Create the BeautifulSoup object and assign it to the variable soup.
Print the first h1 tag using the variable soup.

Mude para o desktop para praticar no mundo realContinue de onde você está usando uma das opções abaixo

Tudo estava claro?

Seção 2. Capítulo 1

Work with the Library

Regular expressions are great for matches but a bit inconvenient. Python provides us with an instrumental library for web scrapping - BeautifulSoup!

BeautifulSoup makes it easy to go through HTML files and extract the parts we are interested in. To import the library, use:

To create the first object and start iterating with the website, use the following code:

BeautifulSoup is highly comfortable to work with since you don't need to write regexes or additional conditions to extract the data from tags.

For instance, let's get the first tag of the type title from the website:


              1
            
print(soup.title)

BeautifulSoup can also help to convert websites into DataFrames (using pandas), which are easier to manipulate. We will learn how to do this in the following chapters.

Tarefa

Print the first h1 of the page tag using BeautifulSoup:

Import the needed library.
Create the BeautifulSoup object and assign it to the variable soup.
Print the first h1 tag using the variable soup.

Tarefa

Print the first h1 of the page tag using BeautifulSoup:

Import the needed library.
Create the BeautifulSoup object and assign it to the variable soup.
Print the first h1 tag using the variable soup.

Mude para o desktop para praticar no mundo realContinue de onde você está usando uma das opções abaixo

Tudo estava claro?

Seção 2. Capítulo 1

Work with the Library

Regular expressions are great for matches but a bit inconvenient. Python provides us with an instrumental library for web scrapping - BeautifulSoup!

BeautifulSoup makes it easy to go through HTML files and extract the parts we are interested in. To import the library, use:

To create the first object and start iterating with the website, use the following code:

BeautifulSoup is highly comfortable to work with since you don't need to write regexes or additional conditions to extract the data from tags.

For instance, let's get the first tag of the type title from the website:


              1
            
print(soup.title)

BeautifulSoup can also help to convert websites into DataFrames (using pandas), which are easier to manipulate. We will learn how to do this in the following chapters.

Tarefa

Print the first h1 of the page tag using BeautifulSoup:

Import the needed library.
Create the BeautifulSoup object and assign it to the variable soup.
Print the first h1 tag using the variable soup.

Tarefa

Print the first h1 of the page tag using BeautifulSoup:

Import the needed library.
Create the BeautifulSoup object and assign it to the variable soup.
Print the first h1 tag using the variable soup.

Mude para o desktop para praticar no mundo realContinue de onde você está usando uma das opções abaixo

Tudo estava claro?

Regular expressions are great for matches but a bit inconvenient. Python provides us with an instrumental library for web scrapping - BeautifulSoup!

BeautifulSoup makes it easy to go through HTML files and extract the parts we are interested in. To import the library, use:

To create the first object and start iterating with the website, use the following code:

BeautifulSoup is highly comfortable to work with since you don't need to write regexes or additional conditions to extract the data from tags.

For instance, let's get the first tag of the type title from the website:


              1
            
print(soup.title)

BeautifulSoup can also help to convert websites into DataFrames (using pandas), which are easier to manipulate. We will learn how to do this in the following chapters.

Tarefa

Print the first h1 of the page tag using BeautifulSoup:

Import the needed library.
Create the BeautifulSoup object and assign it to the variable soup.
Print the first h1 tag using the variable soup.

Mude para o desktop para praticar no mundo realContinue de onde você está usando uma das opções abaixo

Seção 2. Capítulo 1

Mude para o desktop para praticar no mundo realContinue de onde você está usando uma das opções abaixo