Notice: This page requires JavaScript to function properly.
Please enable JavaScript in your browser settings or update your browser.Work with the Library | Beautiful Soup
Web Scraping with Python (res)
course content

Course Content

Web Scraping with Python (res)

Web Scraping with Python (res)

1. HTML Files and DevTools
Intro to Web ScrapingFirst StepsHTML File StructureHierarchy in the HTMLString MethodsAttributesDevTools
2. Beautiful Soup
Work with the LibraryWork with SoupFind/Find_allFind All!
3. CSS Selectors/XPaths
Intro to XPathAttributes in XPathSelectorsCSS SelectorsCSS Selectors in BeautifulSoupXPath vs CSS Selector
4. Tables
Intro to TablesWays to Scrape TableSimple Solution for ScrapingChallenge

bookWork with the Library

Regular expressions are great for matches but a bit inconvenient. Python provides us with an instrumental library for web scrapping - BeautifulSoup!

BeautifulSoup makes it easy to go through HTML files and extract the parts we are interested in. To import the library, use:

To create the first object and start iterating with the website, use the following code:

We assign the Beautiful Object to the variable soup with two parameters. The first one is the HTML file we want to parse. The second argument tells which parser to use. "html.parser" corresponds to Python's built-in HTML parser.

BeautifulSoup is highly comfortable to work with since you don't need to write regexes or additional conditions to extract the data from tags.

For instance, let's get the first tag of the type title from the website:


              
1

            
print(soup.title)
copy

BeautifulSoup can also help to convert websites into DataFrames (using pandas), which are easier to manipulate. We will learn how to do this in the following chapters.

Task

Print the first h1 of the page tag using BeautifulSoup:

  1. Import the needed library.
  2. Create the BeautifulSoup object and assign it to the variable soup.
  3. Print the first h1 tag using the variable soup.

Switch to desktopSwitch to desktop for real-world practiceContinue from where you are using one of the options below
Everything was clear?

How can we improve it?

Thanks for your feedback!

Section 2. Chapter 1
toggle bottom row

bookWork with the Library

Regular expressions are great for matches but a bit inconvenient. Python provides us with an instrumental library for web scrapping - BeautifulSoup!

BeautifulSoup makes it easy to go through HTML files and extract the parts we are interested in. To import the library, use:

To create the first object and start iterating with the website, use the following code:

We assign the Beautiful Object to the variable soup with two parameters. The first one is the HTML file we want to parse. The second argument tells which parser to use. "html.parser" corresponds to Python's built-in HTML parser.

BeautifulSoup is highly comfortable to work with since you don't need to write regexes or additional conditions to extract the data from tags.

For instance, let's get the first tag of the type title from the website:


              
1

            
print(soup.title)
copy

BeautifulSoup can also help to convert websites into DataFrames (using pandas), which are easier to manipulate. We will learn how to do this in the following chapters.

Task

Print the first h1 of the page tag using BeautifulSoup:

  1. Import the needed library.
  2. Create the BeautifulSoup object and assign it to the variable soup.
  3. Print the first h1 tag using the variable soup.

Switch to desktopSwitch to desktop for real-world practiceContinue from where you are using one of the options below
Everything was clear?

How can we improve it?

Thanks for your feedback!

Section 2. Chapter 1
toggle bottom row

bookWork with the Library

Regular expressions are great for matches but a bit inconvenient. Python provides us with an instrumental library for web scrapping - BeautifulSoup!

BeautifulSoup makes it easy to go through HTML files and extract the parts we are interested in. To import the library, use:

To create the first object and start iterating with the website, use the following code:

We assign the Beautiful Object to the variable soup with two parameters. The first one is the HTML file we want to parse. The second argument tells which parser to use. "html.parser" corresponds to Python's built-in HTML parser.

BeautifulSoup is highly comfortable to work with since you don't need to write regexes or additional conditions to extract the data from tags.

For instance, let's get the first tag of the type title from the website:


              
1

            
print(soup.title)
copy

BeautifulSoup can also help to convert websites into DataFrames (using pandas), which are easier to manipulate. We will learn how to do this in the following chapters.

Task

Print the first h1 of the page tag using BeautifulSoup:

  1. Import the needed library.
  2. Create the BeautifulSoup object and assign it to the variable soup.
  3. Print the first h1 tag using the variable soup.

Switch to desktopSwitch to desktop for real-world practiceContinue from where you are using one of the options below
Everything was clear?

How can we improve it?

Thanks for your feedback!

Regular expressions are great for matches but a bit inconvenient. Python provides us with an instrumental library for web scrapping - BeautifulSoup!

BeautifulSoup makes it easy to go through HTML files and extract the parts we are interested in. To import the library, use:

To create the first object and start iterating with the website, use the following code:

We assign the Beautiful Object to the variable soup with two parameters. The first one is the HTML file we want to parse. The second argument tells which parser to use. "html.parser" corresponds to Python's built-in HTML parser.

BeautifulSoup is highly comfortable to work with since you don't need to write regexes or additional conditions to extract the data from tags.

For instance, let's get the first tag of the type title from the website:


              
1

            
print(soup.title)
copy

BeautifulSoup can also help to convert websites into DataFrames (using pandas), which are easier to manipulate. We will learn how to do this in the following chapters.

Task

Print the first h1 of the page tag using BeautifulSoup:

  1. Import the needed library.
  2. Create the BeautifulSoup object and assign it to the variable soup.
  3. Print the first h1 tag using the variable soup.

Switch to desktopSwitch to desktop for real-world practiceContinue from where you are using one of the options below
Section 2. Chapter 1
Switch to desktopSwitch to desktop for real-world practiceContinue from where you are using one of the options below
some-alt