web scraping node js puppeteer

/*
    This code comes from Vincent Lab
    And it has a video version linked here: https://www.youtube.com/watch?v=GAvpZuVzeA8
*/

// AQI	Air Pollution Level	Health Implications	Cautionary Statement (for PM2.5)
// 0 - 50	Good	Air quality is considered satisfactory, and air pollution poses little or no risk	None
// 51 -100	Moderate	Air quality is acceptable; however, for some pollutants there may be a moderate health concern for a very small number of people who are unusually sensitive to air pollution.	Active children and adults, and people with respiratory disease, such as asthma, should limit prolonged outdoor exertion.
// 101-150	Unhealthy for Sensitive Groups	Members of sensitive groups may experience health effects. The general public is not likely to be affected.	Active children and adults, and people with respiratory disease, such as asthma, should limit prolonged outdoor exertion.
// 151-200	Unhealthy	Everyone may begin to experience health effects; members of sensitive groups may experience more serious health effects	Active children and adults, and people with respiratory disease, such as asthma, should avoid prolonged outdoor exertion; everyone else, especially children, should limit prolonged outdoor exertion
// 201-300	Very Unhealthy	Health warnings of emergency conditions. The entire population is more likely to be affected.	Active children and adults, and people with respiratory disease, such as asthma, should avoid all outdoor exertion; everyone else, especially children, should limit outdoor exertion.
// 300+	Hazardous	Health alert: everyone may experience more serious health effects	Everyone should avoid all outdoor exertion

// Import dependencies
const puppeteer = require("puppeteer");
const moment = require("moment");
const fs = require("fs");

(async () => {

    // The location / URL
    const url = "https://aqicn.org/city/denmark/copenhagen/h.c.andersens-boulevard/";

    // Create the browser
    const browser = await puppeteer.launch({
        headless: true
    });

    // Navigate to the website
    const page = await browser.newPage();
    await page.goto(url, { waitUntil: "load" });

    // Get the Air quality index
    const aqi = await page.$(`#aqiwgtvalue`);

    // Extract the index
    const index = await (await aqi.getProperty("textContent")).jsonValue();

    // Extract the title
    const title = await (await aqi.getProperty("title")).jsonValue();

    // Extract the location
    const location = await (await (await page.$(`#aqiwgttitle1`)).getProperty("textContent")).jsonValue();

    // Write the data to a CSV file
    if (fs.existsSync("air-pollution.csv")) {
        fs.appendFileSync("air-pollution.csv", `\n${moment.utc()}, ${index}, ${title}, ${location.replace(/,/g, '')}`)
    } else {
        fs.writeFileSync("air-pollution.csv", `datetime, index, title, location\n${moment.utc()}, ${index}, ${title}, ${location.replace(/,/g, '')}`)
    }

    // Close the browser
    await browser.close();

4.11
9

                                    const puppeteer = require('puppeteer')
const $ = require('cheerio')

(async () => {
	const browser = await puppeteer.launch()
	const page = await browser.newPage()
	await page.goto('https://www.imdb.com/title/tt7126948/?ref_=nv_sr_srsg_0', { waitUntil: 'networkidle0' })

	const content = await page.evaluate(() => document.body.innerHTML)
	const data = $('div[class="title_wrapper"]', content).find('h1').text()

	console.log(data)

	await browser.close()
})()

4.11 (9 Votes)
0
Are there any code examples left?
Create a Free Account
Unlock the power of data and AI by diving into Python, ChatGPT, SQL, Power BI, and beyond.
Sign up
Develop soft skills on BrainApps
Complete the IQ Test
Relative searches
using puppeter web scraping with javascript scraping con node puppeteer web scraping nodejs puppeteer puppeteer js web scraping example web scraping javascript using puppeteer nodejs scrapper pupeeteer get list with puppeteer node js web scraping node js web scrape with puppeteer framework node js web scrape with puppeteer node js puppeteer scraping puppeteer js web scraping scrapping page content with nodejs + puppeteer node puppeteer scrap web scraping with puppeteer node js javascript web scraping puppeteer puppeteer and expressJS web scraper puppeteer and express web scraper build a web scraping with puppeteer node js nodejs puppeteer scraping web scraping with node js and puppeteer Web Scraping with Node.js using Puppeteer node js puppeteer how to webscrape all products and get hregf puppeteer api for automated web scraping using react js puppeteer scraper web scraping puppetter tutorial web crwaler puppeteer network web scrapping puppeter scrape data using puppeter node js puppeteer webscraping save website as json puppiteer puppeteer scraping example scraping web data with puppeteer node js scrape website data with puppeteer puppeteer web scraping example node puppeteer web scrape node js puppeteer web scraping how to make website from website-scraper-puppeteer display exactly the same as the actual website web scraping with Puppeteer webscraping node js puppeteer how to get certain values how to get data from web service in in puppeteer puppeteer production flow with nodejs puppeteer call database scraping with puppeteer how to make an web scrapper with puppeteer web scraping puppeteer puppeteer get data puppeteer scrape web browser automation web scraper node js puppeteer web scraping pupperer example crape web nodejs puppeteer scrape js web scraping node js puppeteer
Made with love
This website uses cookies to make IQCode work for you. By using this site, you agree to our cookie policy

Welcome Back!

Sign up to unlock all of IQCode features:
  • Test your skills and track progress
  • Engage in comprehensive interactive courses
  • Commit to daily skill-enhancing challenges
  • Solve practical, real-world issues
  • Share your insights and learnings
Create an account
Sign in
Recover lost password
Or log in with

Create a Free Account

Sign up to unlock all of IQCode features:
  • Test your skills and track progress
  • Engage in comprehensive interactive courses
  • Commit to daily skill-enhancing challenges
  • Solve practical, real-world issues
  • Share your insights and learnings
Create an account
Sign up
Or sign up with
By signing up, you agree to the Terms and Conditions and Privacy Policy. You also agree to receive product-related marketing emails from IQCode, which you can unsubscribe from at any time.
Creating a new code example
Code snippet title
Source