ScrapeGraph AI - langchain中文

ScrapeGraph AI is a service that provides AI-powered web scraping capabilities. It offers tools for extracting structured data, converting webpages to markdown, and processing local HTML content using natural language prompts.

Installation and Setup

Install the required packages:

pip install langchain-scrapegraph

uv add langchain-scrapegraph

Set up your API key:

export SGAI_API_KEY="your-scrapegraph-api-key"

Tools

See a usage example. There are four tools available:

from langchain_scrapegraph.tools import (
    SmartScraperTool,    # Extract structured data from websites
    SmartCrawlerTool,    # Extract data from multiple pages with crawling
    MarkdownifyTool,     # Convert webpages to markdown
    AgenticScraperTool,  # Extract specifying steps
    GetCreditsTool,      # Check remaining API credits
)

Each tool serves a specific purpose:

SmartScraperTool: Extract structured data from websites given a URL, prompt and optional output schema
SmartCrawlerTool: Extract data from multiple pages with advanced crawling options like depth control, page limits, and domain restrictions
MarkdownifyTool: Convert any webpage to clean markdown format
AgenticScraperTool: Extract specifying steps
GetCreditsTool: Check your remaining ScrapeGraph AI credits