▣ DATA EXTRACTION · SEO

Web Scraping API

What it does

Web Scraper lets you send a URL and get back the page’s captured content and metadata in a structured response. It echoes the scraped url, adds a UTC timestamp, and returns scraped_data for the page.

Use it when you need to inspect a live page without building your own crawler. The response can include title, document-level headers, standard meta_tags, open_graph values, twitter_card values, content_html, content_markdown, content_structured, content_lists, content_keywords_index, detected_language, and extracted links split into internal and external arrays. That makes it useful for page monitoring, content indexing, SEO checks, competitive analysis, and lightweight data collection.

The input is simple: pass a single url query parameter. The output is built for downstream processing, so you can store the returned page text, inspect metadata, or follow discovered links in your own workflow. If you only need page title and metadata, you can ignore the heavier content fields and work with the specific parts you need.

Web Scraper is a good fit for developer tools that need reliable page retrieval and structured page analysis from one request.

GET

Scrape a URL

http://localhost:8080/sharpapi/scrape-url

QUICKSTARTGUIDE

Quickstart

Scrape a page by passing its URL as a query parameter.

curl -X GET "http://localhost:8080/sharpapi/scrape-url?url=https%3A%2F%2Fapyhub.com%2F" \
  -H "apy-token: $APY_TOKEN"

What you'll get back

Returns a JSON object with url and timestamp string fields, plus a scraped_data object containing the extracted page data.

{
  "url": "https://apyhub.com/",
  "timestamp": "2025-01-15T08:56:04.946195Z",
  "scraped_data": {}
}

TRY ITLIVE · 1000 ATOMS

API key

Loading your default key…

The full key is used to call the gateway and stays in this tab — never sent to orbit or saved.

About this endpoint

What it does

Fetches and scrapes the URL provided in the url query parameter, then returns the original URL, the scrape timestamp, and a scraped_data object containing extracted page content and metadata.

Query Parameter(s)

Attribute	Type	Mandatory	Description
url	String	Yes	The URL to scrape. Must be a valid URI.

Response

Returns a JSON object with required top-level url and timestamp string fields, plus a required scraped_data object field. url echoes the scraped URL, timestamp is the UTC time the scrape was performed, and scraped_data contains the extracted page data.

Attribute	Type	Mandatory	Description
url	String	Yes	The URL that was scraped, echoed back from the request.
timestamp	String	Yes	UTC timestamp of when the scrape was performed.
scraped_data	Object	Yes	Scraped page data, including content, metadata, and extracted links.

Query parameters

Name

Type

Description

urlREQUIRED

string

—

▣ COMMON ERRORS

Errors any endpoint can return

400bad_request

Required parameter missing or malformed body.

401unauthorized

API key missing, revoked, or not authorized for this service.

429rate_limited

Your plan's per-second rate exceeded. Retry with exponential backoff.

503upstream_busy

Backend temporarily unavailable. Try again in a few seconds.