Web Scraping API - ApyHub

Web Scraping API

Data Extraction#extract#text
SharpAPI
SharpAPI
verified icon
1000 atoms
Base tier

About

The Web Scraping API is designed to extract structured data from any publicly accessible URL, providing insights into page content, metadata, and structure in a machine-readable JSON format. This API is ideal for applications requiring automated data extraction for SEO, competitive analysis, content aggregation, or market research.
With a simple API call, you can retrieve key information such as:
  • Page title and metadata (description, keywords, author, Open Graph & Twitter card details)
  • Headers and meta tags (content type, viewport, canonical URL, charset)
  • Structured content extraction (headings, paragraphs, key text elements)
  • Internal and external links for site structure analysis
  • Language detection for localization and content categorization
  • Timestamped results for accurate tracking and logging

API Playground

API Documentation

web-scraping
GET
https://api.apyhub.com/sharpapi/api/v1/utilities/scrape_url/

Request example

1
```cURL
2


3
curl --location --request GET 'https://api.apyhub.com/sharpapi/api/v1/utilities/scrape_url/?url=https://apyhub.com \
4
 --header 'apy-token: {{token}}'
Method: GET
Content Type: application/json
Query Parameter(s)
AttributeTypeMandatoryDescription
urlStringYesThe URL to scrape
Sample Response
1
{
2
    "url": "https://apyhub.com/",
3
    "timestamp": "2025-01-15T08:56:04.946195Z",
4
    "scraped_data": {
5
        "title": "AI-powered Workflow Automation API",
6
        "detected_language": "en",
7
        "headers": {
8
            "charset": "utf-8",
9
            "contentType": null,
10
            "viewport": [
11
                "width=device-width",
12
                "initial-scale=1"
13
            ],
14
            "canonical": "https://apyhub.com/",
15
            "csrfToken": "xxx"
16
        },
17
        "meta_tags": {
18
            "author": null,
19
            "image": null,
20
            "keywords": [
21
                "ApyHub",
22
                "AI",
23
                "automation"
24
            ],
25
            "description": "Leverage AI API to streamline workflow in E-Commerce, Marketing, Content Management, HR Tech, Travel, and more."
26
        },
27
        "open_graph": {
28
            "og:title": "AI-powered Workflow Automation API",
29
            "og:type": "website",
30
            "og:URL": "https://apyhub.com",
31
            "og:image": "https://apyhub.com/build/assets/apyhub-website-preview-ARuIroBi.png",
32
            "og:description": "Leverage AI API to streamline workflow in E-Commerce, Marketing, Content Management, HR Tech, Travel, and more. ",
33
            "og:url": "https://apyhub.com"
34
        },
35
        "twitter_card": {
36
            "twitter:card": "summary",
37
            "twitter:site": "@apyhub",
38
            "twitter:creator": "@a2zwebltd"
39
        },
40
        "content_structured": [
41
            {
42
                "tag": "h1",
43
                "content": "Automate workflows with AI-powered API"
44
            },
45
            {
46
                "tag": "h2",
47
                "content": "Leverage AI API for automation in E-Commerce, Marketing,\n Content Management, HR Tech, Travel, and more. [...]"
48
            },
49
           
50
        ],
51
        "content_lists": [
52
            {
53
                "type": "ul",
54
                "items": [
55
                    "E-commerce:",
56
                    "Quickly generate engaging [...]"
57
                ]
58
            },
59
            {
60
                "type": "ul",
61
                "items": [
62
                    "HR Tech:",
63
                    "Generate",
64
                    "complex job descriptions [...]"
65
                ]
66
            }
67
        ],
68
        "content_html": "<h1>Automate workflows with AI-powered API</h1>\n<h2>Leverage AI API for automation in E-Commerce, Marketing,<br />\n Content Management, HR Tech, Travel, and more.</h2> [...]",
69
        "content_markdown": "# Automate workflows with AI-powered API\n\n## Leverage AI API for automation in E-Commerce, Marketing,  \n Content Management, HR Tech, Travel, and more. [...]",
70
        "content_keywords_index": [
71
            "streamline workflow",
72
            "e-commerce",
73
            "marketing",
74
            "content management",
75
            "hr tech",
76
            "travel [...]"
77
        ],
78
        "links": {
79
            "internal": [
80
                "https://apyhub.com/register",
81
                "https://apyhub.com/en/blog/category/workflow-optimization",
82
                "https://apyhub.com/documentation",
83
                "https://apyhub.com/en/ai-automation-cheatsheet",
84
                "https://apyhub.com/policy",
85
                "https://apyhub.com/terms"
86
            ],
87
            "external": [
88
                "https://github.com/sharpapi/",
89
                "https://github.com/sharpapi/sharpapi-laravel-client",
90
                "https://www.nuget.org/packages/SharpApi.Service",
91
                "https://twitter.com/SharpAPI",
92
                "https://www.youtube.com/@SharpAPI"
93
            ]
94
        }
95
    }
96
}

HTTP Response Codes

The method may return one of the following HTTP status codes:
Status CodeDescription
200The request was successful
401Required authentication information is either missing or not valid for the resource.
400Invalid input - if the file is invalid or corrupted
500If any unexpected error occurs while submitting the request.

Authentication

All API requests to ApyHub services need to be authenticated. Currently we support tokens or basic authentication mechanisms. You can generate and view your existing credentials from your workspace settings (on the left side of the navbar) and go to “API Keys".
Points to note:
  • Credential secrets are generated on the fly and are not stored in plain text, so on generating a credential please save the secrets somewhere safe.
  • Use the apy-token as the header parameter to pass the token.
  • Use the Authorization header to send the basic authentication credentials.

Error codes

1
{
2
  "error": {
3
    "code": 105,
4
    "message": "Invalid URL"
5
  }
6
}
To search for a specific error code, enter the code in the search box below. Alternatively, you can click on the button to view a complete list of all error codes.
Table of contents
AboutAPI PlaygroundAPI DocumentationAuthenticationError codesRelated Utility APIsRelated Articles