SDKs overview
scrapestack offers client libraries designed to facilitate interaction with its web scraping API. These SDKs abstract the complexities of HTTP requests, proxy rotation, and JavaScript rendering options, allowing developers to integrate scraping functionalities into their applications with fewer lines of code. The API itself is designed around a single endpoint, where parameters control features such as target URL, proxy location, and rendering options for dynamic content scrapestack API documentation. While scrapestack provides official SDKs for several languages, the API's straightforward RESTful design means it can also be consumed directly using any HTTP client.
The primary benefit of using an SDK is the pre-built functionality for constructing requests, handling API keys, and sometimes parsing responses, which reduces boilerplate code. For instance, an SDK might automatically append the API key to requests or correctly format parameters for JavaScript rendering. Developers can choose to use an official SDK or integrate directly with the API using general-purpose HTTP libraries available in their preferred language, such as requests in Python or axios in Node.js.
Official SDKs by language
scrapestack provides official client libraries for several programming languages, aimed at simplifying the integration process. These SDKs are maintained by scrapestack and offer a structured way to interact with the API, ensuring compatibility with the latest API features and best practices. The official documentation includes code examples for common use cases across these languages, demonstrating how to make requests and interpret responses scrapestack code examples.
| Language | Package/Module | Install Command | Maturity |
|---|---|---|---|
| Python | scrapestack-python |
pip install scrapestack-python |
Stable |
| PHP | scrapestack-php |
composer require apilayer/scrapestack-php |
Stable |
| Node.js | scrapestack-node |
npm install scrapestack-node |
Stable |
Installation
Installing scrapestack SDKs typically involves using the package manager specific to the programming language. These commands fetch the library from its respective repository and make it available for use within your project. Before installation, ensure you have the correct package manager configured for your development environment. For example, Python projects generally use pip, PHP projects use Composer, and Node.js projects use npm or yarn.
Python
To install the Python SDK, use pip, Python's package installer. It is recommended to use a virtual environment to manage project dependencies.
pip install scrapestack-python
PHP
For PHP projects, the SDK is installed via Composer, the dependency manager for PHP. Ensure Composer is installed globally on your system.
composer require apilayer/scrapestack-php
Node.js
Node.js developers can install the SDK using npm (Node Package Manager) or yarn.
npm install scrapestack-node
# or
yarn add scrapestack-node
Quickstart example
The following examples demonstrate how to make a basic web scraping request using the official SDKs. These snippets assume you have already obtained an API key from your scrapestack dashboard scrapestack API key management and installed the relevant SDK. The core functionality involves constructing a request with the target URL and your API key, then processing the API's JSON response.
Python Quickstart
This Python example fetches the HTML content of a specified URL using the scrapestack-python SDK.
import scrapestack
api_key = "YOUR_API_KEY"
target_url = "http://quotes.toscrape.com/"
# Initialize the client
client = scrapestack.Client(api_key)
# Make a request
response = client.scrape(url=target_url)
# Print the HTML content
if response.success:
print(response.content)
else:
print(f"Error: {response.error_code} - {response.error_message}")
PHP Quickstart
This PHP example uses the scrapestack-php SDK to retrieve web page content.
<?php
require_once 'vendor/autoload.php';
use Apilayer\Scrapestack\ScrapestackClient;
$apiKey = 'YOUR_API_KEY';
$targetUrl = 'http://quotes.toscrape.com/';
$client = new ScrapestackClient($apiKey);
try {
$response = $client->scrape($targetUrl);
echo $response->getContent();
} catch (\Exception $e) {
echo 'Error: ' . $e->getMessage();
}
?>
Node.js Quickstart
The Node.js quickstart demonstrates fetching content with the scrapestack-node SDK.
const scrapestack = require('scrapestack-node');
const apiKey = 'YOUR_API_KEY';
const targetUrl = 'http://quotes.toscrape.com/';
const client = new scrapestack.Client(apiKey);
client.scrape(targetUrl)
.then(response => {
if (response.success) {
console.log(response.content);
} else {
console.log(`Error: ${response.error.code} - ${response.error.message}`);
}
})
.catch(error => {
console.error('Request failed:', error);
});
Community libraries
Beyond the official SDKs, developers can integrate with the scrapestack API using general-purpose HTTP client libraries available in virtually any programming language. This approach offers flexibility, particularly for languages where an official SDK might not be available or for projects requiring custom request handling. The scrapestack API is a RESTful service, meaning it communicates over standard HTTP methods and typically returns JSON responses HTTP/1.1 Semantics and Content RFC. This design allows for direct integration using libraries like:
- Python: The
requestslibrary is a popular choice for making HTTP requests in Python. It simplifies sending requests and handling responses, making it suitable for interacting with scrapestack directly Python Requests documentation. - JavaScript/Node.js:
axiosor the built-infetchAPI (in modern Node.js environments and browsers) can be used to send HTTP requests to the scrapestack endpoint. These libraries provide promise-based interfaces for asynchronous operations. - Go: Go's standard library
net/httppackage is robust enough to handle API interactions, allowing developers to construct requests and parse JSON responses directly. - Ruby: The
Net::HTTPlibrary, part of Ruby's standard library, or gems likehttparty, can be used for making web requests. - Java: Libraries such as Apache HttpClient or OkHttp provide comprehensive features for HTTP communication in Java applications.
- cURL: For command-line testing or scripting,
cURLcan be used to send requests directly to the scrapestack API endpoint. This method is often used to quickly verify API responses or for simple shell scripts.
When using community or general-purpose HTTP libraries, developers are responsible for correctly formatting the API URL, including parameters for the target URL, API key, proxy options, and JavaScript rendering. They must also handle parsing the JSON response and error conditions according to the API documentation.