Getting started overview

This guide provides a rapid onboarding path for Restpack, focusing on the steps necessary to move from account creation to a successful first API request. Restpack offers various APIs for web data extraction, including a Scraping API for general web content, a SERP API for search engine results, and a Screenshots API for web page captures. The core process for getting started remains consistent across these services.

The primary method of interaction with Restpack APIs is via standard HTTP requests. Developers typically send a GET or POST request to a Restpack endpoint, including their API key and specific parameters for the desired operation. The API then returns data, commonly in JSON format. This guide will use the Restpack Scraping API as the example for the first request due to its general applicability.

To summarize the immediate steps:

  1. Sign up for a Restpack account to obtain an API key.
  2. Formulate an API request incorporating your key and target URL.
  3. Execute the request using a tool like cURL or an HTTP client library in your preferred programming language.
  4. Process the JSON response containing the extracted data.

The following table provides a quick reference for these initial steps:

Step What to do Where
1. Sign Up Create a new Restpack account. Restpack Pricing page (free tier available)
2. Get API Key Locate your unique API key in the dashboard. Restpack Dashboard (after login)
3. Construct Request Build an HTTP GET request with your API key and target URL. Your code editor or terminal
4. Execute Request Send the request using cURL or an HTTP client. Terminal or application runtime
5. Process Response Parse the JSON data returned by the API. Your application logic

Create an account and get keys

Access to Restpack APIs requires an API key for authentication. This key identifies your account and tracks your usage against your plan limits. Restpack offers a free tier that includes 5,000 requests per month, which is sufficient for initial testing and development.

Account Creation

  1. Navigate to the Restpack pricing page.
  2. Select the "Get Started for Free" option or choose a paid plan if your anticipated usage exceeds the free tier.
  3. Complete the registration process by providing the requested information, typically an email address and password.
  4. Verify your email address if prompted.

Finding Your API Key

Once your account is created and you are logged in, your API key will be accessible through the Restpack dashboard:

  1. Log in to your Restpack account.
  2. On your dashboard, locate the section labeled "Your API Key" or similar. This key is a unique alphanumeric string.
  3. Copy your API key. It is crucial to keep this key confidential, as it grants access to your Restpack account and resources.

Restpack API keys are typically passed as a query parameter named api_key in your HTTP requests. For more advanced security, some APIs support OAuth 2.0 for delegated authorization, but for initial getting started with Restpack, the API key parameter is standard.

Your first request

This section guides you through making your first functional API request using Restpack's Scraping API. The example uses cURL, a command-line tool for making HTTP requests, available on most operating systems. This method demonstrates the fundamental HTTP interaction without requiring specific programming language setup.

Scraping API Endpoint

The base endpoint for the Scraping API is https://api.restpack.io/v2/web_scraping/scrape. You will append your API key and the target URL you wish to scrape as query parameters.

Example Request with cURL

Replace YOUR_API_KEY with the actual API key you obtained from your Restpack dashboard, and YOUR_TARGET_URL_ENCODED with the URL of the webpage you want to scrape, ensuring it is URL-encoded. For instance, if you want to scrape https://example.com, the URL should be https%3A%2F%2Fexample.com.

curl -X GET \
  'https://api.restpack.io/v2/web_scraping/scrape?api_key=YOUR_API_KEY&url=YOUR_TARGET_URL_ENCODED'

Breaking down the cURL command:

  • curl -X GET: Specifies an HTTP GET request.
  • 'https://api.restpack.io/v2/web_scraping/scrape?api_key=YOUR_API_KEY&url=YOUR_TARGET_URL_ENCODED': The API endpoint with two query parameters: api_key for authentication and url for the target webpage.

Response Format

A successful request to the Scraping API typically returns a JSON object containing the scraped content. The exact structure can vary based on the API and parameters used, but a common response for the Scraping API might look like this:

{
  "status": "success",
  "url": "https://example.com",
  "content": "<!DOCTYPE html>\n<html>\n<head>..." // Full HTML content of the page
}

The content field will contain the full HTML of the target URL. You can then parse this HTML using libraries appropriate for your programming language (e.g., Beautiful Soup in Python, Cheerio in Node.js).

Handling Errors

If an error occurs, the API will return a JSON object with an error status and a descriptive message. Common errors include invalid API keys, invalid URLs, or exceeding rate limits. For example:

{
  "status": "error",
  "message": "Invalid API Key"
}

Refer to the Restpack API documentation for a comprehensive list of error codes and messages.

Common next steps

After successfully making your first request, consider these common next steps to further integrate Restpack into your applications:

  • Explore other APIs: Restpack offers additional APIs like the SERP API for search engine results, Screenshots API, and Article Extraction API. Each has unique parameters and use cases. Reviewing their respective documentation can help you find the best fit for your needs.

  • Integrate into your application: Instead of cURL, use an HTTP client library in your preferred programming language. Restpack provides code examples in Python, Node.js, PHP, and Ruby to facilitate this integration.

  • Implement parsing logic: The raw HTML content returned by the Scraping API typically needs to be parsed to extract specific data points. Libraries like Beautiful Soup for Python or Cheerio for Node.js are commonly used for this purpose.

  • Manage proxies and headers: For advanced scraping scenarios, you may need to specify custom proxies, user agents, or HTTP headers. Restpack's APIs often provide parameters to configure these options to avoid detection and ensure successful data extraction. Details are available in the Proxy API documentation.

  • Monitor usage and billing: Keep track of your API request usage through the Restpack dashboard to stay within your plan limits and understand potential billing implications, especially if you are on a paid plan. The Restpack pricing page details the different tiers.

  • Error handling and retries: Implement robust error handling in your application to manage potential API errors, network issues, or rate limiting. Consider implementing retry mechanisms with exponential backoff for transient errors, a common practice in reliable API integrations.

Troubleshooting the first call

Encountering issues during your first API call is common. Here are some troubleshooting steps for Restpack:

  • Invalid API Key: Double-check that you have copied the API key correctly from your Restpack dashboard. Ensure there are no leading or trailing spaces. An "Invalid Key" error message typically indicates this problem.

  • Incorrect URL Encoding: The url parameter for the Scraping API (and similar APIs) must be URL-encoded. If your target URL contains special characters (like &, ?, /), they must be properly encoded. For example, https://example.com?param=value should be https%3A%2F%2Fexample.com%3Fparam%3Dvalue. Many programming languages have built-in functions for URL encoding.

  • Missing Parameters: Verify that all required parameters are included in your request. For the Scraping API, at minimum, api_key and url are essential. Consult the Restpack API reference for specific API requirements.

  • Rate Limits Exceeded: If you are on the free tier or a new paid plan, you might hit rate limits quickly if you make too many requests in a short period. The API will return an error indicating rate limit issues. Wait a few moments before retrying or consider upgrading your plan if this is a persistent issue. The Restpack pricing page outlines request limits for each tier.

  • Network Issues: Ensure your internet connection is stable and that no firewalls or proxy settings are blocking outgoing HTTP requests from your environment to Restpack's API endpoints.

  • Target Website Issues: Sometimes the issue isn't with Restpack but with the target website. The site might be down, have implemented new anti-scraping measures, or have changed its structure. Try scraping a different, known-good URL (like https://example.com) to isolate the problem.

  • Check Restpack Status Page: If you suspect a wider issue, check the Restpack documentation or status page (if available) for any reported service outages or degradation.