Integrating Web Scraping API with Python

Basic Integration with Product Data Scrape

Every web scraping API has a similar pattern. Here is the Product Data Scrape integration:

import httpx

API_KEY = "your_product_data_scrape_api_key"
BASE_URL = "https://api.productdatascrape.com/v1"

async def fetch_product(url):
    async with httpx.AsyncClient() as client:
        response = await client.get(
            f"{BASE_URL}/product",
            params={"url": url},
            headers={"Authorization": f"Bearer {API_KEY}"},
            timeout=30,
        )
        response.raise_for_status()
        return response.json()

Connection Pooling for Production

For high-volume use, reuse a single HTTP client:

class ProductDataScrapeClient:
    def __init__(self, api_key):
        self._client = httpx.AsyncClient(
            base_url=BASE_URL,
            headers={"Authorization": f"Bearer {api_key}"},
            timeout=30,
            limits=httpx.Limits(max_keepalive_connections=20, max_connections=100),
        )
    
    async def get_product(self, url):
        response = await self._client.get("/product", params={"url": url})
        response.raise_for_status()
        return response.json()

Batch Endpoints for Scale

If you need 1,000+ products, the Product Data Scrape batch endpoint is dramatically more efficient.

Sample Batch Response from Product Data Scrape

{
  "job_id": "batch_2026_06_xyz789",
  "status": "completed",
  "submitted_at": "2026-05-15T10:00:00Z",
  "completed_at": "2026-05-15T10:08:34Z",
  "total_urls": 1000,
  "successful": 987,
  "failed": 13,
  "credits_used": 1000,
  
  "results_url": "https://api.productdatascrape.com/v1/jobs/batch_2026_06_xyz789/results.json.gz",
  
  "sample_result": {
    "url": "https://www.amazon.com/dp/B0CHX1W1XY",
    "status": "success",
    "data": {
      "asin": "B0CHX1W1XY",
      "title": "Echo Dot (5th Gen)",
      "price": 44.99,
      "rating": 4.6
    }
  },
  
  "webhook_delivery": {
    "url": "https://your-app.com/scrape-webhook",
    "status": "delivered",
    "delivered_at": "2026-05-15T10:08:35Z"
  }
}

How Product Data Scrape Helps

The Product Data Scrape Python SDK handles connection pooling, retries, async batch, webhooks. Just pip install pds-api.

About Product Data Scrape

Product Data Scrape is the leading provider of managed web scraping services and ready-to-use product datasets. We help 200+ brands, retailers, and AI companies turn the messy public web into clean, structured product data.

Our Services: - Web Scraping API — REST API for developers (1,000 free credits) - Scraper as a Service — Custom scrapers built in 7-10 days - Ready Datasets — 100+ pre-built datasets, free 1,000-row samples in 24 hours

Contact: - Website: https://www.productdatascrape.com - Email: sales@productdatascrape.com