Most small-business data pipelines have the same shape. The plumbing at each end is clean. In the middle, someone turns inconsistent inputs into a format a system can use. That middle step is the only place the Claude API belongs. Put it anywhere else and you add cost and failure modes to work a script does faster, for free.
We build Claude integrations through our AI integration service, so weigh that. Every fact below about Claude was checked against Anthropic’s own documentation in October 2026.
Short answer: Use Claude at the transform step, and only when it needs language understanding: reading PDFs, mapping vendor columns to your schema, classifying free text. Define the output schema first and send it with structured outputs. Check business rules in your own code. Send work that isn’t urgent through the Batch API at half price. In our worked example, 15,000 records a month costs about $60 on Claude Sonnet 5.5 or $3 on Claude Haiku 5.5. Leave clean, tabular data to scripts or dbt.
What Anthropic Documents
As of October 2026, from Anthropic’s documentation:
| Topic | What the docs say |
|---|---|
| Current models | Opus 5.5, Sonnet 5.5, Haiku 5.5 and Fable 5.1 |
| Sonnet 5.5 price | $2 input, $10 output, per million tokens |
| Haiku 5.5 price | $0.10 input, $0.50 output, per million tokens, for prompts up to 100,000 tokens |
| Batch API | 50% off input and output; up to 100,000 requests or 256 MB per batch |
| Batch timing | Most batches finish within an hour; unfinished requests expire after 24 hours |
| Structured outputs | Responses match your JSON schema, with documented exceptions |
| Context window | 1M tokens on all four current models, with up to 128K output tokens |
| PDFs | Up to 600 pages per request (100 below a 1M context window), 32 MB per request |
| Prompt caching | Cache reads cost 10% of the input price on Haiku 5.5, 5% on Opus 5.5 and Sonnet 5.5 |
| Token counting | An endpoint estimates input tokens before you send |
Sources: models overview, pricing, batch processing, structured outputs, PDF support, prompt caching, token counting.
What ETL Actually Means for a Small Business
ETL is getting data from where it lives, reshaping it into the format you need, and moving it to where it’s useful. At enterprise scale it’s a discipline with dedicated tooling. In a small business, it’s often a person with a spreadsheet.
Extract: where your data lives and how to pull it
Typical sources: a CRM, a WooCommerce store, a logistics provider, a form tool, Google Sheets, maybe a POS system. Pulling from them means API calls, CSV exports or database queries. Claude plays no role here. This is plumbing.
Transform: the step where human judgment used to be required
This is where data gets reshaped: dates reformatted, product names standardised, addresses parsed, segments assigned. For clean data, it’s mechanical. For messy or unstructured data, such as PDFs, vendor emails, free-text notes and scanned invoices, a person used to be needed. That’s the step Claude can automate.
Load: getting clean data where it needs to go
The transformed data is written to a database, a dashboard, a CRM or a report. No language model is needed. It’s a write operation.
The core point: Claude earns its place at the transform step, and only when that step needs language understanding, not arithmetic or rule-based logic.
Where Claude API Fits in a Data Transformation Pipeline
Claude is good at reading, interpreting and restructuring text, and every current model also accepts images (source). In ETL, that covers three jobs where it replaces manual work or brittle regex.
Parsing unstructured inputs: PDFs, emails, free-text fields
Supplier invoices are the classic case. Each vendor has its own layout, column names and date format, and someone copies line items into a sheet by hand. Claude can read the PDF directly: the API turns each page into an image and passes the page’s text alongside it, so tables and layout are visible to the model (source). You get line items back in your schema, and anything that fails a check goes to a person. The same pattern fits support emails, vendor feeds and free-text intake forms.
Schema normalization across inconsistent vendor formats
Five suppliers send CSVs that name the same field “SKU”, “item_number”, “product_code” and “part_no”. The traditional fix is a mapping table someone updates whenever a vendor changes format. The Claude fix: send the headers and a few sample rows, and ask for a mapping to your canonical schema. Store that mapping and apply it in code, calling Claude again only when the headers change.
Classification and enrichment at scale
Assigning a product category, tagging a ticket with a reason code, deciding which contract clause applies: these need language understanding, and rule-based systems fail on the edge cases. Claude handles them well when the prompt carries a clear rubric and the schema limits the answer to a fixed list of values with an enum (source). When categories overlap, the same edge case can land differently across runs, so tighten the rubric or add a review queue.
How to Build a Claude API ETL Workflow: A Practical Pattern
This is the pattern we build.
Step 1: Define inputs and outputs before touching the API
Write down exactly what goes in and what must come out. If you can’t define the output schema precisely, with field names, data types and valid values, you’re not ready to build. Ask vaguely and you pay for vague results.
Example input: "12 x Widget Pro (SKU: WP-2200) @ $4.50 each". Example output: {"sku": "WP-2200", "description": "Widget Pro", "quantity": 12, "unit_price": 4.50, "line_total": 54.00}. Define that before you write the prompt.
Step 2: Write the transformation prompt with strict output constraints
Put the format in the request, not only in the prompt. With structured outputs, you send a JSON schema and Claude returns valid JSON that matches it (source). The prompt then says what the input is and how to handle uncertainty. Never ask Claude to guess. Make uncertain fields nullable and add a review flag.
A prompt structure that works:
You are a data extraction assistant. Extract the fields defined in the schema from the input text. If a field cannot be determined from the input, return null for that field and set "review_required" to true. Do not infer values that are not in the text.
Input: [raw text]Edge-case rules go in the prompt. The shape of the answer goes in the schema.
Step 3: Handle errors, edge cases, and token limits
Structured outputs remove JSON syntax errors, not every failure. A refusal or a response cut off at max_tokens may not match the schema, and enum values can come back with different capitalisation (source). Check the stop reason on every response and compare enums case-insensitively.
The schema also can’t enforce limits such as minimum or maxLength; the SDKs move those into the field description and can validate them in your code (source). So keep business rules in your pipeline: does quantity times unit price equal the line total? Anything that fails goes to review, not to your database. Log the raw response next to the parsed one.
Token limits matter for large documents. Current models have a 1M-token context window (source), and Anthropic says a PDF page typically uses 1,500–3,000 text tokens, plus its image cost (source). You pay for all of it. Send one invoice per request, not a 40-page bundle, so a failure affects one record and a retry costs little.
Step 4: Cost estimation before you commit
Our own worked example, on Anthropic’s published prices (source). The token counts are assumptions; measure yours.
- Records: 500 a day, 15,000 a month
- Per record (assumed): 1,000 input tokens, 200 output tokens
- Monthly tokens: 15 million input, 3 million output
- Claude Sonnet 5.5: $30 input + $30 output = about $60 a month
- Claude Haiku 5.5: $1.50 input + $1.50 output = about $3 a month
- Through the Batch API: half of either figure
Count tokens against the model you’ll use: models from Claude 4.7 on use a tokenizer that produces about 30% more tokens for the same text (source). Your instructions and schema repeat on every request, so prompt caching helps, and it stacks with the batch discount (source). Anthropic positions Haiku 5.5 for high-volume extraction and routing, so test it first.
When Claude API Is the Wrong Tool for ETL
The honest version includes when not to use it.
Clean tabular data that doesn’t need language understanding
If your data is already structured, with consistent columns, predictable formats and no free text, a Python script or dbt does the job faster, cheaper and with no risk of an invented value. Don’t add a language model to a pipeline that doesn’t need one.
High-volume pipelines where token costs outpace value
Scale the example to 50,000 records a day, about 1.5 million a month: 1.5 billion input and 300 million output tokens. On published prices that’s about $6,000 a month on Sonnet 5.5, or about $300 on Haiku 5.5, before batch discounts. At that volume, ask whether a deterministic parser can do most of the work, with Claude only on the rows it can’t handle.
Cases where a simple script handles it
Date normalisation, currency stripping, column renaming and deduplication don’t need a language model. Code gives the same answer every time. A Claude call adds latency, cost and an external dependency for no gain.
Related Posts
- AI batch processing: queues, retries and large-volume runs
- AI document processing: extraction and routing into existing systems
- AI data entry and validation automation: checks that keep bad records out
- AI pipeline orchestration design: chaining Claude steps safely
FAQ
Can I use the Claude API inside an existing ETL pipeline like Airflow or dbt?
Yes. It’s an HTTP API, so it works anywhere you can make a network request. In Airflow, the @task decorator turns a plain Python function into a task (source), so the Claude call can be its own step. dbt Python models run on your data platform, such as Snowflake, BigQuery or Databricks, and must return a DataFrame (source). Often it’s simpler to run Claude before dbt.
How much does it cost to run Claude API transformations at SMB data volumes?
It depends on tokens per record, volume and model. In our worked example, 15,000 records a month at 1,000 input and 200 output tokens each costs about $60 on Claude Sonnet 5.5 and about $3 on Claude Haiku 5.5. The Batch API halves both. Cost rises in a straight line with volume, so run Claude only on the step that needs it.
Do I own the code and pipeline if Designodin builds it?
You should, and the contract should say so. The way we build these, the pipeline is plain code in your own repository, running under your own Anthropic API key, so any developer can pick it up. The one external dependency is the Claude API itself.
What data formats can Claude API handle in a transformation step?
Any text: CSV, JSON, XML, HTML, email bodies, Markdown. PDFs go in directly, including tables and charts, as long as they aren’t password-protected (source). JPEG, PNG, GIF and WebP images work too. Word and Excel files need converting to text first (source).
How is Claude API different from just using ChatGPT for data work?
Both can do structured extraction, and we won’t claim one wins on your data without testing. Run both against a sample of real records and compare the failures, not the demos.
Can this replace a full-time data analyst?
Partly. Claude handles the mechanical side: extraction, normalisation, classification, reformatting. It doesn’t replace judgment, like spotting anomalies or asking whether a number makes sense. It can take the retyping off that person’s desk, if the inputs are consistent enough. With messy source data, some of that time moves to pipeline maintenance.