# ETL Tools for Managed Ingestion, Custom Processing and Reliable Pipelines

Choose Fivetran for qualified managed replication, Airbyte for configurable connector and deployment choices, AWS Glue for custom processing in an AWS operating model, or Meltano Open for code-owned ingestion. Useful ETL tools must preserve the source changes and analytical output your business needs. Compare connector behavior, transformation location, recovery and complete cost before ranking products by their number of integrations.

Canonical URL: https://www.martechdevs.com/blog/etl-tools

By [Jafrin Akter Alo](https://www.martechdevs.com/authors/jafrin-akter-alo)

Data Engineer and Co-founder at [Martechdevs](https://www.martechdevs.com) and [Growth and Analytics Partners](https://www.growthandanalytics.com)

[LinkedIn](https://linkedin.com/in/jafrinalo)

Last updated: 2026-11-15

Next step: [Data-pipeline implementation](https://www.martechdevs.com/data-warehouse-cdp). Qualified connectors, accepted models and recoverable progress.

## ETL tools compared

The USD entry offers use different billing units; the Glue figure is AWS's published Spark-job example rate, with regional pricing still requiring confirmation.

| Tool | Public entry basis | Best fit | Main tradeoff |
| --- | --- | --- | --- |
| Fivetran | Free up to 500,000 connection MAR/month; paid usage and applicable connection charges | Managed source replication with a qualified connector | Active-row accounting and connector scope need review |
| Airbyte | Core free to self-host; managed Standard from $20/month with five credits | Configurable sources and a choice of operating model | Source-dependent credits and self-hosting responsibilities differ |
| AWS Glue | Spark-job example uses $0.44 per DPU-hour | Custom AWS data-processing jobs | Engineering, permissions and related services remain part of the system |
| Meltano Open | Free self-hosted software; infrastructure and operations separate | Engineering-owned pipelines in code | The team maintains the runtime, connectors and recovery |

Sources: [Fivetran pricing](https://www.fivetran.com/pricing), [Airbyte pricing](https://airbyte.com/pricing), [AWS Glue pricing](https://aws.amazon.com/glue/pricing/) and [Meltano options](https://meltano.com/pricing). A free software distribution and a managed service represent different operating responsibilities.

The term ETL tools covers several kinds of data integration software. Some products primarily extract and load records; another processing layer may perform the business transformation. The [ETL versus ELT guide](https://www.martechdevs.com/blog/etl-vs-elt) explains why that boundary belongs in the selection.

At Martechdevs, [data warehouse and pipeline implementation](https://www.martechdevs.com/data-warehouse-cdp) starts with the source contract and expected output. We test updates, retries and reconciliation before expanding to the full source inventory, so the tool choice reflects the workflow the team will actually operate.

## Tool decisions

### Fivetran

- **Best for:** Teams seeking managed replication for supported sources with a defined connector and support arrangement.
- **Pros:** The current Standard offer includes managed connectors, fifteen-minute sync scope and dbt Core integration. This can fit a team that wants source movement handled within a managed platform. See [Fivetran plans](https://www.fivetran.com/pricing).
- **Cons:** The final business model still needs an owner. Usage depends on active keys and connection boundaries, so the destination's stored row count is an incomplete budget input.
- **Free plan and startup offer:** Free includes 500,000 connection MAR/month and a separate 5,000 monthly model-run allowance. New connections have fourteen-day free trials. No startup discount is assumed. See [free and evaluation scope](https://www.fivetran.com/pricing).
- **Choose it if:** The actual source objects, historical range and update behavior pass a representative pilot with an acceptable complete operating cost.
- **Skip it if:** A needed field or replication behavior remains unsupported, or the estimate treats all connections and source changes as identical.

### Airbyte

- **Best for:** Teams that need connector configuration or source customization and want to evaluate managed versus self-hosted operation.
- **Pros:** Its sync modes distinguish source reading from destination writing. That makes append, overwrite and deduplicated output explicit for supported streams. See [Airbyte sync modes](https://docs.airbyte.com/platform/using-airbyte/core-concepts/sync-modes).
- **Cons:** Stream capabilities depend on both source and destination support. A connector name alone does not establish deletion handling or the exact historical result.
- **Free plan and startup offer:** Core is free to self-host under Elastic License 2.0. Cloud advertises a fourteen-day no-card trial with 400 credits, described as about 100 GB; actual metering varies by source type. No startup concession is assumed. See [Airbyte pricing](https://airbyte.com/pricing).
- **Choose it if:** The selected streams and operating model fit the requirement, and the team can maintain any custom connector or self-hosted component.
- **Skip it if:** The proposed saving depends on self-hosting while nobody is responsible for infrastructure, upgrades and recovery.

### AWS Glue

- **Best for:** An engineering team with an AWS environment and a defined requirement for custom data processing and transformation.
- **Pros:** Glue runs ETL jobs in a serverless environment and connects them to specified sources, targets, IAM roles and network settings. It supports a managed execution environment for code the team owns. See [Glue execution concepts](https://docs.aws.amazon.com/glue/latest/dg/how-it-works.html).
- **Cons:** Managed runtime does not remove responsibility for job logic, access, source behavior or accepted output. Storage, catalog and other services can add charges.
- **Free plan and startup offer:** Glue Data Catalog has a separate free allowance for its first million metadata objects and million monthly metadata requests. That is not a blanket free ETL-compute allowance. Temporary account or startup credits are excluded from this budget. See [Glue pricing](https://aws.amazon.com/glue/pricing/).
- **Choose it if:** Custom processing is justified and the team can maintain the job, permissions and failure response inside its AWS operating model.
- **Skip it if:** The main requirement is a simple managed SaaS connector and a custom job would create unnecessary maintenance.

### Meltano Open

- **Best for:** Engineers who want to own pipeline configuration and deployment in code using qualified Singer connectors.
- **Pros:** Meltano manages tap and target configuration, stream selection and incremental state. Its documentation makes the state identity and replication methods explicit. See [Meltano replication](https://docs.meltano.com/guide/integration).
- **Cons:** Connector behavior and state recovery still need validation. Self-hosting leaves infrastructure, monitoring and continuing operation with the team.
- **Free plan and startup offer:** Open is the self-hosted open-source option. Meltano Cloud is a distinct managed offering; Open's free availability does not imply a free managed runtime. No startup discount is assumed. See [Meltano deployment choices](https://meltano.com/pricing).
- **Choose it if:** The team values code ownership and has the capacity to maintain the selected taps, targets, state and runtime.
- **Skip it if:** The purpose of the purchase is to remove engineering operation and there is no owner for the proposed self-hosted workflow.

## Data pipeline tools and transformation location

Compare data integration tools against the required source objects and output model. A connector that supplies contacts may not supply the custom object or field history that the business report depends on. A database connection may support a full extract while needing additional configuration for reliable changes.

Record the source key, update marker, deletion requirement, historical range and data types. Include source permissions and API limits. A missing record can result from the source account's access rather than a product defect.

Then locate the transformation. Source movement, type normalization and the business model are different responsibilities. A connector can load valid tables while a downstream join still multiplies order value or changes the intended customer population.

| Requirement | Evidence from the candidate | Responsible owner |
| --- | --- | --- |
| Required source field | Field present with the intended meaning | Source and integration owners |
| Historical correction | Older changed record reaches the accepted model | Pipeline owner |
| Business transformation | Known source fixture produces expected output | Data-model owner |
| Deletion or deactivation | Intended target behavior after the source change | Source and destination owners |
| Recovery | Retried or resumed run preserves correct records | Operating owner |

The [ETL pipeline guide](https://www.martechdevs.com/blog/etl-pipeline) provides a runnable local example of keyed loading and recoverable progress. Use that style of acceptance fixture with the actual candidate rather than relying on a successful first import.

## Incremental state and replay

Incremental processing needs a source mechanism and retained progress. Airbyte's documentation separates incremental reading from append, overwrite and deduplicated destination writes, with available combinations depending on the stream. Those names describe different output histories. See [sync behavior](https://docs.airbyte.com/platform/using-airbyte/core-concepts/sync-modes).

Meltano likewise distinguishes log-based, key-based and full-table replication. Its key-based approach identifies inserts and updates through a replication key but does not capture deleted rows through that mechanism. A source with required deletion handling needs a qualified alternative. See [Meltano replication methods](https://docs.meltano.com/guide/integration).

State is part of the operating data. Meltano documents that saved bookmarks take precedence over an earlier configured start date. Changing that date alone therefore does not establish that an old interval will be replayed. Qualify the actual full-refresh or state-management procedure before attempting historical repair.

Test a restart after partial work. Some data may have reached the target even if the overall run failed. The accepted result should preserve the intended records and progress after the retry, without skipping data or creating an extra business record.

Keep replay separate from ordinary recovery where necessary. Rebuilding historical input can produce different usage and downstream behavior from retrying a small failed batch. The operator should know the intended interval, target keys and effect on current data.

## Billing units and active-row behavior

Fivetran MAR measures distinct active keys within its documented counting boundaries. Repeated ordinary updates to the same key in one calendar month do not each create another active key, while separate connections and specialized source behavior require qualification. See [Fivetran usage definitions](https://fivetran.com/docs/core-concepts/usage-based-pricing).

Consider a hypothetical current-state connection with 500 new keys and 100 other keys each updated four times in the month. Assume the populations do not overlap, with no history-mode expansion, initial-load concession or other special accounting. There are 900 transfers in the simple illustration but 600 distinct active keys.

| Hypothetical quantity | Calculation | Result | Interpretation |
| --- | --- | ---: | --- |
| New-key transfers | Given | 500 | New records |
| Update transfers | 100 × 4 | 400 | Repeated changes to existing keys |
| Combined transfers | 500 + 400 | 900 | Transfer count |
| Distinct active keys | 500 + 100 | 600 | Active-key count under stated assumptions |

The source and configuration must match the assumptions before applying that unit to a bill. Fivetran also documents a $5 base charge for paid connections with monthly usage from one MAR through one million MAR, excluding Free-plan accounts. Use the actual plan and estimator rather than treating the worked key count as a complete invoice. See [connection charges](https://fivetran.com/docs/core-concepts/usage-based-pricing).

Airbyte's credits use source-dependent metering, so the 900-transfer count cannot be converted directly into credits without the relevant source schedule. AWS Glue DPU-hours are processing capacity over time, and Meltano Open's operating cost is a separate infrastructure and support question. Preserve those units in the comparison.

## Scoped compute and operating examples

For a hypothetical Glue Spark workload, assume four DPUs run for twenty billed minutes on each of thirty runs. That is 4 × 20 ÷ 60 × 30 = 40 DPU-hours. At the $0.44/DPU-hour rate in AWS's published Spark example, the compute line is $17.60. Confirm the selected region, engine and billing terms before using that rate in a proposal. See [AWS Glue pricing examples](https://aws.amazon.com/glue/pricing/).

The example excludes source retrieval fees, storage, requests, catalog activity and other services. It also assumes the stated twenty minutes is the billable run duration. A low processing subtotal does not establish the cost of developing and operating the job.

At Airbyte Standard's $20 monthly starting base, twelve unchanged base payments total $240. That is entry-subscription arithmetic, not a price for the Glue workload or the active-row example. Additional usage and required features can change the amount. See [Airbyte plans](https://airbyte.com/pricing).

For a separate self-hosted planning example, assume $120/month of infrastructure and a $350/month operating allowance. The stated total is $470/month, or $5,640/year. These amounts are hypothetical inputs, not Meltano fees or measured staffing costs. Replace them with the actual environment and maintenance arrangement.

The [software pricing tool](https://www.martechdevs.com/tools/software-pricing) helps keep those scopes separate. The [Fivetran and Airbyte comparison](https://www.martechdevs.com/blog/fivetran-vs-airbyte) goes deeper on the managed-versus-configurable choice, while [Airbyte alternatives](https://www.martechdevs.com/blog/airbyte-alternatives) covers a narrower replacement decision.

## Permissions, environments and operations

The data pipeline needs credentials that can retrieve the required source data and write the intended target. It should not depend on an employee's untracked account merely because that account made the trial convenient.

AWS Glue's execution documentation describes the IAM role and network settings supplied for jobs. Those settings are part of the implementation, alongside the code. A managed runtime does not establish that every source is reachable or that the role has the appropriate permissions. See [Glue jobs and access](https://docs.aws.amazon.com/glue/latest/dg/how-it-works.html).

Keep production and evaluation environments identifiable in usage and monitoring. A second connection or worker can create additional cost, and a test write should not be mistaken for a production business record.

Give every important failure an owner. Source access, schema changes, rejected records and incorrect report totals may require different people. The [data incident runbook](https://www.martechdevs.com/blog/data-incident-runbook) helps turn a failed job into a specific recovery action.

## Best fit by pipeline operating model

| Company or operating model | Best fit | Runner-up | Decision reason |
| --- | --- | --- | --- |
| Small data team seeking managed replication | Fivetran | Airbyte managed plan | Qualified source behavior and operating support lead the decision |
| Team with unusual source or connector requirements | Airbyte | Meltano Open | Configuration and connector ownership deserve direct testing |
| AWS engineering team requiring custom processing | AWS Glue | Maintained code pipeline | Runtime and access fit the existing operating environment |
| Team maintaining ingestion as a code project | Meltano Open | Airbyte Core | Implementation control can justify self-hosted responsibility |
| Existing pipeline with one wrong model or cursor | Repair of the current setup | Qualified alternative | A replacement can reproduce the same incorrect rule |

## ETL-tool acceptance sequence

1. Write the required source objects, history and destination result.
2. Identify the transformation and operating owner.
3. Qualify connector fields, update/deletion behavior and permissions.
4. Load a bounded sample with known business keys and expected values.
5. Test an update, retry, partial failure and historical correction.
6. Reconcile records and the final business model, not only transport counts.
7. Measure the relevant billing units and required surrounding services.
8. Approve the complete scope and handover before expanding.

Do not compare candidates using different source permissions or easier samples for one tool. Preserve the fixture and expected output so a change in the result has an explainable cause.

## Data-pipeline tool decision

![Custom processing needs route to Glue or a code-owned pilot, otherwise to Fivetran or Airbyte. Both undergo source-change and recovery validation; connector/state/model repair repeats validation before the complete usage and ownership budget.](https://www.martechdevs.com/assets/blog/etl-tools.svg)

Diagram connections:

- Source and transformation requirements to Custom processing priority. 
- Custom processing priority to Glue or code-owned pilot: Yes. 
- Custom processing priority to Fivetran or Airbyte pilot: No. 
- Glue or code-owned pilot to Change and recovery validation. 
- Fivetran or Airbyte pilot to Change and recovery validation. 
- Change and recovery validation to Accepted source and model output. 
- Accepted source and model output to Connector state or model repair: No. 
- Connector state or model repair to Change and recovery validation. 
- Accepted source and model output to Complete usage and ownership budget: Yes. 
- Complete usage and ownership budget to Selected pipeline configuration

## ETL tool checklist

- [ ] Required source objects, fields and historical coverage.
- [ ] Stable business keys and qualified update/deletion mechanisms.
- [ ] Explicit source-reading and destination-writing behavior.
- [ ] A named transformation location and model owner.
- [ ] Known retry, partial-failure and historical-repair cases.
- [ ] Reconciled destination records and accepted business totals.
- [ ] Correct MAR, credit, compute or infrastructure units.
- [ ] Production and non-production usage included deliberately.
- [ ] Source credentials, target permissions and network ownership.
- [ ] A continuing operator and documented recovery procedure.

The [marketing data audit skill](https://www.martechdevs.com/skills/marketing-data-audit) helps organize the source and reconciliation evidence. Keep that evidence with the selected configuration so the next connector or schema change can repeat the same checks.

## Frequently asked questions

### Does an ETL tool automatically perform every business transformation before warehouse loading?

An ETL tool does not automatically perform every business transformation before warehouse loading. Products marketed in the category can focus on extraction and loading, with business models maintained elsewhere. Identify the actual processing stage and owner for each required output. The accepted design can combine upstream transformation and destination modeling rather than forcing every job into one product label.

### Can Airbyte sync modes be treated as interchangeable ways to refresh a table?

Airbyte sync modes should not be treated as interchangeable refresh settings. They combine source-reading behavior with destination-writing behavior, including append, overwrite and deduplicated output where supported. The choice affects history, repeated runs and the final table. Test the required stream and expected result before changing the mode. See [Airbyte sync modes](https://docs.airbyte.com/platform/using-airbyte/core-concepts/sync-modes).

### Does changing a Meltano start date automatically replay older bookmarked data?

Changing a Meltano start date does not automatically replay older bookmarked data. Its documentation states that saved incremental bookmarks take precedence over the configured start date. Use the qualified refresh or state-management procedure for the intended interval, and verify the destination effect before broad replay. See [Meltano state and start-date behavior](https://docs.meltano.com/guide/integration).

### Does Glue's Data Catalog free allowance make ETL compute free as well?

Glue's Data Catalog free allowance does not make every ETL compute job free. Metadata objects and requests are separate from processing jobs and their applicable usage charges. Estimate the actual engine, capacity, billed duration and surrounding services, with the selected region's terms. Keep temporary credits separate from the normal operating budget. See [AWS Glue pricing](https://aws.amazon.com/glue/pricing/).

### Can Fivetran MAR be estimated by counting every repeated source update?

Fivetran MAR should not be estimated by counting every repeated update as a new active key. Ordinary repeated changes to one key within the same monthly counting boundary generally count once, while connections and special source behavior can change the calculation. Match the source and plan to the documented rules before estimating an invoice. See [Fivetran usage definitions](https://fivetran.com/docs/core-concepts/usage-based-pricing).

## Sources

1. [Fivetran pricing and free allowances](https://www.fivetran.com/pricing)

2. [Fivetran MAR and connection charges](https://fivetran.com/docs/core-concepts/usage-based-pricing)

3. [Airbyte pricing and licensing](https://airbyte.com/pricing)

4. [Airbyte sync modes](https://docs.airbyte.com/platform/using-airbyte/core-concepts/sync-modes)

5. [AWS Glue pricing](https://aws.amazon.com/glue/pricing/)

6. [AWS Glue execution concepts](https://docs.aws.amazon.com/glue/latest/dg/how-it-works.html)

7. [Meltano Open and Cloud options](https://meltano.com/pricing)

8. [Meltano replication and state](https://docs.meltano.com/guide/integration)
