OpenETL

OpenETL is a free, lightweight and stateless open-source ETL (Extract, Transform, Load) framework built in TypeScript. It provides a standard approach for you to pull data from APIs like HubSpot, querying databases like PostgreSQL, or transforming datasets for custom workflows. OpenETL provides a lightweight and flexible solution under the MIT license.

Overview

OpenETL simplifies data integration by offering a modular architecture that connects diverse data sources and targets through reusable adapters and connectors. It's designed for developers who need a fast, extensible way to build ETL pipelines without the overhead of heavy tools. With built-in support for authentication, transformations, and error handling, it's perfect for both small scripts and large-scale data workflows.

Key Features

• Adapter-Based Design: Connect to any data source (e.g., APIs, databases) using standard notations;
• TypeScript Native: Strong typing and modern JavaScript for a robust development experience.
• Flexible Transformations: Manipulate data with built-in functions like concat, uppercase, or custom logic.
• Error Handling & Retries: Configurable retry logic and rate limiting for reliable operations.
• Lightweight & Free: MIT-licensed with no dependencies beyond Node.js essentials.

Example

Here's a quick peek at how OpenETL moves contacts from HubSpot to PostgreSQL:

import Orchestrator from 'openetl';
import { hubspot } from '@OpenETL/hubspot';
import { postgresql } from '@OpenETL/postgresql';

// Load only what you need to use
const adapters = { hubspot, postgresql };

// General vault to keep your credentails
const vault = {
  'hs-auth': {
      type: 'oauth2',
      credentials: {
          client_id: 'your-id',
          client_secret: 'your-secret',
          refresh_token: 'your-token'
      }
  },
  'pg-auth': {
      type: 'basic',
      credentials: {
          username: 'your-user',
          password: 'your-pass',
          host: 'localhost',
          database: 'your-db'
      }
  },
};

const orchestrator = Orchestrator(vault, adapters);

orchestrator.runPipeline({
  id: 'hs-to-pg',
  source: {
    adapter_id: 'hubspot',
    endpoint_id: 'contacts',
    credential_id: 'hs-auth',
    fields: [
        'firstname',
        'lastname'
    ],
  },
  target: {
    adapter_id: 'postgresql',
    endpoint_id: 'table_insert',
    credential_id: 'pg-auth',
    config: {
        schema: 'public',
        table: 'contacts'
    },
  },
});

Quick Start

Try It Out in 5 Minutes

1. Install: Run npm install OpenETL in your Node.js project.
2. Setup: Clone the repo from GitHub and add your credentials to a vault (see Getting Started).
3. Run: Copy the example above into a .ts file, tweak the credentials, and execute with ts-node yourfile.ts.

Next Steps

Dive deeper with our Getting Started guide to set up your environment, explore adapters, and build your own pipeline.

FAQ

What platforms does OpenETL support?

We just started, but we are working to bring up to 50 different adapters in 2025. Currently supported adapters include HubSpot and PostgreSQL.

How do I create a new adapter?

Define your data source's endpoints, implement connect, download, and upload methods, and plug it into OpenETL. Check out our Adapter Development Guide for a step-by-step guide.

Where can I get help?

Join our community via Support or explore issues on Github. We're here to help!

Why Contribute?

Impact of Your Contribution

Adding an adapter (e.g., for Stripe or MySQL) empowers the community to integrate new data sources, making OpenETL more versatile. Your code could be the bridge that connects someone's workflow!

Who Should Contribute?

• Developers familiar with APIs or databases who want to share their expertise.
• Open-source enthusiasts looking for a practical, impactful project.
• Anyone who loves TypeScript and data wrangling!

Get Involved

Join the Community

Chat with us on our Support channels—Discord, forums, or email—to share ideas, ask questions, or collaborate.