source-connectors

The Python data-extraction platform behind our NetSuite, Salesforce, Jira, SharePoint and Redshift integrations. Every connector exposes the same check_connection() / discover() / read() / read_df() API, so a pipeline built against one connector reads the same way against any other.

Connectors

• NetSuite — Objects (REST API), SuiteQL, and RESTlet scripts, all authenticated with OAuth 1.0a
• Salesforce — SOQL, with OAuth2 or direct token authentication
• Jira Cloud — JQL queries against the REST API
• SharePoint — via the Microsoft Graph API
• Amazon Redshift — via the Data API, provisioned or Serverless

Most connectors expose check / discover / read from the command line; NetSuite SuiteQL, NetSuite RESTlet and Redshift add a query / call command in place of read.

Typed, schema-checked output

read_df() returns a dict of pandas DataFrames keyed by stream name, cast to the dtypes a schema declares. An unrecognised stream raises immediately, an unrecognised column is dropped, and a missing column is logged and skipped — so a pipeline fails loudly on the mistakes that matter and stays quiet on the ones that don’t.

python -m source_connectors netsuite-suiteql query \
  --realm 1234567_SB1 --consumer-key KEY --consumer-secret SECRET \
  --token-key TOKEN --token-secret TOKEN_SECRET \
  --query "SELECT id, companyName FROM customer WHERE lastModifiedDate > '2024-01-01'" \
  --output-file customers.jsonl --state-file customers.state.json

Salesforce SOQL, NetSuite SuiteQL, NetSuite RESTlet and Jira Cloud also cross-check the number of records fetched against what the API itself declared, attaching the result to df.attrs["completeness"] rather than silently trusting a paginated read completed.

Availability

This is under active internal development and not yet public. Get in touch if you’d like early access or a connector built for a system not listed here.