> ## Documentation Index
> Fetch the complete documentation index at: https://docs.adriel.com/llms.txt
> Use this file to discover all available pages before exploring further.

# How to connect

> How to connect the Custom URL Import data source to Adriel: fetch JSON or CSV from any HTTP/HTTPS endpoint on a daily schedule.

Custom URL Import ingests data from any HTTP/HTTPS endpoint that returns JSON or CSV. Connecting it fetches the endpoint on a daily schedule, parses the response, and caches the rows for dashboard queries — covering partner-hosted export endpoints, internal reporting APIs, and public datasets that lack a native connector.

<Note>
  **Custom URL Import is a paid add-on**

  It is not included in standard plans. Contact sales for availability and pricing.
</Note>

## Before you connect

The following are required:

* An HTTP/HTTPS URL that returns JSON or CSV.
* Any authentication credentials the source requires — a static key or token, or refresh-token rotation details.
* A decision on refresh mode: whether the endpoint accepts a date filter (date-parameterized) or always returns the full list (daily full-list refresh).
* For date-parameterized endpoints, one or more unique-key columns that identify a row.

## Connect Custom URL Import

<Steps>
  <Step title="Set the connection name">
    On the **Data Sources** page, search for **Custom URL Import** and select it. Fill in the **Connection Name**. This field labels the connection credentials and helps differentiate between multiple sets of credentials.

    Optionally, expand **Configuration (optional)** and use **Save setup for reuse?** to store a subset of the setup fields on the connection — for example the data source name, the fetch and refresh windows, CSV column names, the filter key and value, and the currency. Saved fields are filled in automatically the next time a data source is created from this connection, so similar imports can be set up without re-entering the same values. All saved fields are optional.
  </Step>

  <Step title="Fill in the required data source settings">
    Complete the five required fields:

    * **Name**: An identifiable name for the data source.
    * **Data Type**: Choose **CSV** or **JSON** to match the format returned by the API.
    * **URL**: The API URL used to retrieve the data. Replace any date parameters with `$date` — for example, `https://api.example.com/data?from=20250101&to=20250201&type=csv` becomes `https://api.example.com/data?from=$date&to=$date&type=csv`. For a POST method, no date parameter is required in the URL; date parameters can live in the body parameters instead. `$date` is a placeholder — the actual date range for the fetch is defined by the connector configuration, not by the literal values in the URL.
    * **Date format**: The date format expected by the API, as documented by the source. See [date-fns format](https://date-fns.org/v4.1.0/docs/format). Use lowercase tokens such as `yyyy-MM-dd` or `yyyy/MM/dd`; uppercase `YYYY` or `DD` is rejected.
    * **HTTP Method**: The method used for the API call — **GET** or **POST**.
  </Step>

  <Step title="Set optional configurations">
    The remaining fields are optional and grouped by purpose. Leave any of them blank to use the default behavior.

    <AccordionGroup>
      <Accordion title="Channel display">
        * **Alternative Channel Name**: Customizes the channel display name. Defaults to `customUrlImport` if left blank.
        * **Alternative Channel Icon**: Customizes the channel icon. Upload an SVG file.
      </Accordion>

      <Accordion title="Row identity and refresh window">
        * **Unique Identifier Columns**: The columns that uniquely identify each row (typically date, name, or IDs). Separate multiple columns with commas — for example, `ID,NAME`. When a matching value exists, that row is updated; otherwise a new row is added. Leaving this blank replaces all data daily. Entering a column that does not exist in the dataset prevents the connector from being created.
        * **Number of Days to Fetch**: How many days of historical data to cache when the connector is created. Defaults to 180 days. Values above the API limit are capped.
        * **Number of Days to Refresh**: How many days of past data to re-fetch on each daily refresh. Defaults to today. Values above the API limit are capped.
        * **Replace Entire Cache On Refresh**: When enabled, every refresh deletes all existing rows in the cache before inserting the newly fetched rows. Keep it disabled to preserve older dates that fall outside the refreshed range. Intended for endpoints that always return the complete current list.
      </Accordion>

      <Accordion title="Response parsing">
        * **CSV Column Names** (CSV): If the API response has no column headers, provide them in the exact order they appear, separated by commas — for example, `date, campaign, clicks, impressions`.
        * **Response Body Path** (JSON): If report rows are nested in the response, enter the JSON path to the array. For `{"data": {"reports": []}}`, enter `data.reports`. If rows are not nested, enter the key on its own.
        * **Body Params** (POST): The body parameters for the POST request, as valid JSON. Replace date parameters with `$date` — for example, `{"start_date": $date, "end_date": $date, "timezone": "UTC", "metrics": []}`.
        * **Keys To Omit Before Flatten** (JSON): Comma-separated keys to drop from each record before it is flattened, using dot notation for nested keys — for example, `user.password,metadata.debug`.
      </Accordion>

      <Accordion title="Row filtering">
        * **Filter Key**: A field used to keep only matching rows from the response. Nested values are supported with dot notation — for example, `data.user.name`.
        * **Filter Value**: The value that **Filter Key** must match for a row to be imported. Enter it exactly as it appears in the source, matching capitalization.
      </Accordion>

      <Accordion title="Currency">
        * **Currency**: Use when the source currency differs from the workspace currency. Provide a three-letter ISO code (for example, `USD`, `EUR`). After creation, open **Data display settings** in the data source settings and set the type of each metric to convert as **Currency**. The metric is converted using the latest exchange rate.
      </Accordion>

      <Accordion title="Authentication and token refresh">
        * **HTTP Query Header**: Required if the API needs a key or bearer token. Provide as valid JSON — for example, `{"x-api-key": "YOUR_API_KEY"}` or `{"Authorization": "Bearer YOUR_API_TOKEN"}`. The location is documented in the API's authentication, headers, or tokens sections.
        * **Does the API require a refresh token?**: Enable this when the token expires and must be refreshed regularly. The token parameter is then referenced as `$token` in the URL, the HTTP Query Header, or the Body Params (for POST), and the retrieved token is reused for one hour by default. Enabling it reveals the following fields:
          * **Refresh Token Url**: The endpoint that issues a fresh access token.
          * **Refresh Token Request Body**: The request body sent to that endpoint, as valid JSON.
          * **Refresh Token Response Path**: The JSON path to the access token in the response. For `{"data": {"token": []}}`, enter `data.token`.
          * **Token Cache TTL**: How long, in seconds, to keep the token before refreshing it. Defaults to 3600 (one hour).
          * **Refresh Token Rotation Path**: Only needed when the token endpoint returns a new refresh token on each call. Enter the JSON path to that rotating refresh token in the response.
      </Accordion>

      <Accordion title="Pagination">
        * **Does the API use pagination?**: Enable this when the API returns paged data. Enabling it reveals the following fields:
          * **Pagination Mode**: Choose **Page Number** when requests use `page=1,2,3…` (referenced as `$page`), or **Offset** when requests use `offset=0,100,200…` (referenced as `$offset`). Add the matching placeholder to the URL, HTTP Query Header, or Body Params.
          * **Offset Step** (Offset mode): The number of records to advance per page. Must be at least 1.
          * **Total Count Path**: Optional but recommended. The JSON path to the total record count in the response — for example, `pagination.total`.
          * **Total Number of Pages Path**: Optional fallback when a total count is unavailable. The JSON path to the total page count — for example, `pagination.totalPages`.
      </Accordion>
    </AccordionGroup>
  </Step>

  <Step title="Submit">
    Click **Submit** to complete the connection. Initial data availability can take up to one business day.
  </Step>
</Steps>

## What gets imported

For JSON responses, the configured response-body path is extracted and the result is flattened with an underscore delimiter. For CSV responses, the body is parsed directly. Numeric columns become metrics and text columns become breakdowns. Date-parameterized mode tags each row with a `_date` column.

For the complete field list and refresh-mode logic, see the [Custom URL Import data reference](/data-sources/a-f/custom-url-import/data-reference).

## Troubleshooting

<AccordionGroup>
  <Accordion title="Refresh fetches the wrong rows or duplicates them">
    The unique-key configuration does not match the row identity in the response. Verify that the declared keys uniquely identify each row, particularly in date-parameterized mode where the upsert depends on them.
  </Accordion>

  <Accordion title="Pagination loops or returns only the first page">
    CSV responses do not accumulate paginated pages — only the first page is consumed. For multi-page CSV, switch the source to a JSON wrapper or have the endpoint return a single page. For JSON, verify the pagination mode: page-number is 1-based, and offset requires an offset step of at least 1.
  </Accordion>

  <Accordion title="Authentication fails partway through a refresh">
    The static token has expired or the rotation endpoint is misconfigured. For static tokens, refresh and replace the value in the headers. For rotation, verify the refresh token URL and the JSON path that reads the new token from the response.
  </Accordion>

  <Accordion title="A date or token placeholder is not being substituted">
    The `$date`, `$plusOneDate`, and `$token` placeholders are substituted in the URL, the request headers, and the POST body. Confirm the placeholder is spelled exactly and placed in one of those inputs, that the date format uses lowercase tokens, and — for `$token` — that the refresh-token option is enabled.
  </Accordion>
</AccordionGroup>

## Related

<CardGroup cols={2}>
  <Card title="Custom URL Import data reference" href="/data-sources/a-f/custom-url-import/data-reference">
    Fields, refresh modes, pagination, and limits for Custom URL Import.
  </Card>

  <Card title="Custom URL Import FAQs" href="/data-sources/a-f/custom-url-import/faqs">
    Common questions and expected behaviors for the Custom URL Import data source.
  </Card>

  <Card title="How to connect CSV Files" href="/data-sources/a-f/csv-files/how-to-connect">
    Upload data files directly instead of fetching from a URL.
  </Card>

  <Card title="How to connect Amazon S3" href="/data-sources/a-f/amazon-s3/how-to-connect">
    Import bucket-hosted exports from Amazon S3.
  </Card>
</CardGroup>