Connect to Arcion

Arcion unlocks the value of transactional and operational data with real-time, distributed change data capture (CDC) that has built-in heterogeneous schema management, high availability (HA), and auto-scaling.

Connect to Arcion using Partner Connect

This section describes how to connect to Arcion using Partner Connect.

Note

Partner Connect only supports Databricks SQL warehouses (formerly Databricks SQL endpoints) for Arcion. To connect Arcion to a Databricks cluster, connect to Arcion manually.

To connect to Arcion using Partner Connect, do the following:

1. Connect to ingestion partners using Partner Connect.

   If you create a new Arcion account, be sure to choose the Arcion Cloud (Beta) option. (Do not choose the self-hosted option.)

   Important

   If you sign in to your organization’s Arcion account, a Replications page may display, listing one or more existing replication entries with the Databricks logo. These entries might contain details for compute resources in workspaces that are separate from yours. If you still want to reuse one of these connections, and you trust the compute resource and have access to it, choose that destination and then skip ahead to next steps. Otherwise, continue with the following steps.

2. If you are not on the Replications page, click the Arcion Cloud logo.

3. On the Replications page, click New.

4. For the General step, on the Set up a replication page, enter a Name and an optional Description.

5. Choose a Replication mode and a Write mode.

6. Click Next.

7. For the Source step, do one of the following:

   • If a Source page appears, do one of the following:

     • If your target source is not listed:

       1. Click Create New.

       2. Click a Connection Type, and then click Continue.

       3. Continue on with the Configure New Connection page steps.

     • If your target source already exists and appears in the list of available sources, do the following:

       1. Click the source.

       2. Click the ellipses icon inside of the selected source and then click Edit.

       3. Continue on with the Configure New Connection page steps.

   • If a Configure New Connection page appears, do the following:

     1. Enter the connection details, if they are not already displayed.

     2. Click Continue.

     3. Click Sync Connector.

     4. After the sync succeeds, click Continue.

8. To specify any non-default extractor settings for this source, do the following:

   1. Select the source, if it is not already selected.

   2. To use an existing extractor, select the Use an extractor check box, and then click the extractor.

   3. To create a new extractor, click New Extractor, specify the extractor’s settings, and then click Save.

9. Select the source if it is not already selected, and then click Test Connection.

10. After the connection succeeds, click Continue to Destination.

11. For the Destination step, select the tile with the Databricks logo and the name of your SQL warehouse. By default, the name of the SQL warehouse is ARCION_ENDPOINT.

12. To specify any non-default applier settings for this destination, do the following:

    1. Select the destination, if it is not already selected.

    2. To use an existing applier, select the Use an applier check box, and then click the applier.

    3. To create a new applier, click New Applier, specify the applier’s settings, and then click Save & Back.

13. Select the destination if it is not already selected, and then click Test Connection.

    Note

    The connect fails if the SQL warehouse named ARCION_ENDPOINT is not running. See Manage SQL warehouses.

14. After the connection succeeds, click Continue to Filter.

15. For the Filter step, select the schemas, tables, and columns that you want to the destination., and then click Save.

16. To specify any non-default mapper settings, click Map Tables, specify the settings, and then click Save.

17. To specify any non-default per table configs, click Per Table Configs, specify the settings, and then click Save.

18. Click Start Replication.

19. Skip ahead to next steps.

Connect to Arcion manually

This section describes how to connect to Arcion manually.

Note

To connect a new SQL warehouse with Arcion faster, use Partner Connect.

Requirements

Before you connect to Arcion, you need the following:

• A Databricks personal access token.

  Note

  As a security best practice when you authenticate with automated tools, systems, scripts, and apps, Databricks recommends that you use OAuth tokens or personal access tokens belonging to service principals instead of workspace users. To create tokens for service principals, see Manage tokens for a service principal.

• A SQL warehouse.

  • To create a SQL warehouse in your workspace, see Configure SQL warehouses.

• Connection details for your SQL warehouse. See Get connection details for a SQL warehouse. Specifically, you need the SQL warehouse’s Server Hostname, Port, and HTTP Path field values.

  Tip

  If the Arcion tile in Partner Connect in your workspace has a check mark icon inside of it, you can get the connection details for the connected SQL warehouse by clicking the tile and then expanding Connection details. Note however that the Personal access token here is hidden; you must create a replacement personal access token and enter that new token instead when Arcion asks you for it.

Steps to connect

To connect to Arcion manually, do the following:

1. Create a new Arcion account or to sign in to your existing Arcion account.

2. If you are not on the Replications page, click the Arcion Cloud logo.

3. On the Replications page, click New.

   Important

   If you sign in to your organization’s Arcion account, on the Replications page, one or more existing replication or connection entries with the Databricks logo might be listed. These entries might contain connection details for compute resources in workspaces that are separate from yours. If you still want to reuse one of these connections, and you trust the compute resource and have access to it, choose that destination and then skip ahead to next steps. Otherwise, continue with the following steps.

4. For the General step, on the Set up a replication page, enter a Name and an optional Description.

5. Choose a Replication mode and a Write mode.

6. Click Next.

7. For the Source step, do one of the following:

   • If a Source page appears, do one of the following:

     • If your target source is not listed:

       1. Click Create New.

       2. Click a Connection Type, and then click Continue.

       3. Continue on with the Configure New Connection page steps.

     • If your target source already exists and appears in the list of available sources, do the following:

       1. Click the source.

       2. Click the ellipses icon inside of the selected source and then click Edit.

       3. Continue on with the Configure New Connection page steps.

   • If a Configure New Connection page appears, do the following:

     1. Enter the connection details, if they are not already displayed.

     2. Click Continue.

     3. Click Sync Connector.

     4. After the sync succeeds, click Continue.

8. To specify any non-default extractor settings for this source, do the following:

   1. Select the source if it is not already selected.

   2. To use an existing extractor, select the Use an extractor check box, and then click the extractor.

   3. To create a new extractor, click New Extractor, specify the extractor’s settings, and then click Save.

9. Select the source if it is not already selected, and then click Test Connection.

10. After the connection succeeds, click Continue to Destination.

11. For the Destination step, do one of the following:

    • If your target destination already exists and appears in the list of available sources, select it.

    • If your target destination is not listed, do the following:

      1. Click Databricks, and then click Continue.

      2. For Connection Name, enter a name for this connection.

      3. For JDBC URL, enter the JDBC URL value that you got earlier.

      4. For Host, enter the Server Hostname value that you got earlier.

      5. For Port, enter the Port value that you got earlier.

      6. For Username, enter the word token.

      7. For Password, enter your token that you got earlier.

      8. Click Test Connection.

      9. After the connection succeeds, click Save Connection.

      10. Click Back.

12. To specify any non-default applier settings for this destination, do the following:

    1. Select the destination, if it is not already selected.

    2. To use an existing applier, select the Use an applier check box, and then click the applier.

    3. To create a new applier, click New Applier, specify the applier’s settings, and then click Save & Back.

13. Select the destination if it is not already selected, and then click Test Connection.

    Note

    The connect fails if the Databricks compute resource is not running. See Manage SQL warehouses or Restart a cluster.

14. After the connection succeeds, click Continue to Filter.

15. For the Filter step, specify any non-default filter settings, and then click Save.

16. To specify any non-default mapper settings, click Map Tables, specify the settings, and then click Save.

17. To specify any non-default per table configs, click Per Table Configs, specify the settings, and then click Save.

18. Click Start Replication.

19. Continue to next steps.

Next steps

Explore one or more of the following resources on the Arcion website:

• Quickstart

• Configuring Source

• Configuring Destination

• Arcion Documentation

• Contact support