Renomear uma tabela ingerida
Aplica-se a : Criação de pipeline baseado em API
Por default, as tabelas de destino criadas pelo pipeline de ingestão gerenciar LakeFlow Connect têm o mesmo nome das tabelas de origem. No entanto, você pode, opcionalmente, renomear uma tabela ingerida. Por exemplo, se você ingerir um objeto em duas tabelas de destino no mesmo esquema, precisará renomear uma das tabelas de destino para diferenciá-las (não há suporte para duplicatas). Se o senhor renomear uma tabela no pipeline, ela se tornará um pipeline somente de API e não será mais possível editar o pipeline na interface do usuário.
Exemplo: Google analítica
- Databricks Asset Bundles
- Databricks notebook
- Databricks CLI
YAML
resources:
pipelines:
pipeline_ga4:
name: <pipeline-name>
catalog: <target-catalog> # Location of the pipeline event log
schema: <target-schema> # Location of the pipeline event log
ingestion_definition:
connection_name: <connection>
objects:
- table:
source_url: <project-id>
source_schema: <property-name>
destination_catalog: <target-catalog>
destination_schema: <target-schema>
destination_table: <custom-target-table-name> # Table rename
Google analítica:
Python
pipeline_spec = """
{
"name": "<pipeline>",
"ingestion_definition": {
"connection_name": "<connection>",
"objects": [
{
"table": {
"source_catalog": "<project-id>",
"source_schema": "<property-name>",
"source_table": "<source-table>",
"destination_catalog": "<target-catalog>",
"destination_schema": "<target-schema>",
"destination_table": "<custom-target-table-name>",
}
}
]
}
}
"""
JSON
{
"resources": {
"pipelines": {
"pipeline_ga4": {
"name": "<pipeline>",
"catalog": "<target-catalog>",
"schema": "<target-schema>",
"ingestion_definition": {
"connection_name": "<connection>",
"objects": [
{
"table": {
"source_url": "<project-id>",
"source_schema": "<property-name>",
"destination_catalog": "<destination-catalog>",
"destination_schema": "<destination-schema>",
"destination_table": "<custom-destination-table-name>"
}
}
]
}
}
}
}
}
Exemplo: Salesforce
- Databricks Asset Bundles
- Databricks notebook
- Databricks CLI
YAML
resources:
pipelines:
pipeline_sfdc:
name: <pipeline-name>
catalog: <target-catalog> # Location of the pipeline event log
schema: <target-schema> # Location of the pipeline event log
ingestion_definition:
connection_name: <connection>
objects:
- table:
source_schema: <source-schema>
source_table: <source-table>
destination_catalog: <target-catalog>
destination_schema: <target-schema>
destination_table: <custom-target-table-name> # Table rename
Python
pipeline_spec = """
{
"name": "<pipeline>",
"ingestion_definition": {
"connection_name": "<connection>",
"objects": [
{
"table": {
"source_catalog": "<source-catalog>",
"source_schema": "<source-schema>",
"source_table": "<source-table>",
"destination_catalog": "<target-catalog>",
"destination_schema": "<target-schema>",
"destination_table": "<custom-target-table-name>",
}
}
]
}
}
"""
JSON
{
"resources": {
"pipelines": {
"pipeline_sfdc": {
"name": "<pipeline>",
"catalog": "<target-catalog>",
"schema": "<target-schema>",
"ingestion_definition": {
"connection_name": "<connection>",
"objects": [
{
"table": {
"source_schema": "<source-schema>",
"source_table": "<source-table>",
"destination_catalog": "<destination-catalog>",
"destination_schema": "<destination-schema>",
"destination_table": "<custom-destination-table-name>"
}
}
]
}
}
}
}
}
Exemplo: Dia de trabalho
- Databricks Asset Bundles
- Databricks notebook
- Databricks CLI
YAML
resources:
pipelines:
pipeline_workday:
name: <pipeline>
catalog: <target-catalog> # Location of the pipeline event log
schema: <target-schema> # Location of the pipeline event log
ingestion_definition:
connection_name: <connection>
objects:
- report:
source_url: <report-url>
destination_catalog: <target-catalog>
destination_schema: <target-schema>
destination_table: <custom-target-table-name> # Table rename
Python
pipeline_spec = """
{
"name": "<pipeline>",
"ingestion_definition": {
"connection_name": "<connection>",
"objects": [
{
"report": {
"source_url": "<report-url>",
"destination_catalog": "<target-catalog>",
"destination_schema": "<target-schema>",
"destination_table": "<custom-target-table-name>",
}
}
]
}
}
"""
JSON
{
"resources": {
"pipelines": {
"pipeline_workday": {
"name": "<pipeline>",
"catalog": "<target-catalog>",
"schema": "<target-schema>",
"ingestion_definition": {
"connection_name": "<connection>",
"objects": [
{
"report": {
"source_url": "<report-url>",
"destination_catalog": "<destination-catalog>",
"destination_schema": "<destination-schema>",
"destination_table": "<custom-destination-table-name>"
}
}
]
}
}
}
}
}