TiDB

The basic requirements for data integration are the same as data migration, refer to Data Migration Instructions for more details.

Migration Content

Data integration currently only migrates Database, Table, and their data, and does not migrate other content such as Views.

Feature Description

Execution Order

• The data sources are sorted according to the order they are added, with tasks for earlier ones executed with higher priority.
• When the task type for a data source is a full task, subsequent tasks will only start after the full task has been completed.

Retaining Original Data

In the data source configuration, there is an option to retain the original data in the target database, which defaults to “Yes”.

• If you keep the default “Yes” selection, new data will merely be added continuously to the target database. In case of conflicts, it will execute according to the set conflict resolution strategy.
• If “No” is selected, the corresponding databases and tables in the destination will be deleted before the task starts, as per the ETL settings.

Data Conflict Resolution Strategy

In the data source configuration, there is an option for the “Data Conflict Resolution Method” to establish how to handle conflicts when data collisions occur.

• If you choose to replace, it will replace the existing data using the “replace into” statement.
• If you choose to retain, it will keep the existing data and ignore the new data.

ETL

• When filling in ETL information, if a database or table is checked, a mapping name must be provided. If no mapping is needed, by default, use the same database/table name as the data source.
• When filling in ETL information, if a database is checked without selecting a table, all tables under that database will be migrated.

Functional Limitations

• Data integration tasks currently do not support DDL statements, as DDL statements may alter the structure of the tables in the target database, compromising the ETL functionality.