Step-by-Step Guide to Integrating Azure DataFactory with MySQL Flexible Server

Connecting Azure DataFactory to a MySQL Flexible Server can be complex, but understanding the right steps based on your server’s installation method simplifies the process. Here's a straightforward guide that will help you set up and connect Azure DataFactory depending on the chosen connectivity method of your MySQL Flexible Server.

Scenario A: Public Access (Allowed IP Addresses)

In this installation mode, external (public) access to the server can be disabled, and a "private endpoint" must be created in order to access the MySQL server on the virtual network.
It is necessary to configure a "Virtual network enabled" integration runtime and a "Managed private endpoint" on the Data Factory side. This "Managed private endpoint" can be used to create a "linked service" that reaches the MySQL flexible server.

Here are the steps to connect using this method:

Step 1: Set Up a Managed Private Endpoint

Establish a managed private endpoint to securely link your DataFactory with the MySQL server:

1. Go to your DataFactory instance in the Azure portal.
2. In DataFactory, under the "Manage" tab, find and select "Managed private endpoints."
3. Create a new endpoint that points to your MySQL Flexible Server through the virtual network.

Step 2: Configure Virtual Network Integration Runtime

You need a virtual network-enabled integration runtime within Azure DataFactory:

1. Go to your DataFactory instance in the Azure portal.
2. In DataFactory, under the "Manage" tab, find and select "Integration runtimes"
3. Select "Virtual network enabled" from the options.

Step 3: Create a Linked Service

Finally, use the managed private endpoint to create a linked service:

1. Still in DataFactory, under the "Manage" tab.
2. Select "Linked services" and then "New."
3. Set up the linked service to connect to your MySQL server using the private endpoint as the connection method.

Step 4: Test the Connection

Finally, test your configuration to ensure that everything is set up correctly:

1. From the linked service configuration, select the option to test the connection.
2. Verify that the connection to your MySQL Flexible Server is successful.

Scenario B: Private Access (VNet Integration)

"In this installation mode, for Data Factory access to function properly, a continuously running Windows virtual machine (on the same virtual network as the MySQL server) is required, on which a 'self-hosted integration runtime' must be installed and configured.

A 'linked service' can then be created, which connects to the MySQL flexible server using the 'self-hosted integration runtime' running on the VM.

Follow these steps:

Step 1: Prepare a Virtual Machine

Ensure you have a Windows virtual machine running continuously in the same virtual network as your MySQL server:

1. In Azure, either find an existing VM or set up a new one in the same virtual network as your MySQL Flexible Server.

Step 2: Install Self-Hosted Integration Runtime

Install the self-hosted integration runtime on your virtual machine:

1. Download and install the integration runtime software from the Azure DataFactory portal.
2. Follow the setup wizard to configure it, ensuring it connects to the same virtual network.

Step 3: Create a Linked Service

Establish a linked service in DataFactory that uses the self-hosted integration runtime:

1. Still in DataFactory, under the "Manage" tab.
2. Select "Linked services" and then "New."
3. Choose to set up the linked service using the self-hosted integration runtime from your virtual machine.

Step 4: Test the Connection

Finally, test your configuration to ensure that everything is set up correctly:

1. From the linked service configuration, select the option to test the connection.
2. Verify that the connection to your MySQL Flexible Server is successful.

By following these detailed steps, you can effectively connect Azure DataFactory to a MySQL Flexible Server, regardless of the initial connectivity method selected during the server’s setup. Remember, the connectivity method cannot be changed after installation, so it’s crucial to choose the right scenario from the start based on your accessibility and security requirements.

‍