Aller au contenu

The Model Deployment

azure_helper.steps.deploy_aml_model

DeploymentSettings

Bases: BaseModel

Basic settings needed for the deployment, whether it is with ACI or AKS.

Parameters:

Name Type Description Default
deployment_service_name str

the name of the service you want to deploy or update.

 required
cpu_cores int

The number of cpu cores needed. Defaults to 1.

 required
gpu_cores int

The number of gpu cores needed. Defaults to 0.

 required
memory_gb int

The memory in gb needed. Defaults to 1.

 required
enable_app_insights bool

Enable app insights monitoring. Defaults to True.

 required
Source code in azure_helper/steps/deploy_aml_model.py 
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
class DeploymentSettings(BaseModel):
    """Basic settings needed for the deployment, whether it is with ACI or AKS.

    Args:
        deployment_service_name (str): the name of the service you want to deploy or update.
        cpu_cores (int): The number of cpu cores needed. Defaults to 1.
        gpu_cores (int): The number of gpu cores needed. Defaults to 0.
        memory_gb (int): The memory in gb needed. Defaults to 1.
        enable_app_insights (bool): Enable app insights monitoring. Defaults to True.
    """

    deployment_service_name: str
    cpu_cores: int = 1
    gpu_cores: int = 0
    memory_gb: int = 1
    enable_app_insights: bool = True

DeployModel

Source code in azure_helper/steps/deploy_aml_model.py 
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
class DeployModel:
    def __init__(
        self,
        aml_interface: AMLInterface,
        aml_env_name: str,
        model_name: str,
        script_config_path: Path,
        deployment_settings: DeploymentSettings,
    ) -> None:
        """Instantiate the deployment of the model.

        Args:
            aml_interface (AMLInterface):  The AMLInterface which will be responsible to deploy the model.
            aml_env_name (str): The name of the AMLEnvironment you will use to deploy the model. It is not necessarily
                the same used to train the model.
            model_name (str): The name of the model you deploy.
            script_config_path (Path): The location of the inference script.
            deployment_settings (DeploymentSettings): the basic settings of the deployment.
        """

        self.aml_interface = aml_interface
        self.workspace = aml_interface.workspace

        self.aml_env_name = aml_env_name
        self.model_name = model_name
        self.script_config_path = script_config_path
        self.deployment_settings = deployment_settings

    def get_inference_config(
        self,
    ) -> InferenceConfig:
        """Fetch the inference script config needed to interact the endpoint deployed.

        Returns:
            InferenceConfig: The instantiated inference config.
        """

        aml_env = Environment.get(
            workspace=self.workspace,
            name=self.aml_env_name,
        )
        # scoring_script_path = os.path.join(__here__, "score.py")
        scoring_script_path = str(self.script_config_path)
        return InferenceConfig(
            entry_script=scoring_script_path,
            environment=aml_env,
        )

    def deploy_aciservice(
        self,
        *args,
        **kwargs,
    ):
        """Deploy an ACI service to serve the model."""
        inference_config = self.get_inference_config()

        aci_deployment = AciWebservice.deploy_configuration(
            *args,
            **kwargs,
            cpu_cores=self.deployment_settings.cpu_cores,
            memory_gb=self.deployment_settings.memory_gb,
            enable_app_insights=self.deployment_settings.enable_app_insights,
        )

        model = self.workspace.models.get(self.model_name)

        service = Model.deploy(
            workspace=self.workspace,
            name=self.deployment_settings.deployment_service_name,
            models=[model],
            inference_config=inference_config,
            deployment_config=aci_deployment,
        )

        service.wait_for_deployment(show_output=True)
        log.info(service.state)
        log.info(service.scoring_uri)

    def deploy_aksservice(
        self,
        aks_cluster_name: str,
        *args,
        **kwargs,
    ):
        """Deploy an AKS service to serve the model.

        Args:
            script_config_path (Path): The location of the script for the inference config.
            aks_cluster_name (str): The name of the k8s cluster on which you want to deploy. Contrary to an ACI deployment,
                you need a pre-existing k8s cluster in your workspace to use AKS deployment.
        """
        # Verify that cluster does not exist already
        try:
            aks_target = ComputeTarget(self.workspace, name=aks_cluster_name)
            log.info(
                f"k8s cluster {aks_cluster_name} found in workspace {self.workspace}",
            )
        except ComputeTargetException:
            log.warning(
                f"k8s cluster {aks_cluster_name} was not found in workspace {self.workspace}. Now provisioning one.",
            )
            # Use the default configuration (can also provide parameters to customize)
            provisioning_config = AksCompute.provisioning_configuration()

            # Create the cluster
            aks_target = ComputeTarget.create(
                workspace=self.workspace,
                name=aks_cluster_name,
                provisioning_configuration=provisioning_config,
            )
            aks_target.wait_for_completion(
                show_output=True,
                timeout_in_minutes=10,
            )

        inference_config = self.get_inference_config()

        aks_deployment = AksWebservice.deploy_configuration(
            *args,
            **kwargs,
            cpu_cores=self.deployment_settings.cpu_cores,
            memory_gb=self.deployment_settings.memory_gb,
            enable_app_insights=self.deployment_settings.enable_app_insights,
        )

        model = self.workspace.models.get(self.model_name)

        service = Model.deploy(
            workspace=self.workspace,
            name=self.deployment_settings.deployment_service_name,
            models=[model],
            inference_config=inference_config,
            deployment_config=aks_deployment,
            deployment_target=aks_target,
        )

        service.wait_for_deployment(show_output=True)
        log.info(service.state)
        log.info(service.scoring_uri)

    def update_service(
        self,
    ):
        """Update an already existing service, ACI or AKS."""
        inference_config = self.get_inference_config()
        service = Webservice(
            name=self.deployment_settings.deployment_service_name,
            workspace=self.workspace,
        )
        model = self.workspace.models.get(self.model_name)
        service.update(models=[model], inference_config=inference_config)
        log.info(service.state)
        log.info(service.scoring_uri)

__init__(aml_interface, aml_env_name, model_name, script_config_path, deployment_settings)

Instantiate the deployment of the model.

Parameters:

Name Type Description Default
aml_interface AMLInterface

The AMLInterface which will be responsible to deploy the model.

 required
aml_env_name str

The name of the AMLEnvironment you will use to deploy the model. It is not necessarily the same used to train the model.

 required
model_name str

The name of the model you deploy.

 required
script_config_path Path

The location of the inference script.

 required
deployment_settings DeploymentSettings

the basic settings of the deployment.

 required
Source code in azure_helper/steps/deploy_aml_model.py 
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
def __init__(
    self,
    aml_interface: AMLInterface,
    aml_env_name: str,
    model_name: str,
    script_config_path: Path,
    deployment_settings: DeploymentSettings,
) -> None:
    """Instantiate the deployment of the model.

    Args:
        aml_interface (AMLInterface):  The AMLInterface which will be responsible to deploy the model.
        aml_env_name (str): The name of the AMLEnvironment you will use to deploy the model. It is not necessarily
            the same used to train the model.
        model_name (str): The name of the model you deploy.
        script_config_path (Path): The location of the inference script.
        deployment_settings (DeploymentSettings): the basic settings of the deployment.
    """

    self.aml_interface = aml_interface
    self.workspace = aml_interface.workspace

    self.aml_env_name = aml_env_name
    self.model_name = model_name
    self.script_config_path = script_config_path
    self.deployment_settings = deployment_settings

get_inference_config()

Fetch the inference script config needed to interact the endpoint deployed.

Returns:

Name Type Description
InferenceConfig InferenceConfig

The instantiated inference config.
Source code in azure_helper/steps/deploy_aml_model.py 
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
def get_inference_config(
    self,
) -> InferenceConfig:
    """Fetch the inference script config needed to interact the endpoint deployed.

    Returns:
        InferenceConfig: The instantiated inference config.
    """

    aml_env = Environment.get(
        workspace=self.workspace,
        name=self.aml_env_name,
    )
    # scoring_script_path = os.path.join(__here__, "score.py")
    scoring_script_path = str(self.script_config_path)
    return InferenceConfig(
        entry_script=scoring_script_path,
        environment=aml_env,
    )

deploy_aciservice(*args, **kwargs)

Deploy an ACI service to serve the model.

Source code in azure_helper/steps/deploy_aml_model.py 
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
def deploy_aciservice(
    self,
    *args,
    **kwargs,
):
    """Deploy an ACI service to serve the model."""
    inference_config = self.get_inference_config()

    aci_deployment = AciWebservice.deploy_configuration(
        *args,
        **kwargs,
        cpu_cores=self.deployment_settings.cpu_cores,
        memory_gb=self.deployment_settings.memory_gb,
        enable_app_insights=self.deployment_settings.enable_app_insights,
    )

    model = self.workspace.models.get(self.model_name)

    service = Model.deploy(
        workspace=self.workspace,
        name=self.deployment_settings.deployment_service_name,
        models=[model],
        inference_config=inference_config,
        deployment_config=aci_deployment,
    )

    service.wait_for_deployment(show_output=True)
    log.info(service.state)
    log.info(service.scoring_uri)

deploy_aksservice(aks_cluster_name, *args, **kwargs)

Deploy an AKS service to serve the model.

Parameters:

Name Type Description Default
script_config_path Path

The location of the script for the inference config.

 required
aks_cluster_name str

The name of the k8s cluster on which you want to deploy. Contrary to an ACI deployment, you need a pre-existing k8s cluster in your workspace to use AKS deployment.

 required
Source code in azure_helper/steps/deploy_aml_model.py 
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
def deploy_aksservice(
    self,
    aks_cluster_name: str,
    *args,
    **kwargs,
):
    """Deploy an AKS service to serve the model.

    Args:
        script_config_path (Path): The location of the script for the inference config.
        aks_cluster_name (str): The name of the k8s cluster on which you want to deploy. Contrary to an ACI deployment,
            you need a pre-existing k8s cluster in your workspace to use AKS deployment.
    """
    # Verify that cluster does not exist already
    try:
        aks_target = ComputeTarget(self.workspace, name=aks_cluster_name)
        log.info(
            f"k8s cluster {aks_cluster_name} found in workspace {self.workspace}",
        )
    except ComputeTargetException:
        log.warning(
            f"k8s cluster {aks_cluster_name} was not found in workspace {self.workspace}. Now provisioning one.",
        )
        # Use the default configuration (can also provide parameters to customize)
        provisioning_config = AksCompute.provisioning_configuration()

        # Create the cluster
        aks_target = ComputeTarget.create(
            workspace=self.workspace,
            name=aks_cluster_name,
            provisioning_configuration=provisioning_config,
        )
        aks_target.wait_for_completion(
            show_output=True,
            timeout_in_minutes=10,
        )

    inference_config = self.get_inference_config()

    aks_deployment = AksWebservice.deploy_configuration(
        *args,
        **kwargs,
        cpu_cores=self.deployment_settings.cpu_cores,
        memory_gb=self.deployment_settings.memory_gb,
        enable_app_insights=self.deployment_settings.enable_app_insights,
    )

    model = self.workspace.models.get(self.model_name)

    service = Model.deploy(
        workspace=self.workspace,
        name=self.deployment_settings.deployment_service_name,
        models=[model],
        inference_config=inference_config,
        deployment_config=aks_deployment,
        deployment_target=aks_target,
    )

    service.wait_for_deployment(show_output=True)
    log.info(service.state)
    log.info(service.scoring_uri)

update_service()

Update an already existing service, ACI or AKS.

Source code in azure_helper/steps/deploy_aml_model.py 
177
178
179
180
181
182
183
184
185
186
187
188
189
def update_service(
    self,
):
    """Update an already existing service, ACI or AKS."""
    inference_config = self.get_inference_config()
    service = Webservice(
        name=self.deployment_settings.deployment_service_name,
        workspace=self.workspace,
    )
    model = self.workspace.models.get(self.model_name)
    service.update(models=[model], inference_config=inference_config)
    log.info(service.state)
    log.info(service.scoring_uri)

Deploy to AKS

When deploying to Azure Kubernetes Service, key-based authentication is enabled by default. You can also enable token-based authentication. Token-based authentication requires clients to use an Azure Active Directory account to request an authentication token, which is used to make requests to the deployed service.

Limitations

  • Using a service principal with AKS is not supported by Azure Machine Learning. The AKS cluster must use a system-assigned managed identity instead.

Tutorial

Prepare an application for Azure Kubernetes Service (AKS)

Create a Kubernetes cluster

1
2
3
4
5
6
az aks create \
    --resource-group myResourceGroup \
    --name myAKSCluster \
    --node-count 2 \
    --generate-ssh-keys \
    --attach-acr <acrName>

Install kubectl

1
az aks install-cli

Connect to cluster using kubectl

1
az aks get-credentials --resource-group myResourceGroup --name myAKSCluster

Update the manifest file

In these tutorials, an Azure Container Registry (ACR) instance stores the container image for the sample application. To deploy the application, you must update the image name in the Kubernetes manifest file to include the ACR login server name.

Deploy the application

To deploy your application, use the kubectl apply command.

1
kubectl apply -f manifest.yaml