Azure Automated Machine Learning Service | 2023
→ Welcome family to the new service which I am currently using and it should be worth your time.
→ If you are working on a project that uses ML features in your application or is just curious about Azure ML service, then this is for you.
→ Being a JS specialist, trying to integrate various areas of IT in applications.
Why use this service?
→ Automatically build machine learning models with speed and scale.
→ Model Creation requires lots of learning of core ML which data scientist and their team do.
→ If you are a developer and want to use ML hassle-free in apps, Automated ML service is for you.
Why is this service in demand?
→ If you are working on a project where lots of scanned documents are there.
→ Manual use of data say more than 1000 documents may lead to errors or requires lots of time and effort.
→ Using Azure’s Automated ML — IT people with non-ML knowledge can do it rapidly.
Who uses this service?
Azure Demo -Automated ML:
→ Before starting there are two things we want you to know:
a. Automated ML — Here we can create custom models with minimum ML expertise.
b. Azure ML Designer — Enables no-code development and allows graph drag-drop to create an ML pipeline for using Automated ML.
→ Let us now go to Azure Machine Learning. Open Azure Portal.
→ If you are not having an Azure cloud subscription, would suggest you follow along with the screenshots below or signup for a free Azure subscription or watch the video.
→ Anyways team, this should be sufficient because have done it yesterday with the latest June 2023 view.
→ Now inside the ML portal, the first thing we do is create a workspace (say ml-happylearnings786).
→It is good practice to create environment specific (say production: ml-happylearnings786, dev: dev-ml-happylearnings786, etc.)
NOTE:
Creating a Resource group is important. All apps using related to this environment will be here.
Suppose later we do not need dev environment Azure services as moved to production, deleting this resource group is enough to inactive all services for that environment and not include in Azure Bill 🙂
→ Click Create New, and enter the resource group name (say machine-learning-rg). Click the OK button.
→ The next step is to name the workspace — should be something related to the project and environment.
→ Click Review + Create, followed by some buttons as shown below:
→ Click Create, once validation is passed
→ Click on Go to resource, when deployment is complete.
→ We are now inside ML Workspace.
Automated Machine Learning — Dataset and Compute:
→ We have completed our first task — Creating ML Workspace ✅
→ Now there are several pieces of this workspace — we need to know:
a. Compute — for processing it we need to create one compute
b. Dataset — data we will feed into ML
→ Need not worry, we will see in real-time soon.
→ They will be used all in combined
→ If you are on home page of Azure, search Machine Learning
→ Here we will see our recently created ML. Click it.
→ Click on Launch Studio
→ We can skip the tour. Click Cancel button for now.
→ Now the landing page of ML studio will have:
STEP 1: Create Compute
→ Now the first step is to create Compute before using anything related to ML studio
→ There are four different types of compute types:
a. Compute instances
b. Kubernetes clusters
c. Attached computes
b. Compute clusters: We will be using Compute clusters as we can ass multiple instances. Click + New button
→ In the popup screen give a name to the popup screen (say ml-compute). We can change the default settings and click Next.
→ We just make idle time 900 secs before — Basically this means, compute will shut down after 900 seconds of idle time. Click Create button
→ We have successfully done our second piece: Created Compute ✅
STEP 2: Create Dataset/Data
→ The second step is to create Dataset.
→ To use ML, we need data. Click on Data from the left explorer.
→ Click + Create button, as done in the above screen.
→ Enter the name of the data (say ml_data)
→ Now there are various ways to feed data to ML — local file import, from storage account, or sample Azure Open datasets
→ Data is something i.e. very sensitive and private to the business. If you get a chance to use ML in app development, they will give us data in one of the above ways.
→ For now Azure has some ready datasets for learners i.e. Open AI Sample Data.
→ Chose Type as Tabular from Type dropdown.
→ Click the Next button.
→ Below are the various ways to feed data into Azure ML. Select From Azure Open Datasets and click Next.
→ Here Azure team has great job with some ready sample data sets. Let us select a sample diabetic.
→ These are sample data that may be selected. Let chose Sample: Diabetes, and click Next.
→ Review the page and click Create
→ Click on Explore Tab to see the data
→ We can also go to the profile section and see the metric.
Eg1: Age Column
So reviewing one metric — we have lots of data from the age range of 25–60
Eg 2: BMI column
→ So we have to check each column and make sure we have appropriate data. We also see different stats — eg: Min, Max, missing count, etc.
We have 11 columns out of which:
10 — features (input)
1 — prediction (output)
Another eg. to make simple is —
If we want to buy a car:
a. Car model, Car Company, specifications of car parts — features
b. Price — label
→ Finally our third piece is ready- data i.e. Dataset/Data, Compute, ML Workspace ✅
STEP 3: Create an ML Model
→ Click on Automated ML from the left explorer.
→ We have to create a new Automated ML job. Click the below button.
→ We have already created a dataset previously. It will come here. Select it. Click Next.
NOTE:
Experiment: The process of creating model in Azure ML is called Experiment.
→ We have to enter the name (say automated-ml-diabetes), the name given based on the data name.
→ In the next dropdown we have to choose a target column (i.e. labels or output)
NOTE: One column will be chosen as label and all others are by default chosen as features (i.e. input)
Click the Target column — all columns from the data set will pop up here. Chose Y as the label (o/p)
→ Click the Target column — all columns from the data set will pop up here. Chose Y as a label
Diabetes dataset Link:
→ Now in the next step we have to create compute — i.e. cluster compute (i.e. are piece 2 task ie. ml-compute)
→ We notice our computer is rightly selected. Click the Next button.
→ Now in this step we have to choose the task and type:
Classification: o/p — Boolean
Regression: o/p — Integers
→ Our case output is data from the table i.e. numeric, hence selecting regression and scrolling down
→ Now there are two extra settings we have to do:
a. View additional config:
→ Click the below link on the same page.
→ Azure asks for some metrics (this is for regression case)
Default: Normalized root mean squared error
→ We will go with default only i.e. Normalized root mean square error
→ In the end when we will run the ML pipeline, we will choose Regression as model
→ Output of that model will be based on these metrics.
→ Azure chooses the best models. For our case, we will choose the first four models and block other models by checking them
→ Reason: Training will take a longer time
→ In the same popup, there is one config called Exit Criterion
→ Exit criterion: We give time 0.5 hours to stop the run.
b. View featurization settings:
→ One we select as target column becomes — Label (our case Y)
→ All other default features i.e. input
→ We can customize features from here
→ Suppose we can skip some columns as features, set default value using Impute with, etc.
→ We will choose default only, let us click the cancel button (i.e. other than Y, all are features, Y= label = target column).
→ Click Next
→ Now chose the default setting i.e. Auto and click the Finish button.
→ It now takes some time to Run and Finish.
→ Finally completed in around 37 mins.
Exploring Automated ML Training:
→ Click on Automated ML from the left explorer.
→ Click on the Run instance created i.e. sweet_feather_m12y032j
→ We see the best model chosen — Algorithm name VotingEnsemble.
→ We see the status of the run here:
→ Now let us see a number of models created during this process:
→ These models are created and evaluated based on a specific metric
i.e. Normalized root mean squared metric
Deploy Model:
→ In our case we got VitongEnsemble as the best model from all the created models.
→ Select the best model suggested by Azure and click the Deploy button.
→ In the 2023 interface, two options are now asked:
a. Realtime endpoint — hourly charge is asked (newly introduced)
b. Webservice — standard charge is asked
→ We will go with the Web service option as it has standard pricing.
→ Give a name to the model deploy say. automate-ml-votingensemble
→ Now compute type can be AKS or Azure Container Instance. Click Deploy
→ Now model deployment is triggered and in process, we also get notifications from Microsoft.
→ We can also go to Endpoints from the left and check
→ Model deployment — in progress
→ We receive notification of deployment success
→ We can even test documents by sending data:
{
"Inputs": {
"data": [
{
"AGE": 48,
"SEX": 1,
"BMI": 21.6,
"BP": 87,
"S1": 183,
"S2": 103.2,
"S3": 70,
"S4": 3,
"S5": 3.8918,
"S6": 69
}
]
},
"GlobalParameters": 0.0
}→ These are columns (i.e. features) we provided in the dataset.
→ We can now use this model by passing data. Click Test button
→ Now suppose we consume this program-wise, we have a REST endpoint for the same. Mainly ML uses — Python, C#, or R language.
Eg. Python script
import urllib.request
import json
import os
import ssl
def allowSelfSignedHttps(allowed):
# bypass the server certificate verification on client side
if allowed and not os.environ.get('PYTHONHTTPSVERIFY', '') and getattr(ssl, '_create_unverified_context', None):
ssl._create_default_https_context = ssl._create_unverified_context
allowSelfSignedHttps(True) # this line is needed if you use self-signed certificate in your scoring service.
# Request data goes here
# The example below assumes JSON formatting which may be updated
# depending on the format your endpoint expects.
# More information can be found here:
# https://docs.microsoft.com/azure/machine-learning/how-to-deploy-advanced-entry-script
data = {
"Inputs": {
"data": [
{
"AGE": 0,
"SEX": 0,
"BMI": 0.0,
"BP": 0.0,
"S1": 0,
"S2": 0.0,
"S3": 0.0,
"S4": 0.0,
"S5": 0.0,
"S6": 0
}
]
},
"GlobalParameters": 0.0
}
body = str.encode(json.dumps(data))
url = 'http://739b7ece-b25c-4bc4-8ec6-25588f476693.koreacentral.azurecontainer.io/score'
headers = {'Content-Type':'application/json'}
req = urllib.request.Request(url, body, headers)
try:
response = urllib.request.urlopen(req)
result = response.read()
print(result)
except urllib.error.HTTPError as error:
print("The request failed with status code: " + str(error.code))
# Print the headers - they include the requert ID and the timestamp, which are useful for debugging the failure
print(error.info())
print(error.read().decode("utf8", 'ignore'))→ Azure also provides R and C#. These three tabs will be provided.
→ Our Azure Automate ML is ready to be used by the Designer ✅
Closing Thoughts:
In this current article, we have understood various areas of Azure Automate ML service — i.e. creating ML workspace, compute, dataset, ML model — choosing the right metrics, created Automated ML job.
Overall we understood every area of Automate ML service. Now in the next article, we will implement it in the Azure Designer section to create an ML pipeline.
These are important and cool features of Azure AI. Once use in development it becomes exciting.
Thank you for reading till the end 🙌 . If you enjoyed this article or learned something new, support me by clicking the share button below to reach more people and/or give me a follow on Twitter and subscribe Happy Learnings !! to see some other tips, articles, and things I learn about and share there.
