Databricks-Machine-Learning-Professional Databricks Certified Machine Learning Professional Questions and Answers

A model signature is a specification of the input and output schema of a model, which defines the names, data types, and shapes of the model inputs and outputs. Logging a model signature with an MLflow model has several benefits, such as:

• The signature can be used to document what type of data is expected as input and output by the model, which can help users and developers understand how to use the model correctly.
• The signature can be used to enforce schema validation when serving models via REST API, which can help prevent errors and ensure data quality. For example, Azure Machine Learning enforces compliance with the model signature for MLflow models that contain a signature1.
• The signature can be used to enable automatic conversion of input data to the appropriate format for the model, such as pandas DataFrame, numpy array, or PyTorch tensor. This can simplify the data preprocessing and inference steps and reduce the code complexity.

References:

• MLflow Models Documentation, Model Signature and Input Example, p. 6-7
• mlflow.models.signature Documentation, p. 1-2
• Guidelines for deploying MLflow models - Azure Machine Learning, p. 1

To enable MLflow Autologging for all supported machine learning libraries in a notebook, you can use the mlflow.autolog() function. This function enables autologging for each library by invoking the corresponding autologging function, such as mlflow.sklearn.autolog() or mlflow.spark.autolog(). This way, you do not need to call each autologging function separately. The mlflow.autolog() function also takes care of the compatibility and configuration issues that may arise from different versions of the Databricks Runtime for Machine Learning or workspace-wide settings in the Admin Console. The other options are incorrect because:

• Option A: mlflow.sklearn.autolog() will only enable autologging for scikit-learn, not for other libraries.
• Option B: mlflow.spark.autolog() will only enable autologging for Spark MLlib, not for other libraries.
• Option C: spark.conf.set(“autologging”, True) is not a valid way to enable autologging, as there is no such configuration option in Spark.
• Option D: It is possible to automatically log MLflow runs using the autologging functions provided by MLflow. References: Automatic Logging with MLflow Tracking, MLflow quickstart (Python)

To view the data visualizations that are logged with each run, you can go to the Artifacts section of the MLflow Run page in Databricks. The Artifacts section shows the files and directories that are logged as artifacts for a run. You can browse the artifact hierarchy and preview the files, such as images, text, or HTML1. You can also download the artifacts or copy their URIs for further use2. The other options are incorrect because:

• Option A: The MLflow Model Registry Model page shows the information and metadata of a registered model, such as its name, description, versions, stages, and lineage. It does not show the data visualizations that are logged with each run3.
• Option B: The Artifacts section of the MLflow Experiment page shows the artifacts that are logged for an experiment, not for a specific run. It does not allow you to preview the files or browse the artifact hierarchy4.
• Option C: Logged data visualizations can be viewed in Databricks using the Artifacts section of the MLflow Run page1.
• Option E: There is no Figures section of the MLflow Run page in Databricks. The Figures section is only available in the open source MLflow UI, which shows the plots that are logged as figures for a run5. References: View run artifacts, Log, list, and download artifacts, Manage models, View experiment artifacts, Logging Visualizations with MLflow

The correct way to log a scikit-learn model using MLflow is to use the mlflow.sklearn.log_model function, which logs the model as an MLflow artifact for the current run1. This function takes the scikit-learn model object and an artifact path as required arguments, and optionally accepts other arguments such as conda environment, code paths, serialization format, etc2. The other options are incorrect because they either use the wrong flavor (spark instead of sklearn), the wrong function (load_model or track_model instead of log_model), or do not exist (track_model is not a valid MLflow function). References: MLflow Models, mlflow.sklearn

The correct code block is the one that matches the event type and the model name specified in the webhook. The event type is MODEL_VERSION_TRANSITIONED_TO_STAGING, which means the webhook will be triggered when a model version is transitioned to the staging stage. The model name is model, which means the webhook will only apply to the registered model with that name. Therefore, the code block that satisfies both conditions is:

client.transition_model_version_stage (name=model, version=model_version, stage=“Staging”)

This code block will transition the model version to the staging stage and invoke the webhook to run the associated job.

References: MLflow Model Registry Webhooks on Databricks, MLflow Python API

The DESCRIBE HISTORY command can be used to view the commit history of a Delta table, including the schema changes, operations, and timestamps. This command can help identify when a column was dropped from the feature set and by which operation. The other commands are either invalid or do not provide the required information. References:

• Delta Lake - View Commit History
• Databricks Certified Machine Learning Professional Exam Guide - Section 1: Experimentation - Data Management

MLflow provides a module called mlflow.shap that can be used to automatically calculate and log a Shapley feature importance plot using the SHAP library. The mlflow.shap.log_explanation function takes a predict function and a sample of input data, and computes the SHAP values and base values for each feature. It then logs the following artifacts to the current run:

• Base values
• SHAP values (computed using shap.KernelExplainer)
• Summary bar plot (shows the average impact of each feature on model output) The mlflow.shap.log_explanation function can be used with any model that has a predict function, such as scikit-learn, xgboost, etc1234 References:
• mlflow.shap — MLflow 2.9.1 documentation
• Extract feature importance from a mlflow 1.9 PyFuncModel model
• 9.6 SHAP (SHapley Additive exPlanations) | Interpretable Machine Learning
• load MLFlow model and plot feature importance with feature names

To restore the original scikit-learn model object, the correct method is mlflow.sklearn.load_model, which takes the model URI as an argument and returns a scikit-learn model. This method can load models with the sklearn flavor, which preserves the original attributes and methods of the model. The mlflow.load_model method is a generic method that can load models with any flavor, but it returns a pyfunc model, which does not have the feature_importances_ attribute. The client.list_artifacts method is used to list theartifacts associated with a run, not to load a model. The client.pyfunc.load_model method is similar to mlflow.load_model, but it takes a local file path instead of a model URI. The MLflow Experiments UI can show the logged parameters, metrics, and artifacts of a run, but it cannot show the model object itself. References:

• MLflow Models
• MLflow Python API

 To read data from a stream source, such as Kafka, socket, or rate, the spark.readStream method should be used instead of spark.read. The spark.readStream method returns a streaming DataFrame that represents the unbounded input data stream. The spark.readStream method supports the same options and formats as the spark.read method, such as schema, delta, csv, json, etc. The spark.readStream method can also read from a Delta table as a stream source, by specifying the format("delta") and the path or table name of the Delta table123

The other options are incorrect because:

• A. Replacing “inference” with the path to the location of the Delta table does not change the fact that spark.read is used to read from a stream source, which is not supported. The spark.readStream method should be used instead, and the path or table name of the Delta table can be specified as an option or argument.
• B. Replacing schema(schema) with option("maxFilesPerTrigger", 1) does not change the fact that spark.read is used to read from a stream source, which is not supported. The spark.readStream method should be used instead, and the schema can be specified as an option or argument. The option("maxFilesPerTrigger", 1) is an optional configuration that limits the number of files processed in each trigger for file-based stream sources, such as delta, csv, json, etc. It does not affect the reading of data from a stream source4
• D. Replacing format("delta") with format("stream") does not change the fact that spark.read is used to read from a stream source, which is not supported. The spark.readStream method should be used instead, and the format can be specified as an option or argument. The format("stream") is not a valid format for reading data from a stream source. The supported formats are delta, kafka, socket, rate, etc1
• E. Replacing predict with a stream-friendly prediction function does not change the fact that spark.read is used to read from a stream source, which is not supported. The spark.readStream method should be used instead, and the prediction function can be applied to the streaming DataFrame as usual. The predict function does not need to be changed, as long as it can accept a streaming DataFrame as input and return a column of predictions as output5

References:

• Input Sources - Structured Streaming Programming Guide - Spark 3.2.0 Documentation
• Structured Streaming + Delta Lake - Databricks
• Structured Streaming Programming Guide - Spark 3.2.0 Documentation
• Configuration - Structured Streaming Programming Guide - Spark 3.2.0 Documentation
• Machine Learning with Structured Streaming - Databricks

Structured Streaming is a scalable and fault-tolerant stream processing engine built on the Spark SQL engine. It allows users to express streaming computations the same way as batch computations on static data, using DataFrame and Dataset APIs. Structured Streaming can handle both unbounded and bounded data sources, and can process data in micro-batches or continuously. Structured Streaming can be used to provide continuous processing of data for machine learning applications, such as data ingestion, preprocessing, feature engineering, and model inference. StructuredStreaming can also integrate with MLflow and Delta Lake to enable end-to-end machine learning pipelines with tracking, reproducibility, and governance123 References:

• Structured Streaming Programming Guide - Spark 3.2.0 Documentation
• Structured Streaming + Kafka Integration Guide (Kafka broker version 0.10.0 or higher) - Spark 3.2.0 Documentation
• Machine Learning with Structured Streaming - Databricks
• [Continuous Machine Learning with Structured Streaming and MLflow - Databricks]

MLflow Model Serving allows you to deploy MLflow models for real-time serving via REST API. When you enable Model Serving for a registered model, all model versions in the Staging and Production stages are automatically deployed. You can also manually deploy model versions in other stages, such as None or Archived, by using the UI or the API. However, only the model versions in the Staging and Production stages are automatically deployed by default. References: MLflow Model Serving, Deploy MLflow models

To compute predictions for spark_df when the missing feature values can be found in the Feature Store by searching for features by customer_id, you can use the following code block:

Python

# Get the missing features from the Feature Store using the model URI and the customer_id column

df = fs.get_missing_features(spark_df, model_uri, lookup_key="customer_id")

# Score the DataFrame using the model URI and the Feature Store Client

fs.score_batch(model_uri, df)

AI-generated code. Review and use carefully. More info on FAQ.

The fs.get_missing_features method takes a Spark DataFrame, a model URI, and a lookup key as arguments. It returns a new Spark DataFrame that contains the originalcolumns plus the missing features that are required by the model. The missing features are retrieved from the Feature Store by joining the DataFrame with the feature tables using the lookup key. The lookup key must match the primary key of the feature tables. The model URI must point to a registered model that was trained using features from the Feature Store1.

The fs.score_batch method takes a model URI and a Spark DataFrame as arguments. It applies the model to the DataFrame and returns a new Spark DataFrame that contains the original columns plus a prediction column. The model URI must point to a registered model that was trained using features from the Feature Store2.

The other options are incorrect because:

• Option A: fs.score_model is not a valid method name, as it is missing an underscore. The correct method name is fs.score_batch2.
• Option B: fs.score_model without getting the missing features will not work, as the model expects the DataFrame to have all the features that were used for training. The correct way is to use fs.get_missing_features before fs.score_batch12.
• Option D: fs.score_batch without getting the missing features will not work, as the model expects the DataFrame to have all the features that were used for training. The correct way is to use fs.get_missing_features before fs.score_batch12.
• Option E: fs.score_batch without specifying the lookup key will not work, as the fs.get_missing_features method requires a lookup key to join the DataFrame with the feature tables. The correct way is to use fs.get_missing_features with the lookup key “customer_id” before fs.score_batch12. References: Get missing features, Score batch

To log feature importance data from a CSV file as an artifact of an MLflow run, the correct method is mlflow.log_artifact, which takes the local file path and the artifact path as arguments.The mlflow.log_model method is used to log a model with a specific flavor, such as pyfunc or spark. The mlflow.log_data method does not exist. The mlflow.log_model_and_data method also does not exist, and it is not necessary to log both the model and the data in this case. References:

• MLflow Tracking API
• MLflow Models

The correct line of code is “MODEL_VERSION_TRANSITIONED_STAGE”, which is the name of the event that occurs when a model version transitions to any stage in the MLflow Model Registry. This event can be used to trigger a webhook that will execute a Databricks Job based on the model version and the stage it transitions to. The other options are either invalid event names or too specific for the task.

References: MLflow Model Registry Webhooks, MLflow Model Registry Events

Jensen-Shannon (JS) distance is a symmetric measure of the distance between two probability distributions. It is based on the Kullback-Leibler (KL) divergence, but unlike KL, it does not depend on which distribution is considered the reference and which is considered the comparison. JS distance is also bounded between 0 and 1, which makes it easier to interpret and compare across different features.

Kolmogorov-Smirnov (KS) test is a non-parametric test that compares the cumulative distribution functions (CDFs) of two samples. It calculates the maximum difference between the CDFs and compares it to a critical value to determine if the samples are from the same distribution. KS test is sensitive to the shape, location, and scale of the distributions, but it can also be too sensitive for large datasets, where small changes can result in significant differences.

JS distance is more robust than KS test for numeric feature drift detection because it can handle large datasets without being too sensitive to minor fluctuations. It can also capture changes in the shape and spread of the distributions, not just the location. JS distance does not require any manual threshold or cutoff determinations, unlike KS test, which requires choosing a significance level and a critical value. JS distance can also be generalized to multivariate cases, while KS test is limited to univariate cases. References:

• Which test is the best? We compared 5 methods to detect data drift on large datasets
• The Jensen-Shannon Divergence: A Measure of Distance Between Probability Distributions
• Kullback–Leibler vs Kolmogorov-Smirnov distance