AI Hosting & Deployment

Monitoring & Cost Optimization

This lesson covers monitoring and cost optimization in AI, including how to check a system's live performance and trigger alerts, and how to optimize costs by reducing computational resources. We will also explore how to save and load models, and how to optimize hyperparameters.

Why It Matters

Monitoring and cost optimization are crucial in AI as they help ensure a system's performance and efficiency. By monitoring a system's live performance, we can identify issues before they become major problems. Additionally, optimizing costs helps reduce waste and save resources, which is especially important in AI where computational resources can be expensive.

Key Points

• Monitoring a System's Live Performance: We can use monitoring code to check a system's live performance at regular intervals and trigger alerts when necessary. This ensures that we can identify and address issues before they become major problems.

• Saving and Loading Models: We can save a model using libraries like joblib, which allows us to easily load the model later and use it for making predictions.

• Optimizing Hyperparameters: We can use techniques like grid search or random search to optimize hyperparameters, which can improve a model's performance.

• Cost Optimization: We can reduce costs by optimizing computational resources, such as reducing the number of nodes or the amount of memory used.

• Using Cloud Services: Cloud services like Google Cloud AI Platform can help optimize costs by automatically scaling resources up and down based on demand.

• Model Deployment: We can deploy models to production environments using services like Google Cloud AI Platform, which provides a simple way to deploy and manage models.

Key Concepts

tabu search

A technique used to escape plateaus in optimization problems by keeping a list of recently visited states and forbidding the algorithm to return to those states.

simulated annealing

A technique used to escape plateaus in optimization problems by randomly perturbing the current assignment.

constraint weighting

A technique used to concentrate the search on the important constraints by giving them a numeric weight.

A∗ search

A search algorithm that uses a heuristic function to guide the search towards the goal state.

Code Examples

Saving a model using joblib

import joblib joblib.dump(final_model, "my_california_housing_model.pkl")

Loading a model using joblib

import joblib final_model = joblib.load("my_california_housing_model.pkl")

From the books

“New features can be added quickly. (7) All input feature code is tested. Tests for Model Development (1) Every model speciﬁcation undergoes a code review. (2) Every model is checked in to a repository…”

“a simple web service that takes care of load balancing and scaling for you. It takes JSON requests containing the input data (e.g., of a district) and returns JSON responses containing the predictions…”

“ready for production (e.g., polish the code, write documentation and tests, and so on). Then you can deploy your model to your production environment. The most basic way to do this is just to save the…”

← Serverless & Edge Deployment