The Amazon SageMaker JumpStart SDK has been simplified for building, training, and deploying foundation models. The code for prediction is now easier to use. This post demonstrates how to get started with using foundation models using the simplified SageMaker Jumpstart SDK in just a few lines of code. You can find more information about the SDK for deployment and training in the documentation. SageMaker JumpStart provides pre-trained models and solution templates for various problem types, as well as example notebooks. You can access these resources through the SageMaker JumpStart landing page or use the SageMaker Python SDK. The post also provides instructions on how to deploy and invoke the models, as well as customize the new classes in the SageMaker SDK. Overall, the simplified SageMaker JumpStart SDK offers an easy way to work with task-based and foundation models, and supports customization for specific requirements.
Simplified SageMaker JumpStart SDK for Building, Training, and Deploying Models
We are excited to introduce a simplified version of the Amazon SageMaker JumpStart SDK that makes it easy to build, train, and deploy foundation models. With just a few lines of code, you can get started with using pre-trained models and fine-tune them for your specific tasks.
Solution Overview
SageMaker JumpStart provides pre-trained, open-source models for various problem types, allowing you to quickly start your machine learning (ML) projects. You can incrementally train and fine-tune these models before deployment. JumpStart also offers solution templates and example notebooks for common ML use cases.
To demonstrate the capabilities of the new SageMaker JumpStart SDK, we show you how to use the pre-trained Flan T5 XL model from Hugging Face for text generation and summarization tasks. You can also use other models like Llama2, Falcon, or Mistral AI for text generation.
To deploy the model, you can use the model ID provided for each pre-trained model. In this example, we use the model ID for the Flan T5 XL model. After deployment, you can easily invoke the model to generate summaries of text.
Deploy and Invoke the Model
To deploy the Flan T5 XL model, you can use the simplified SageMaker JumpStart SDK. Simply instantiate the model object with the model ID and call the deploy method. Here’s an example:
from sagemaker.jumpstart.model import JumpStartModel
pretrained_model = JumpStartModel(model_id="huggingface-text2text-flan-t5-base")
pretrained_predictor = pretrained_model.deploy()
Once the model is deployed, you can invoke it by passing the text to the predictor. The response from the model will be returned as a Python dictionary. Here’s an example:
text = "Summarize this content - Amazon Comprehend uses natural language processing (NLP) to extract insights about the content of documents..."
query_response = pretrained_predictor.predict(text)
print(query_response["generated_text"])
This will generate a summary of the provided text using the Flan T5 XL model.
Fine-tune and Deploy the Model
The SageMaker JumpStart SDK also provides a JumpStartEstimator class for simplified fine-tuning. You can provide the location of your fine-tuning data and optionally pass validation datasets. After fine-tuning, you can deploy the model using the deploy method of the Estimator object. Here’s an example:
from sagemaker.jumpstart.estimator import JumpStartEstimator
estimator = JumpStartEstimator(
model_id=model_id,
)
estimator.set_hyperparameters(instruction_tuned="True", epoch="3", max_input_length="1024")
estimator.fit({"training": train_data_location})
finetuned_predictor = estimator.deploy()
Customize the New Classes in the SageMaker SDK
The new SDK allows you to customize the deployment and invocation based on your requirements. You can override the defaults and customize parameters such as instance type, VPC configuration, and more. Here’s an example of overriding the instance type:
finetuned_predictor = estimator.deploy(instance_type='ml.g5.2xlarge')
You can also customize the input payload format type using serializers and content types. Here’s an example of setting the payload input format as JSON:
from sagemaker import serializers
from sagemaker import content_types
pretrained_predictor.serializer = serializers.JSONSerializer()
pretrained_predictor.content_type = 'application/json'
Conclusion
The simplified SageMaker JumpStart SDK makes it easy to build, train, and deploy models with just a few lines of code. You can use pre-trained models or fine-tune them for your specific tasks. The SDK also allows for customization to meet your requirements. Explore the available models and start leveraging AI to redefine your work processes.
About the Authors
Evan Kravitz, Rachna Chadha, Jonathan Guinegagne, and Dr. Ashish Khetan are experts in the field of AI and ML, with experience in developing and applying machine learning algorithms. They are part of the Amazon SageMaker JumpStart team, working to make AI accessible and impactful.
If you’re interested in evolving your company with AI, connect with us at hello@itinai.com. For more insights into leveraging AI, follow us on Telegram t.me/itinainews or Twitter @itinaicom.
Spotlight on a Practical AI Solution
Consider the AI Sales Bot from itinai.com/aisalesbot, designed to automate customer engagement and manage interactions across all stages of the customer journey. Discover how AI can redefine your sales processes and customer engagement. Explore solutions at itinai.com.
