Pretrain Large Vision and Language Models for Beginners: With a practical guide for distributed training on AWS and Amazon SageMaker (English Edition)

Price: 22,15 €
(as of May 17, 2023 01:28:13 UTC – Details)

Conceptual fundamentals and practical guidance from industry experts to pretrain the large vision and language models of the future.

Key FeaturesLearn how and where to develop, train, tune, and apply your own pretrained modelsMaster distributed training concepts for models & datasets, with code examples for AWS and SageMakerEvaluate, deploy, and operationalize your custom models with bias detection and pipeline monitoringBook Description

Large models have forever changed machine learning. From BERT to GPT-3, Vision Transformers to DALL-E, when billions of parameters are combined with large datasets and hundreds to thousands of GPUs, the result is nothing short of record-breaking. The recommendations, advice, and code samples in this book will help you pretrain your large models from scratch on AWS and Amazon SageMaker and apply them to hundreds of use cases across your organization.

With advice from seasoned AWS ML expert Emily Webber, this book provides everything you need to go from project ideation, dataset preparation, training, evaluation, and deployment for large language, vision, and multimodal models. With step-by-step explanations of essential concepts and practical examples, you’ll go all the way from mastering the concept of pretraining itself to preparing your dataset and model, configuring your environment, training, evaluating, and deploying your models.

From applying the scaling laws to distributing your model and dataset over multiple GPUs, you’ll learn how to successfully train, evaluate, and deploy your model on Amazon SageMaker. By the end of this book, you will have everything you need to embark on your own project to pretrain the large language models of the future, purpose-built for your organization.

What you will learnPrepare to train large models from the right dataset to your GPU needsConfigure environments on AWS and SageMaker for optimal performanceSelect the right hyperparameters for your model, given your constraintsDistribute your model and dataset with different types of parallelismAvoid pitfalls with job restarts, intermittent health checks, and moreEvaluate your model with quantitative and qualitative insightsDeploy your models with runtime improvements and MonitoringDetect and mitigate bias in your deploy and retrain pipelinesWho This Book Is For

If you’re a machine learning enthusiast or researcher who wants to get started on your very own large modeling project, this book is for you. Applied scientists, data scientists, machine learning engineers, solution architects, product managers, and students will all enjoy the material. Basic Python is a must, and introductory concepts around cloud computing will be very helpful. We’ll assume some level of deep learning fundamentals but will explain advanced topics.

Table of ContentsAn introduction to pretrainingDataset preparation: part oneModel preparationInto the GPUParallelization basicsDataset preparation: part twoFind the right hyperparametersMake sure your loss goes downTroubleshoot ongoing performanceDetermine the right length of training timeFinetune and compare with open source modelsDetect and mitigate biasHow small can you go? Use cases: scale across organizationsOngoing operations, monitoring and maintenance

ASIN ‏ : ‎ B0BFFDF4MQ
Éditeur ‏ : ‎ Packt Publishing (9 juin 2023)
Langue ‏ : ‎ Anglais
Synthèse vocale ‏ : ‎ Activée
Confort de lecture ‏ : ‎ Non activé
X-Ray ‏ : ‎ Non activée
Word Wise ‏ : ‎ Non activé
Pense-bêtes ‏ : ‎ Non activé