article thumbnail

Use Snowflake as a data source to train ML models with Amazon SageMaker

AWS Machine Learning

We create a custom training container that downloads data directly from the Snowflake table into the training instance rather than first downloading the data into an S3 bucket. 1 with the following additions: The Snowflake Connector for Python to download the data from the Snowflake table to the training instance.

Scripts 102
article thumbnail

Promote pipelines in a multi-environment setup using Amazon SageMaker Model Registry, HashiCorp Terraform, GitHub, and Jenkins CI/CD

AWS Machine Learning

Under Advanced Project Options , for Definition , select Pipeline script from SCM. For Script Path , enter Jenkinsfile. upload_file("pipelines/train/scripts/raw_preprocess.py","mammography-severity-model/scripts/raw_preprocess.py") s3_client.Bucket(default_bucket).upload_file("pipelines/train/scripts/evaluate_model.py","mammography-severity-model/scripts/evaluate_model.py")

Scripts 97
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

­­Speed ML development using SageMaker Feature Store and Apache Iceberg offline store compaction

AWS Machine Learning

SageMaker Feature Store automatically builds an AWS Glue Data Catalog during feature group creation. Customers can also access offline store data using a Spark runtime and perform big data processing for ML feature analysis and feature engineering use cases. Table formats provide a way to abstract data files as a table.

Scripts 72
article thumbnail

Optimize for sustainability with Amazon CodeWhisperer

AWS Machine Learning

Amazon CodeWhisperer currently supports Python, Java, JavaScript, TypeScript, C#, Go, Rust, PHP, Ruby, Kotlin, C, C++, Shell scripting, SQL, and Scala. times more energy efficient than the median of surveyed US enterprise data centers and up to 5 times more energy efficient than the average European enterprise data center.

article thumbnail

Use RAG for drug discovery with Knowledge Bases for Amazon Bedrock

AWS Machine Learning

Knowledge Bases for Amazon Bedrock supports multiple vector databases, including Amazon OpenSearch Serverless , Amazon Aurora , Pinecone, and Redis Enterprise Cloud. For enterprise implementations, Knowledge Bases supports AWS Key Management Service (AWS KMS) encryption, AWS CloudTrail integration, and more. Nihir Chadderwala is a Sr.

APIs 110
article thumbnail

How BigBasket improved AI-enabled checkout at their physical stores using Amazon SageMaker

AWS Machine Learning

During each training iteration, the global data batch is divided into pieces (batch shards) and a piece is distributed to each worker. Each worker then proceeds with the forward and backward pass defined in your training script on each GPU.

article thumbnail

Amazon SageMaker Feature Store now supports cross-account sharing, discovery, and access

AWS Machine Learning

This new capability promotes collaboration and minimizes duplicate work for teams involved in ML model and application development, particularly in enterprise environments with multiple accounts spanning different business units or functions. You need to provide your consumer AWS account ID before running the notebook.