AWS Machine Learning

MARCH 20, 2023

Overview of the technology EC2 C6i instances are powered by third-generation Intel Xeon Scalable processors (also called Ice Lake) with an all-core turbo frequency of 3.5 Refer to the appendix for instance details and benchmark data. Quantizing the model in PyTorch is possible with a few APIs from Intel PyTorch extensions.

Calibration 83

Calibration Scripts Benchmark APIs 83

AWS Machine Learning

OCTOBER 5, 2022

We explored nearest neighbors, decision trees, neural networks, and also collaborative filtering in terms of algorithms, while trying different sampling strategies (filtering, random, stratified, and time-based sampling) and evaluated performance on Area Under the Curve (AUC) and calibration distribution along with Brier score loss.

Calibration 95

Calibration Engineering Automotive Analytics 95

AWS Machine Learning

APRIL 8, 2024

In this post, we explore the latest features introduced in this release, examine performance benchmarks, and provide a detailed guide on deploying new LLMs with LMI DLCs at high performance. Be mindful that LLM token probabilities are generally overconfident without calibration.

Engineering 89

Engineering Calibration APIs Enterprise 89

AWS Machine Learning

NOVEMBER 29, 2023

Each trained model needs to be benchmarked against many tasks not only to assess its performances but also to compare it with other existing models, to identify areas that needs improvements and finally, to keep track of advancements in the field. Evaluating these models allows continuous model improvement, calibration and debugging.

Benchmark 87

Benchmark Metrics Engineering APIs 87