Case Study
Traeger Grills Cuts Costs and Grows Demand with AWS IoT Services
A story about cutting AWS costs at a growing enterprise is rare. A story about cutting them by a game-changing 50% — while recovering from the single worst business day in the history of a 35-year-old company — is practically epic.
When I took the stage with AWS Principal Solutions Architect Aditya Pendyala, the chalk talk room at Caesars Forum in Las Vegas was packed. Past the obligatory ten minutes of questions about brisket and pulled pork, I gave 60 cloud engineers my war story and hard-earned lessons. Here are the highlights.
The Brand
Introducing Traeger
Traeger has a fanatical customer base of foodies and grillers — the Traegerhood — who come to us for ad-free, mouthwatering recipes, effortlessly perfect cooks, and a gourmet grilling lifestyle on the WiFIRE app. They need the experience to just work, and we deliver that.
Unfortunately, when I inherited the IoT team, we were running a cloud system that wasn't built to scale — and we were about to find out just how loud the Traegerhood could get when we disappointed them.
The Architecture
Serverless-First at Launch
When I inherited it, a zoomed-out view of our AWS cloud looked something like this.
IoT Core processed information from our grills — think of it as a short-order cook yelling "one moo moo on a raft, two oinkers swimming upriver," then "order up!" when it's ready.
DynamoDB stored data like usernames and passwords and served up recipe history and preferences — the waiter who remembers not just your name, but your favorite table, your usual cocktail, and that dish with the capers you always recommend.
An API Gateway managed requests like a host fielding reservations and walk-ins, and Cognito was our bouncer, checking IDs at the door.
Between front and back of house sat the app's main logic, organized into 400 Lambda containers — a brigade of highly specialized waiters. It sounds chaotic, but it let our devs build each component in parallel and assemble them at the end. A necessity, given the brutally tight migration timeline.
Migration by the Numbers
Building the system wasn't a slow simmer — it was a CapEx-fueled broil. In just three months, the cloud team migrated 111,000 devices, 700,000 users, and 150 terabytes of data, with zero downtime.
I didn't lead the team that built it, but I joined the companywide celebration when it shipped. We toasted our success with skewered meats.
Then in July 2022, post-COVID restructuring consolidated cloud management under Technology. I'd inherited a system nobody knew every part of.
Growth Explodes
Meanwhile, the migration powered incredible growth. In just 18 months, we doubled the number of connected grills to 300,000. At the height of our hubris, we published a whitepaper boasting we were ready to accommodate millions more.
We were wrong.
The Lesson
The Hard Lesson
It turns out it's not the size of your cloud that matters, but the principles behind it. We'd built a big, static system we thought could handle whatever we threw at it.
But technology and consumers are always changing, so static architecture is just waiting to fail. Even with more capacity than ever, there was still a hard cap on the devices we could connect — and it wasn't anywhere near the millions.
The Crisis
Thanksgiving 2022
Thanksgiving is our Super Bowl Sunday. Everyone and their brother-in-law fires up their grill and app around the same time and runs them all day — sharing holiday recipes, compulsively checking cook temps, and salivating over their turkeys.
The active Thanksgiving Traegerhood is about 5x larger than any other single day of the year. There's no other day that gives even a mild preview of the cascading failures we were about to see.
A Sub-Optimal Turkey Day
If you want a case study in the dangers of building dependencies too tightly in the cloud, this is it.
Chaos was brewing alongside my morning coffee. By 9:00am the app wasn't loading recipes, and I got a frantic call from my CEO. I put my senior developer on it, and by 10:15 he'd shipped a hotfix: a caching layer to relieve our overworked CMS.
Unfortunately, the fix kicked every WiFIRE user off the app. Hundreds of thousands had to line up to log back in — and a mob that size doesn't wait quietly.
The Traegerhood exploded into a social-media fireball as our system hit 450,000 throttles per second. Users waited longer to log back in than they did for pumpkin pie. I was the Grinch of Thanksgiving, and I knew I had to fix it before next year.
What Went Wrong
Grillers started requesting recipes all at once, overwhelming our CMS — which was designed to pull each recipe from the database every single time. It began rate-limiting and failing to render recipes, and people couldn't plan their cooks.
Caching recipes was the right idea, but launching it as a hotfix was a mistake. None of our specialized Lambda waiters got the memo, so they kept trying to process requests the old way. That invalidated every user session and forced everyone to log back in.
We also didn't know Cognito was already being hit with 3x its capacity before the patch — we just didn't have that visibility. And unless you ask them to wait their turn, apps keep tapping the bouncer's shoulder like toddlers avoiding bedtime. They hugged Cognito to death, and authentication failed completely.
The Fix
Evolving for Scale
The easy answer is that my senior developer blew it with his patch. The truth is we inherited a black-box legacy system, built in a few months and never given the maintenance it needed.
More importantly, we hadn't understood that scaling for 300,000 users looks wildly different from scaling for millions — and planning for a once-a-year, 15x peak is its own special challenge.
After deep retrospection, we identified four critical weaknesses and their fixes: no visibility, an exposed Cognito, rigid compute, and weak caching.
Observability First
First, we flipped on all the lights. We added New Relic, an observability layer, on top of everything so we could predict sign-on and throttling issues long before a line formed.
Now we can see every link in the data chain, set alerts for the first signs of trouble, and diagnose issues in real time. It might interrupt my morning coffee now and again, but the building won't be on fire by the time I get to work.
A lot of teams resist observability — it's an added cost and an extra layer to build into everything. Expect pushback. Insist anyway, especially at enterprise scale. You can't fix what you can't see.
Insulating Cognito
Next, we gave our bouncer some backup. Identity systems are the weak point of the internet — few can handle more than Cognito's 1,200 transactions per second, so they're easily swarmed. They need something that calms app clients down and asks them to wait politely in line.
Put your identity system at the center of the fortress and add a container to run interference, so your bouncer doesn't have to fight off an unruly app mob. Then scale workloads around it to insulate it further.
Right-Sizing Compute
After that, we built more flexibility into data processing. We'd been running every information request through our 400 specialized Lambda boxes. Run a restaurant that way and you'd be wildly overstaffed and inefficient.
So we moved most traffic to ECS containers — a more generalized wait staff — and kept a few functions in Lambda for genuinely specialized requests. Because sometimes you just need a sommelier.
We have 1.7 million users today and can handle 10 million. By the time we reach 10 million, I'll make sure we can scale to 100 million.
Caching the Hot Path
Finally, we added lazy-load caching so the system doesn't have to run data all the way to and from the database on every request. We don't prepopulate the cache — it automatically stores what users ask for, then checks whether it's changed since the last request. It's like keeping an organized recipe book on hand for the most popular dishes.
Everyone should be doing lazy-load caching. Cache records early, cache them often. And do not deploy caching during a peak.
The Payoff
Thanksgiving 2023 & Beyond
After a year of planning and rebuilding, the real test arrived: Thanksgiving 2023. Compared to 2022, it was unrecognizable.
In the year since, our fleet of grills had grown by another 200,000. Yet on Thanksgiving Day 2023, sign-ins maxed out around 100 per second, versus a 2022 peak near 3,500. And those 450,000 throttles per second? Down to about one throttle every three seconds.
My CEO and I got to enjoy our coffee in peace, and hundreds of thousands of cooks nationwide cut into perfect turkeys after a stress-free, set-it-and-forget-it day. The best, most forgettable Thanksgiving the platform team had ever had.
What did it really cost? Far less than you'd think. We reduced per-device cloud costs by 50% while maintaining 99.9% uptime, and the platform now truly scales to millions. The only successful infrastructure is adaptive, not static — but the best infrastructure is invisible and forgotten, because it just works.
Let's Talk
Questions?
Have questions about my team's AWS transformation? Planning a migration of your own? Let's talk shop — you'll find me on Signal, GitHub, LinkedIn, or plain old email below.
The Brand
Introducing Traeger
Traeger has a fanatical customer base of foodies and grillers — the Traegerhood — who come to us for ad-free, mouthwatering recipes, effortlessly perfect cooks, and a gourmet grilling lifestyle on the WiFIRE app. They need the experience to just work, and we deliver that.
Unfortunately, when I inherited the IoT team, we were running a cloud system that wasn't built to scale — and we were about to find out just how loud the Traegerhood could get when we disappointed them.
The Architecture
Serverless-First at Launch
When I inherited it, a zoomed-out view of our AWS cloud looked something like this.
IoT Core processed information from our grills — think of it as a short-order cook yelling "one moo moo on a raft, two oinkers swimming upriver," then "order up!" when it's ready.
DynamoDB stored data like usernames and passwords and served up recipe history and preferences — the waiter who remembers not just your name, but your favorite table, your usual cocktail, and that dish with the capers you always recommend.
An API Gateway managed requests like a host fielding reservations and walk-ins, and Cognito was our bouncer, checking IDs at the door.
Between front and back of house sat the app's main logic, organized into 400 Lambda containers — a brigade of highly specialized waiters. It sounds chaotic, but it let our devs build each component in parallel and assemble them at the end. A necessity, given the brutally tight migration timeline.
Migration by the Numbers
Building the system wasn't a slow simmer — it was a CapEx-fueled broil. In just three months, the cloud team migrated 111,000 devices, 700,000 users, and 150 terabytes of data, with zero downtime.
I didn't lead the team that built it, but I joined the companywide celebration when it shipped. We toasted our success with skewered meats.
Then in July 2022, post-COVID restructuring consolidated cloud management under Technology. I'd inherited a system nobody knew every part of.
Growth Explodes
Meanwhile, the migration powered incredible growth. In just 18 months, we doubled the number of connected grills to 300,000. At the height of our hubris, we published a whitepaper boasting we were ready to accommodate millions more.
We were wrong.
The Lesson
The Hard Lesson
It turns out it's not the size of your cloud that matters, but the principles behind it. We'd built a big, static system we thought could handle whatever we threw at it.
But technology and consumers are always changing, so static architecture is just waiting to fail. Even with more capacity than ever, there was still a hard cap on the devices we could connect — and it wasn't anywhere near the millions.
The Crisis
Thanksgiving 2022
Thanksgiving is our Super Bowl Sunday. Everyone and their brother-in-law fires up their grill and app around the same time and runs them all day — sharing holiday recipes, compulsively checking cook temps, and salivating over their turkeys.
The active Thanksgiving Traegerhood is about 5x larger than any other single day of the year. There's no other day that gives even a mild preview of the cascading failures we were about to see.
A Sub-Optimal Turkey Day
If you want a case study in the dangers of building dependencies too tightly in the cloud, this is it.
Chaos was brewing alongside my morning coffee. By 9:00am the app wasn't loading recipes, and I got a frantic call from my CEO. I put my senior developer on it, and by 10:15 he'd shipped a hotfix: a caching layer to relieve our overworked CMS.
Unfortunately, the fix kicked every WiFIRE user off the app. Hundreds of thousands had to line up to log back in — and a mob that size doesn't wait quietly.
The Traegerhood exploded into a social-media fireball as our system hit 450,000 throttles per second. Users waited longer to log back in than they did for pumpkin pie. I was the Grinch of Thanksgiving, and I knew I had to fix it before next year.
What Went Wrong
Grillers started requesting recipes all at once, overwhelming our CMS — which was designed to pull each recipe from the database every single time. It began rate-limiting and failing to render recipes, and people couldn't plan their cooks.
Caching recipes was the right idea, but launching it as a hotfix was a mistake. None of our specialized Lambda waiters got the memo, so they kept trying to process requests the old way. That invalidated every user session and forced everyone to log back in.
We also didn't know Cognito was already being hit with 3x its capacity before the patch — we just didn't have that visibility. And unless you ask them to wait their turn, apps keep tapping the bouncer's shoulder like toddlers avoiding bedtime. They hugged Cognito to death, and authentication failed completely.
The Fix
Evolving for Scale
The easy answer is that my senior developer blew it with his patch. The truth is we inherited a black-box legacy system, built in a few months and never given the maintenance it needed.
More importantly, we hadn't understood that scaling for 300,000 users looks wildly different from scaling for millions — and planning for a once-a-year, 15x peak is its own special challenge.
After deep retrospection, we identified four critical weaknesses and their fixes: no visibility, an exposed Cognito, rigid compute, and weak caching.
Observability First
First, we flipped on all the lights. We added New Relic, an observability layer, on top of everything so we could predict sign-on and throttling issues long before a line formed.
Now we can see every link in the data chain, set alerts for the first signs of trouble, and diagnose issues in real time. It might interrupt my morning coffee now and again, but the building won't be on fire by the time I get to work.
A lot of teams resist observability — it's an added cost and an extra layer to build into everything. Expect pushback. Insist anyway, especially at enterprise scale. You can't fix what you can't see.
Insulating Cognito
Next, we gave our bouncer some backup. Identity systems are the weak point of the internet — few can handle more than Cognito's 1,200 transactions per second, so they're easily swarmed. They need something that calms app clients down and asks them to wait politely in line.
Put your identity system at the center of the fortress and add a container to run interference, so your bouncer doesn't have to fight off an unruly app mob. Then scale workloads around it to insulate it further.
Right-Sizing Compute
After that, we built more flexibility into data processing. We'd been running every information request through our 400 specialized Lambda boxes. Run a restaurant that way and you'd be wildly overstaffed and inefficient.
So we moved most traffic to ECS containers — a more generalized wait staff — and kept a few functions in Lambda for genuinely specialized requests. Because sometimes you just need a sommelier.
We have 1.7 million users today and can handle 10 million. By the time we reach 10 million, I'll make sure we can scale to 100 million.
Caching the Hot Path
Finally, we added lazy-load caching so the system doesn't have to run data all the way to and from the database on every request. We don't prepopulate the cache — it automatically stores what users ask for, then checks whether it's changed since the last request. It's like keeping an organized recipe book on hand for the most popular dishes.
Everyone should be doing lazy-load caching. Cache records early, cache them often. And do not deploy caching during a peak.
The Payoff
Thanksgiving 2023 & Beyond
After a year of planning and rebuilding, the real test arrived: Thanksgiving 2023. Compared to 2022, it was unrecognizable.
In the year since, our fleet of grills had grown by another 200,000. Yet on Thanksgiving Day 2023, sign-ins maxed out around 100 per second, versus a 2022 peak near 3,500. And those 450,000 throttles per second? Down to about one throttle every three seconds.
My CEO and I got to enjoy our coffee in peace, and hundreds of thousands of cooks nationwide cut into perfect turkeys after a stress-free, set-it-and-forget-it day. The best, most forgettable Thanksgiving the platform team had ever had.
What did it really cost? Far less than you'd think. We reduced per-device cloud costs by 50% while maintaining 99.9% uptime, and the platform now truly scales to millions. The only successful infrastructure is adaptive, not static — but the best infrastructure is invisible and forgotten, because it just works.
Let's Talk
Questions?
Have questions about my team's AWS transformation? Planning a migration of your own? Let's talk shop — you'll find me on Signal, GitHub, LinkedIn, or plain old email below.
Further Reading
The full cost optimization story — how Traeger cut per-device cloud costs by 50% — was published as an AWS case study.
Read the AWS Blog Post (opens in new tab) →