We refer to our back-end cloud data processing system as ‘Boxer.’ It’s short for ‘Weather Service in a Box’ and also a dog joke. We love dog jokes. Boxer gathers standard data sources (GFS, HRRR, RTMA, etc.) and processes them for rapid display. In the future, it’ll do more. In this post, we’ll discuss Infrastructure as Code and how it applies to us.
The Imperative for Cloud Agility
One of my goals for Boxer was to quickly set up and tear down instances of the service. Some of this is strategic and necessary for our product offering, but much of it is my own bias. I’ve seen companies bog down when developers can’t use a full version of the system, and updating production is slow and risky.
To mitigate this, we’ve bet heavily on Infrastructure as Code, specifically CDK. So when I see this on our CloudFormation console, it makes me happy. If you’re unfamiliar with CloudFormation, it manages a set of AWS resources that can easily be stood up and torn down.
Embracing AWS CDK and Python
Meteorology uses Python, so we use Python. We’re interacting with CloudFormation via CDK using Python bindings, and it’s working pretty well.
From our console, I can see one of our engineers is wrapping up RTMA support in one stack. I know that thick-duck is the stack a customer is testing against. We were doing some security testing in windy-swamp, and that can probably be torn down.
Enhancing Scalability
When we’re done, we can quickly stand these Boxer stacks up and tear them down again. For development, we tell our Lambdas and container services they’re of a given stack and even replace parts for more extended tests. There will always be some dev overhead in cloud development, but we’re mitigating it pretty well. So far, so good!