Quote:
Originally Posted by PJo336
Our setup for a new project is "fairly" automated but how do you handle permissions in AWS?
We use terraform to create roles and ecr repos and what not, which takes time to get through, we use some KMS setups to encrypt values, we use protobuf schemas. All these things are "easy" but tedious and take time. And those are a lot of the grunt work I am kind of talking about.
[Note: suzzer, I meant to lay this out to you the other day but forgot]
So basically the way we create our infrastructure is with a fairly simple in house tool (it's actually on github but I don't know how easy it would be for other people to use it)
There's one master git repo that describes the whole enterprise. So you start there - you make a config file for your new service, that lays out "what kind" of service it is, from a predefined list (like, api server, batch processor, sqs queue listener, maybe a few other things). You commit to this repo, which kicks off a cluster build.
The cluster build creates a git repo for your project and applies a template to it, so api servers get a skeleton created for them - some config files and a flask entrypoint file, etc. It also creates build job in bamboo. This job will look for any modifications to the git repo and
* check them out
* run the build
* run tests
* deploy to CI
* smoke test CI
* deploy to prod
* smoke test prod
You check out the new repo (actually, you use the tool and say "update any repos in this cluster that need updating)
The repo for the service has a service description file (a json file) which is basically a list of "service modifications" along with associated parameters. A service modification is basically any, uh, modification to the service. So you might add the following service modifications
* create RDS database
* create SQS queue, with such and such deadletter policy, giving RW permissions to user X
* add autoscaling policy with such and such settings
Each service modification essentially is:
* a fragment of a cloudformation template
* a cloudformation template paramter file
* a json file that describes the defaults for the service mod
So our tooling takes the base cloudformation template for your chosen service type, adds the cloudformation template fragments for each service modification, with their chosen values overlaid over the default values, and then submits that to cloudformation. A few minutes later, the service is up.
This is essentially idempotent - you keep resubmitting your cloudformation template and it'll do whatever needs to be done to change what you have into what you want.
We've given up on the "you can do anything you want" and instead just have a bunch of pre-defined things to do. Like, "we make SQS queues this way, and these are the parameters you can tweak"
Sometimes this means that to do what I want I need to **** around with making a new service modification, and I'll waste half a day on it. But generally things will fit OK into one of the existing things we have.
Let's be honest - 99% of the time you're making a service which fits into one a few categories. It's like building with LEGO instead of carving from a wooden block.
Our company is split into a few logical sections - one of the other main sections uses terraform and it seems like they spend days or weeks setting up infra. And their deploys take a long long time.
Like I said, generally about 30 minutes to first prod deploy, any given deploy, from git commit to prod is under 10 minutes.