It’s been a while since I posted anything here, but a lot has happened on the front. I will give a quick update about the things that have interested me since then
In 2013 I created my first auto scalable event store architecture for a huge client in Node.js, involving custom web servers receiving events from different endpoints in different formats, meta-tagging them and then injecting them into amazon queues, with processors on the other end enriching and transforming the events for storage in AWS DynamoDB. Post processors would be run periodically to store aggregates in S3. It was required to auto-scale to handle 200.000 events per second. (Yes, you read that right). I created a stateless architecture with the code for all the roles (server, processor, post-processor etc), built into one repo, which would be tarred and deployed onto S3 by our Bamboo server, to allow new nodes to be bootstrapped with that. The node itself was already booted by puppet with a role to perform, and thus knew it’s role to play. For hot upates and rollbacks we’d tell a Saltstack master to update a certain range of nodes, which would then pull the wanted source from the S3 registry again and update themselves without downtime. Pretty nifty, but rather proprietary.
The company I worked for used Puppet for configuration management, but also for app deployment, which I thought was the wrong approach. Puppet is imo not designed for realtime deployment, but rather for booting and maintaining vm’s from config. That is how I came across Saltstack’s powerful realtime command capabilities, and decided to script our deployment process to be controlled by Saltstack. I actually haven’t updated on that front in a long time, but I saw it fit the bill for our needs and I was so bold to build it into our POC.
Too bad we hadn’t learned about Google’s Go language back then, otherwise I would have scratched myself behind the ears and probably opted for that, instead of Node.js for our highly concurrent applications.