When the pipe bursts
7am every Wednesday, it was time to release the code. I’d take a weeks worth of code that I knew nothing about, by 2 dozen people, and ship it to production. We tested the code in isolation. We tested it together in our lower environments. But even then when it was up on production who knew what was going to happen. Even if the release itself went well, there was always a chance that a feature was not ready for users. Finding that out at the last minute could push a release to a later day. This would delay other important releases that teams worked hard on.
By batching up releases, we thought we were doing everyone a favor. Customer success only has to learn about the release once a week! One person can own the release process and be an authority on it! We can test everything together all at once, and know for sure that it will work!. We were so wrong.
By batching up releases, we introduced a ton of risk. The release became an all or nothing affair. If any feature in a release wasn’t ready for customers, everything we had already communicated to customers was at risk. We would mitigate that risk by creating new problems. Rushing through something or introducing new issues.
The person (me) who was doing the deployment had no context for what exactly I was deploying. Sure the steps we could document the steps and what was in the release. But if something went wrong,, all I knew I could do was stop the whole train. If anything broke, it was impossible to figure out what may have caused it. Even if you are an expert of a system, a week’s worth of work by 20+ people is impossible to isolate defects.
After a few too many stressful releases, we decided one of the best investments we could make quickly was feature flags. With feature flags, you separate the release of code to customers from the deployment to production. Code for a new feature can live in a production environment for any duration of time without disruption to your customer. When the feature is ready, any team member can turn it on. If there is an issue for any reason, Anyone can shut the feature off, limiting impact.
We saw improvements to our process right away. We stopped batching deployments, instead we would deploy to production regularly. Each change getting sent to production was easy to discern, it could be as small as a couple lines of code. If there was an issue it would be easy to track and identify.
When a feature was ready for customers, anyone on our team, especially our product owners, could turn it on. We even went further. To deliver value customers sooner, we could build smaller features and only release it to users who had no known edge cases. Once we had more time to identify how to approach the edge cases those customers would receive the updates. Teams became more confident, less stressed, and could get feedback on their work in a much shorter timeframe.
This was a transformative experience for me. I’ve been preaching the gospel ever since. I knew I could help get teams to work faster by enabling feature flags for more teams.
That’s why I built Molasses.
The Great Molasses Flood
I started building in June. I focused on the problems I saw with the current tools. I surveyed those who had used feature flagging and A/B testing services in the past to get their opinion. What I found was that while there are great solutions here and there, but there was space for improvement.
Despite being a developer focused tool, so many feature flag services are developer hostile. They can be slow, and make blocking remote calls. They are naive in their ability to target users, sometimes only by IP address or one parameter. They had no way to audit changes. They lacked integrations with platforms for planning, chatops or observability. Teams are slow to remove feature flags, accumulating technical debt. The gaps in these feature flag systems are filled with custom software. Taking away from expensive developer and operations resources.
Molasses allows teams to get up and running with feature flags right away. It supports many different languages and frameworks. Molasses’ clients are fast, non-blocking, and the service pushes updates to the client instantly, no waiting. You can roll out features to specific users and target specific groups. Molasses lets teams focus on the features they are building, not building feature flag technology.
Molasses unlocks A/B testing. We can examine the effectiveness of a change right away. Want to start rolling out a feature to a set of alpha or beta users? No problem! Molasses will let you adjust who receives a feature over time.
Molasses isn’t the only tool in your toolchain. It integrates with the systems you already use, like Datadog or Slack. You can use those updates to know if a recent change is causing issues. When your feature is finally rolled out, Molasses has the code clean itself up. It opens a PR in your Github repo removing any stale code.
I’m proud of what I’ve built so far. I’m excited to watch teams speed up with Molasses.
Molasses is a feature flag and A/B testing service.