Just yesterday, I wanted to automate creating Lambda layers and publishing them to multiple regions using GitHub Actions.
Little did I know, I was about to make an unexpected and amusing mistake that would leave me with 830 commits and Lambda versions.
In this blog post, I’ll share the story of how it went down, and provide some code examples to help you avoid making the same mistake.
The goal was to create a GitHub Action that would handle the following tasks:
- Create a Lambda layer.
- Publish the layer to multiple AWS regions.
- Update the README file with the new layer ARNs.
Manually running the GitHub action to keep debugging my
action.ymlwas a hassle.
So, to test my GitHub Action on every update, I added an ‘on-push’ trigger.
At the end of the workflow, after creating a Lambda layer:
- I update README with a new layer version ARN
- Commit changes to README.md
- Push changes back to the main repo
Here’s a simplified version of the GitHub Actions workflow that caused the issue:
name: Publish Lambda Layer on: push: jobs: publish-layer: runs-on: ubuntu-latest steps: # ... (Steps to set up AWS credentials, create Lambda layer, and publish it to multiple regions) - name: Update README run: | # ... (Code to update the README file with the new layer ARNs) - name: Commit changes run: | # ... config git username and email git add README.md git commit -m "Update README with new layer ARNs" git push
After I fixed all the issues by staying up till 5 AM, I went to bed.
The next morning
The deed was done.
- I saw my life flash before my eyes.
- The layer version had gone up to 830.
- My eyes fell on the number of commits.
Now was the time to blame my GitHub action, instead of AWS.
You won’t believe what I saw after I went to check GitHub actions tab:
Yes! This is the real life example of recursion, my friend.
Why it happened?
If you are still unable to see the problem, here is the much awaited explanation.
Remember, the action was set to run on every
push the action was
publishing a new Lambda layer every time it ran.
Every time we publish a new layer, we update the README.
After updating the README, we push it to the repo.
What happens when we push it to the repo?
It creates another Lambda layer version.
And it kept pushing and updating again and again and again …
Until I removed the
Cleaning the mess it made
Despite my accidental adventure resulting in 830 commits and Lambda versions, I was determined to rectify the situation and clean up the mess I had created. To do this, I turned to a powerful Git command called
rebase, which allowed me to rewrite the commit history interactively.
Here’s how I saved the day using
git rebase -i and a little help from Neovim:
First, I identified the commit hash just before the unintended commits began. Let’s call this commit
I then executed the following command: git
rebase -i commit_hash. This command opened an interactive rebase session, listing all the commits from the specified hash until the latest commit in my text editor, Neovim.
In Neovim, I carefully reviewed the list of 830 commits. I marked the majority of them (800+) as ‘drop’, leaving only the essential commits marked as ‘pick’.
After saving and closing the Neovim editor, Git performed the rebase operation, effectively removing the unwanted commits from the repository history.
Finally, I force-pushed the updated commit history to the remote repository using
git push -f. This step overwrote the remote history with the cleaned-up version, removing the unwanted commits permanently.
Checkout the full code on GitHub.
I hope you enjoyed reading my rollercoaster of an article. If you’d like me to take on more such adventures, share this article with your tech-savvy friends.
‘Til next time!