My Accidental Adventure with 830 Commits and Lambda Versions

Just yesterday, I wanted to automate creating Lambda layers and publishing them to multiple regions using GitHub Actions.

Little did I know, I was about to make an unexpected and amusing mistake that would leave me with 830 commits and Lambda versions.

In this blog post, I’ll share the story of how it went down, and provide some code examples to help you avoid making the same mistake.

The Plan:

The goal was to create a GitHub Action that would handle the following tasks:

Create a Lambda layer.
Publish the layer to multiple AWS regions.
Update the README file with the new layer ARNs.

The Mistake:

Manually running the GitHub action to keep debugging my action.yml was a hassle.

So, to test my GitHub Action on every update, I added an ‘on-push’ trigger.

At the end of the workflow, after creating a Lambda layer:

I update README with a new layer version ARN
Commit changes to README.md
Push changes back to the main repo

Here’s a simplified version of the GitHub Actions workflow that caused the issue:

name: Publish Lambda Layer

on:
  push:

jobs:
  publish-layer:
    runs-on: ubuntu-latest

    steps:
    # ... (Steps to set up AWS credentials, create Lambda layer, and publish it to multiple regions)

    - name: Update README
      run: |
        # ... (Code to update the README file with the new layer ARNs)

    - name: Commit changes
      run: |
        # ... config git username and email
        git add README.md
        git commit -m "Update README with new layer ARNs"
        git push

After I fixed all the issues by staying up till 5 AM, I went to bed.

The next morning

The deed was done.

I saw my life flash before my eyes.
The layer version had gone up to 830.
My eyes fell on the number of commits.

Now was the time to blame my GitHub action, instead of AWS.

You won’t believe what I saw after I went to check GitHub actions tab:

Yes! This is the real life example of recursion, my friend.

Why it happened?

If you are still unable to see the problem, here is the much awaited explanation.

Remember, the action was set to run on every push the action was publishing a new Lambda layer every time it ran.

Every time we publish a new layer, we update the README.

After updating the README, we push it to the repo.

What happens when we push it to the repo?

It creates another Lambda layer version.

And it kept pushing and updating again and again and again …

Until I removed the on-push trigger.

Cleaning the mess it made

Despite my accidental adventure resulting in 830 commits and Lambda versions, I was determined to rectify the situation and clean up the mess I had created. To do this, I turned to a powerful Git command called rebase, which allowed me to rewrite the commit history interactively.

Here’s how I saved the day using git rebase -i and a little help from Neovim:

First, I identified the commit hash just before the unintended commits began. Let’s call this commit commit_hash.
I then executed the following command: git rebase -i commit_hash. This command opened an interactive rebase session, listing all the commits from the specified hash until the latest commit in my text editor, Neovim.
In Neovim, I carefully reviewed the list of 830 commits. I marked the majority of them (800+) as ‘drop’, leaving only the essential commits marked as ‘pick’.
After saving and closing the Neovim editor, Git performed the rebase operation, effectively removing the unwanted commits from the repository history.
Finally, I force-pushed the updated commit history to the remote repository using git push -f. This step overwrote the remote history with the cleaned-up version, removing the unwanted commits permanently.

Checkout the full code on GitHub.

I hope you enjoyed reading my rollercoaster of an article. If you’d like me to take on more such adventures, share this article with your tech-savvy friends.

‘Til next time!
~ Shivam