---
date: 2020-04-09
authors: joshblack
---

# v10.11.0 Release

**Summary**

<!-- What is a one or two-line summary of the event that occurred? -->

When we flipped our v10.11.0 release from `next` to `latest` on npm we received
several reports about builds breaking for IBM users when using the latest
release.

**Impact**

<!-- What was the scope of impact from the event occuring? How many teams were
impacted? -->

Teams that were pulling in the v10.11.0 releases of Carbon were unable to build
their projects successfully. This release seemed to be consumed through
automated dependency resolution in CI for packages that may not pin exact
versions of Carbon. In particular, if a group used the `^` operator it would
accept minor updates as part of semver.

The impact of this was between 5 to 10 teams who identified this issue and
raised in through IBM communications. This issue had the potential to impact all
teams that used a default default theme than `$carbon--theme--white`.

**Root causes**:

<!-- Looking back, what ended up being the main reasons why this event occurred?
-->

The root cause was due to an `@error` expression being applied in the wrong
position for a helper mixin that we used for new code. This new code was helping
us render the Tag component using different colors in different themes. The
`@error` expression was used inside of the loop that determined which color to
use for tag for a given theme. This caused the helper to fail if the first theme
that was checked wasn't a match.

This was not caught as we defined `$carbon--theme--white` as the first theme in
the list and so locally we never ran a custom theme without building in
`$carbon--theme--white` initially and then switching over.

**Detection**

<!-- How did we find out or discover that this event had occurred? -->

At approximately 12:30pm CT we received reports through IBM internal
communications that builds were failing when using the latest version of Carbon.

**Resolution**

<!-- How did we end up addressing this event in order to mitigate impact? -->

After identifying that there was an unknown issue in the build, we rolled back
both `carbon-components` and `carbon-components-react` to their previous known
stable versions.

**Action Items**

<!-- What are the action items that came out of this postmortem? Reference
issues and Pull Requests in the "Bug" column with the appropriate owners -->

| Action item                                                 | Owner      | Bug                                                                             |
| ----------------------------------------------------------- | ---------- | ------------------------------------------------------------------------------- |
| Address the underlying issue with the `@error` expression   | @joshblack | https://github.com/carbon-design-system/carbon/pull/5838                        |
| Identify how to handle custom themes with custom tag colors | @joshblack | https://github.com/carbon-design-system/carbon/pull/5838#issuecomment-611739312 |

## Lessons learned

**What went well**

- We were able to quickly rollback our release to the previous stable version
- We were able to restore broken builds in <1hr from initial reports

**What went wrong**

- We did not effectively review code that was brought into the repo
- We do not have any test harness that could have caught this permutation of
  Sass configuration issues

**Where we got lucky**

## Timeline

2012-04-09 **(all times in UTC)**

- 17:30 Initial reports begain coming in around broken builds
- 17:35 Initial response to reports requesting steps to reproduce
- 18:30 Decision to rollback v10.11.0 for `carbon-components` and
  `carbon-components-react`

## Supporting information

<!-- Any additional information that you might reference earlier on in the
postmortem -->
