What Is Regression Testing?
Regression testing verifies that new code changes have not broken existing functionality. Every bug fix, feature addition, or refactoring carries the risk of unintended side effects. Regression testing catches these before users do.
In mobile development, regression risk is amplified by:
- Platform updates - A new iOS or Android version may change behavior your app depends on
- Third-party SDK updates - Updating one library can cause unexpected interactions with others
- Feature flags - Different user segments see different code paths
- Device diversity - A fix for one device class can break another
The Regression Testing Strategy
Automated Regression Suite
Your automated regression suite should cover every critical user flow. This is not the same as your full E2E test suite. Regression tests focus specifically on functionality that has been verified as working and must continue to work.
What to include:
- Authentication (login, logout, token refresh, session expiry)
- Core feature loop (the primary reason users open your app)
- Payment flows (purchase, subscription, restore purchases)
- Data persistence (create, read, update, delete operations)
- Navigation (deep links, push notification taps, widget taps)
- Error handling (offline state, API errors, invalid input)
How often to run:
- On every pull request: smoke tests (top 5-10 critical paths)
- On merge to main: full regression suite
- Before every release: full regression suite on multiple devices
Manual Regression Checklist
Automation cannot catch everything. Maintain a manual checklist for scenarios that are hard to automate:
- Visual consistency (fonts, colors, spacing look correct)
- Animation smoothness (transitions, loading indicators)
- Haptic feedback (where applicable)
- System integration (sharing, deep linking from external apps)
- Accessibility (VoiceOver/TalkBack flow-through)
- Dark mode and light mode appearance
Update this checklist whenever a new bug is found and fixed. If a user reports a bug, add a test for that exact scenario to prevent it from recurring.
Release QA Process
Phase 1: Code Freeze
Set a code freeze date 3-5 days before the planned release. After code freeze, only bug fixes are merged. No new features, no refactoring, no "small improvements."
This creates a stable branch that QA can test without the ground shifting beneath them.
Phase 2: Automated Testing
Run your complete test suite:
- Unit tests - All must pass. Zero tolerance for failures.
- Integration tests - Verify API contracts, database operations, and service interactions.
- E2E tests - Run on at least 3 device configurations (small iOS, large iOS, mid-range Android as a minimum).
- Performance benchmarks - Compare startup time, memory usage, and frame rates against the previous release.
Phase 3: Manual QA
Walk through the manual regression checklist. Test on real devices, not just simulators. Pay special attention to:
- New features in this release (fresh eyes, not the developer who built it)
- Fixed bugs from this release (verify the fix, check for side effects)
- Areas adjacent to changes (if you modified the checkout flow, also test the cart and order history)
Phase 4: Stakeholder Sign-Off
Before submitting to the store, get explicit approval from:
- Engineering lead (technical readiness)
- Product owner (feature completeness)
- QA lead (quality verification)
Document the sign-off with a timestamp. This creates accountability and prevents "I thought someone else tested that" scenarios.
Staged Rollouts
Never release to 100% of users on day one. Both Apple and Google support staged rollouts.
Google Play Staged Rollout
Google Play lets you release to a percentage of users and increase gradually:
| Day | Rollout % | Action |
|---|---|---|
| 1 | 1% | Monitor crash rate, ANR rate, and user feedback |
| 2 | 5% | Review metrics, expand if stable |
| 3-4 | 25% | Watch for device-specific issues at scale |
| 5-6 | 50% | Near full rollout, final monitoring |
| 7 | 100% | Full release |
If crash-free rate drops below 99% at any stage, halt the rollout and investigate.
App Store Phased Release (iOS)
Apple offers a 7-day phased release that automatically distributes to increasing percentages:
- Day 1: 1%
- Day 2: 2%
- Day 3: 5%
- Day 4: 10%
- Day 5: 20%
- Day 6: 50%
- Day 7: 100%
You can pause the phased release at any point if issues are detected. Note that users who manually check for updates will still get the new version immediately.
Release Monitoring
After the release goes live, actively monitor:
First 24 hours (Critical window):
- Crash-free rate in Firebase Crashlytics or Sentry
- App Store / Play Store reviews (filter by current version)
- Support ticket volume
- Key business metrics (conversion rate, session length, retention)
First 7 days:
- Compare all metrics against the previous version
- Monitor for slow-building issues (memory leaks that accumulate over days)
- Check for OS-specific problems reported by users
Setting Up Alerts
Configure automatic alerts for:
- Crash-free rate drops below threshold
- New crash type affecting more than 0.1% of users
- ANR rate exceeds Google Play threshold (0.47%)
- Significant drop in daily active users or revenue
Rollback Planning
Sometimes a release needs to be pulled. Have a plan ready before you need it.
Android: You can halt the staged rollout and revert to the previous version. Users who already updated keep the new version unless they uninstall and reinstall.
iOS: You cannot "un-release" an update, but you can remove the app from sale temporarily or submit an expedited review for a hotfix. Apple's expedited review typically processes within 24 hours for critical issues.
Feature flags save rollbacks. If the problematic feature is behind a feature flag, you can disable it remotely without any store update. This is the strongest argument for using feature flags in production.
The Release Checklist
Use this checklist before every store submission:
Code Quality:
- All automated tests pass
- No critical or high-severity bugs open
- Code reviewed and approved
- No debug logging or test credentials in production build
Store Metadata:
- Version number incremented
- Release notes written and reviewed
- Screenshots updated (if UI changed)
- Privacy declarations current
Technical:
- dSYMs / mapping files uploaded for crash symbolication
- Analytics events verified for new features
- Feature flags configured correctly for production
- API backward compatibility confirmed
Distribution:
- Staged rollout configured (not 100% on day one)
- Monitoring dashboards set up
- On-call engineer identified for the release window
- Rollback plan documented