Crunchyroll didn't have any real way of testing out ideas on the fly, and so the company wanted a tool that could be used to do just that. Having an A/B testing tool for the various new features that are being developed for Crunchyroll 2.0 was a must-have for the future success of Crunchyroll. Features like newly created paid tiers for US and International, as well as testing marketing placement for e-commerce, Crunchyroll Expo, as well as other more in depth experiments were all being requested by various teams, and therefore needed to be part of Config Delta.
Config Delta is a robust A/B testing tool developed for the many different facets of the Crunchyroll platform. Its main focus is to help facilitate the retention of users through various funnels, such as supporting marketing tests, audience development needs, as well as business development requests. Through Config Delta, users are able to create in depth experiments that they are then able to test on free, paid, or anonymous users, on all of the different Crunchyroll platforms.
Being a more technical rounded designer, I was asked to help design Config Delta in what was a very quick turnaround. In around four months time, we went from kickoff to first release, ready for the introduction of our new Crunchyroll paid tier lists.
Crunchyroll+ paid tiers
The two biggest constraints that I had when designing this tool were time, and resources. The turnaround time from concept to release was around four months, to be exact. With that in mind, I had to hit the ground running, and start wireframing up flows as well as being in constant communication with Product Managers. I had two great engineers that I worked with, and so resources for this project were limited to me making sure that I stayed within a framework that both engineers could understand and work within. That meant that design work fell in-line within the Material-UI framework.
We started off with an initial kickoff where stakeholders gave a quick overview of the project. Over the next couple of weeks, I sat down with PMs, and we went and created quick whiteboard user flows so that I would have a better understanding of the complexities of Config Delta.
Quick user journey for creation of a new experiment within Config Delta
User journey for what Admin/Contributors/Users see when viewing an experiment within Config Delta
After figuring out and identifying pain points within the user journeys, I went ahead and started to create wireframes of those journeys. Once again, there was a healthy amount of back and forth communication with the devs and stakeholders involved in this project, so I never felt like the project was going over my head. Through an iterative process, I was able to hone in on the most important features of Config Delta, and create wireframes that satisfied both stakeholders, PMs and engineers.