How Beta-Testing Led To A Better Customer Experience

This blog contains excerpts taken from the VTA/Digital Echidna session panel held at Bay Area Drupal Camp 2019. 

Santa Clara Valley Transportation Authority (VTA)'s Administrator of Digital Communications, Cody Kraatz, is the one responsible for integrating digital into VTA's DNA. He focuses on web and mobile product management, social media, community engagement, and strategic communications to enhance and expand VTA's brand.

Kraatz was at BADcamp last week to talk about the new VTA site. 

The idea for new VTA site started when a website usability study he commissioned returned low user-satisfaction rates. Chief complaints included ‘counterintuitive navigational terminologies, confusing page layouts, and high volume of unfiltered content’.

This is the kind of stuff Kraatz loves to dig into and correct. A look at the legacy site’s website analytics helped narrow some areas ripe for improvement: transit data display and consumption, and optimization for mobile users. For example:

• Of 9.8 million pageviews annually, over 70 per cent of pageviews focused on transit services data
• The legacy VTA site was not fully optimized for mobile usage even though of the 4.27 million sessions on VTA.org, 69 per cent were from mobile devices

Closing the Customer Experience Gap

Business Goals

VTA hired Drupal development firm Digital Echidna and engaged Acquia Cloud services, investing in a year-long site redevelopment project that would cross all VTA departments and include input from the public. 

This process included the build of a minimum viable product (MVP) for beta testing, solicitation and collection of customer feedback, and several iteration cycles. The driving factor overall was the goal to redevelop this digital experience for its end-users, and not for internal organizational audiences. 

Kraatz was asked how he measurably moved the needle with customers, resulting in a more positive perception of the VTA brand among customers, and increased satisfaction.

His response: 

"Our legacy website better served its internal audiences than it did primary customers of VTA and failed to support its largest segment of online traffic - mobile users. This time, the site was first and foremost for the end-users of the site. We do endorse a transit app for things an app does best, but still need a top of the line mobile experience for all the other services that VTA offers, not contained within that app."

Initial beta testing on the MVP site helped define the key interactions to inform the build. 

Key Interactions on the Beta Site

• Plan a trip
• Find a schedule and route map
• Get real time information and service alerts
• Find out how much a trip will cost and how to pay
• As a non-rider, find out in simple terms how to start riding

User testing showed an immediate increase in customer satisfaction. 

Iterations

Further beta-testing led to Drupal development agency and partner Digital Echidna to the following iterations:

• A General Transit Feed Specification (GTFS-realtime feed) system mitigates the need to enter updates to schedules manually or in a piece-meal fashion; by consuming service alerts from a standardized GTFS-realtime feed (which VTA's third-party app also consumes) the new website gets alerts to its customers, faster
• Trip planning integration so commuters can plan a start and an endpoint of their trip, and have the information display in a logical and user-friendly manner
• Service Alerts and Emergency Alerts to inform commuters of delays or service outages
• Adding a Project page to communicate all of VTA’s mobility projects - especially timelines, maps, and news relating to congestion management 
• Mapping and trip planning services for bicycle, pedestrian, and local road improvements projects   

In July 2019, the new site absorbed the MVP beta site. 

Available statistics July 18 - 26: 

• 9,555 real-time predictions
• 4,811 trips planned
• 1,075 vendors registered

More than 70 percent of survey respondents rated the new site 3-5 stars on several key metrics, consistent with professionally moderated usability testing. 

While it is true a segment of the user base was not excited about a change from what they’d become accustomed to, the new site received more “extremely easy” ratings and higher ranks for its visual appeal, than did the legacy site. 

Says Kraatz, 

“As frequent users become accustomed to the new site, I expect the average rating to increase from 3-star average to 4 stars (out of five).”

If you are interested in learning more about this project, Acquia products and services, or to see if we can do the same for your organization, please contact Digital Echidna. 

---

Did you enjoy this article? Get more just like it by signing up to receive Digital Echidna’s free e-newsletter, delivered to your inbox every month. Subscribe today.