VMware Innovates E-Learning and Boosts Accessibility with AWS AI/ML and Professional Services

VMware has long had a reputation for innovation. Founded in 1998, it was one of the first companies to virtualize x86 architecture, and it continues to innovate through its virtualization platforms. Its current and most popular platform, the VMware vSphere Hypervisor, enables the virtualization of practically any x86 or x64 architecture. In addition, VMware helps its customers streamline digital innovation through app modernization and cloud solutions, such as VMware Cloud on AWS, CloudHealth by VMware, and VMware Carbon Black.

To ensure that its field employees, partners, and strategic customers can design and deliver VMware solutions and accelerate their adoption, VMware’s Livefire Solutions group provides advanced technical enablement throughout the world. It offers hands-on architecture and design sessions around virtual cloud networking, digital workspace, intrinsic security, modern apps, and the cloud. These extremely popular deliveries are known for their collaborative nature and interactivity, as they are delivered by engineers and architects and go beyond a traditional training offering.

When attendees started asking for on-demand access to some of the training content to supplement the in-person deliveries, the Livefire Solutions innovation group brainstormed on ways to provide the content in an e-learning format. Given the fast development and addition of new services to the VMware portfolio, they needed to ensure that the training content could be created quickly and stayed up-to-date.

One key requirement for the Livefire Solutions group was the ability to create on-demand training content in a way that was more scalable from a cost perspective than traditional video recording. “VMware has recording studios in different locations, and we would be incurring a lot of nonbusiness-value costs by flying people to these studios,” says Bal Birdy, staff solutions architect in the Livefire Solutions group and product lead for Olive, The Virtual Trainer. “Additionally, we’re a small team, and we have more demand than supply, so a solution would also need to be scalable from a practical perspective.”

A small-scale proof of concept (POC) proved to VMware that learners would be receptive to a Virtual Avatar-based digital learning course developed with a text-to-speech engine. To bring this concept into production, it engaged with the Amazon Web Services (AWS) Professional Services to develop a virtual trainer platform that would enable users to create engaging and immersive experiences.

The AWS Professional Services Spatial Practice team specializes in developing and enabling customers in their augmented reality (AR), virtual reality (VR), and 3D journey using AWS services to simulate highly customized experiences. VMware worked with the Spatial Practice team to develop a production-grade pilot for the virtual trainer. The AWS team was involved in all phases of the project, from conceptualization—where they determined the best approach for scalability, performance, and creative appeal—to development, deployment, and production.

The initial phase of the engagement involved building a narrative that described the visual aesthetic, concepting that aesthetic, and storyboarding the user experience to better understand the art needs from the AWS Professional Services team. Through a series of brainstorming sessions, the team developed and finalized 2D sketches of what the 3D environment should look and feel like and how users would interact with the content.

The concepts informed the engagement’s 3D artist and technical artist on which assets needed to be built, how they should be assembled, and how to best translate the desired look of the 3D scene to a real-time engine. The team built an asset list and began constructing those assets, while technical consultants built the backend features that would power the 3D scene.

All of the completed assets were assembled within the “Digital Content Creation” package and then exported into an open-source format, and imported to the real-time engine, where the framework was layered onto that content. This exercise allowed the customer to view, one-to-one, how the engagement would be delivered, leaving no questions or doubts on the visual appeal or what the final delivered aesthetic would look like.

The team used an open-source 3D engine to present the application on the web and integrated it with Amazon Sumerian hosts—virtual avatars with built-in animation, speech, and behavior for interacting with users. Trainers who write the on-demand courses in Microsoft PowerPoint create the script spoken by the Amazon Sumerian hosts. After adding this script to the notes section of the slides, Amazon Polly, the AWS text-to-speech engine, translates it to actual speech, making it appear as if the Amazon Sumerian hosts are speaking. Amazon Lex powers the conversational interface, where learners can ask questions by typing or speaking aloud and receive a response.  

For the automation, the team uses Amazon Simple Storage Service (Amazon S3) to upload the PowerPoint files, which are then processed using AWS Lambda. The team relies on Amazon Translate, one of the many AI services AWS offers, to provide closed captioning and automated language translation capabilities, enabling the Livefire team to translate on-demand training into the 19 languages spoken by their global audience. 

The team integrated the solution with the AWS Cloud Development Kit (AWS CDK), which facilitates a fully automated CI/CD pipeline for fast updates and push and pulls between the staging and production environments. Finally, they distributed the on-demand training using Amazon CloudFront to ensure low latency globally for the highly interactive web experience. 

VMware achieved considerable savings with the virtual trainer solution. Content creation now costs VMware 70 percent less and is 80 percent faster than with traditional video recording methods for the lifetime of the content. Localization opportunities have become more accessible for the organization. In addition, updates that traditionally would have taken weeks or days now take hours or minutes.

The solution also promotes broader inclusion and increases operational value. “Because we're programmatically creating our content, we can now reach differently abled partners and customers,” says Birdy. “We’re realizing great benefits in terms of inclusivity and enablement. Amazon Translate creates closed captioning, which supports those who are hard of hearing, and we're currently determining how to use Amazon Transcribe to offer voice control of a scene to support motor-control and visual impairment groups and other communities.”

Groups throughout VMware can use the scalable, distributed solution that the AWS and VMware teams developed. “There's an opportunity for us to leverage the solution for other groups, especially in the realm of localization,” says Tom MacKay, senior director of global strategic programs at VMware. “AWS Translate could be a quick and very cost-effective way for us to get other types of content into our global markets.”