Table of Contents
Abstract
Project Overview
.The Need
.The Project: Addressing the Need
.Key Founding Partnerships
.Curriculum Design
.Plan for Scale
.Evaluation
Lessons Learned
.Employer Engagement
.Curriculum Design
.Instruction Delivery
.Technology Assessment and Development
.Expanding for Scale
.Evaluation
Training Impact: A Word for Practitioners
Acknowledgments
Authors and Contributors
Notes
Abstract
This report analyzes the design, implementation and growth of Skills and Opportunity for the New American Workforce, an industry-contextualized English language training program that offers a bridge between workers seeking to boost English proficiency and employers seeking to enhance workers’ skills and retain and promote a diverse workforce. This project is a first-of-its-kind initiative of the National Immigration Forum, launched with generous support from the Walmart Foundation. A blended course comprising 40% face-to-face instruction and 60% online learning, this worksite offering has proved to be a highly effective solution for limited-English-proficient (LEP) workers across various industries looking to build workplace skills and open career pathways.
The goal of this report is to share lessons learned and promising practices identified during various phases of the project with the broader workforce development, immigration and adult education fields. We examined employer engagement, curriculum design, instruction delivery, technology tool development and program evaluation to gain insights in the following strategic areas:
- Enlisting key experts and innovators as project partners.
- Expanding our curriculum portfolio to new industry contexts and developing additional instruction models.
- Refining our technology capabilities.
- Incorporating feedback from corporate partners.
- Iterating to support growth.
- Developing a financial model that ensures growth and sustainability.
- Adapting our external evaluation process to large-scale implementation.
Project Overview
The Need
Julia is a deli associate who is afraid of answering customer calls. Gerardo is an assistant meat manager who aspires to be a manager but lacks confidence in his oral English ability and digital skills. William is a ride-sharing driver who wants to apply for an engineering job but is not satisfied with his conversational skills. Mariano is a grocery clerk who needs to sit by someone who speaks English in staff meetings to make sure he understands and is understood. Teresa is a baker who feels embarrassed to tell her customers that she doesn’t speak English. What do Julia, Gerardo, William, Mariano and Teresa have in common? A strong desire to hone their English skills so they can achieve their goals: be better employees, move up the career ladder and ultimately achieve upward mobility.
An estimated 1 out of every 10 working-age adults in the United States has limited English language skills,[1] holding many back from advancing to more-skilled and better-paying jobs and hindering the competitiveness of many businesses.
Many of these employees, while experienced and talented, lack language skills and are given limited responsibilities on the jobsite, a situation that severely limits their chances at advancement and economic stability.
The retail sector; adjacent sectors such as manufacturing, transportation and warehousing; and accommodation and food service employ more than 6 million LEP workers, or approximately 43% of all LEP workers.[2]
Research shows that employers also grapple with the challenge of filling middle-skilled jobs that require a higher level of skills[3] than entry-level employment, and involve high employee turnover costs—replacing the average middle-skilled worker costs an employer 20% of that worker’s salary.[4]
Studies have shown that increased English language proficiency leads to many positive outcomes benefiting work operations, such as increased employee satisfaction and retention, increased wages and job promotions and a more skilled and capable workforce.
The Project: Addressing the Need
The National Immigration Forum (Forum), with generous funding from the Walmart Foundation, created Skills and Opportunity for the New American Workforce (SONAW), a scalable, industry-contextualized English language training that helps companies build stronger career pathways for their employees.
The initiative is part of the of the Forum’s New American Workforce program, which engages hundreds of innovative companies that are in the forefront as strategic diversity leaders, who recognize and celebrate the value of their immigrant workforce.
Key Founding Partnerships
The Walmart Foundation, as part of its ongoing commitment to create economic mobility for the retail workforce, supported the Forum’s effort to design and pilot blended, retail-contextualized English training to help incumbent workers develop their worksite vocabulary and communication skills. For this work, the Forum enlisted the help of two close partners, Miami Dade College and the Community College Consortium for Immigrant Education (CCCIE). In its first two years, SONAW successfully trained nearly 1,000 retail workers employed by Publix in Miami, Kroger in Houston and Whole Foods in the New York City area and Houston. The Forum also engaged local instruction partners across the country to deliver the training in each region at the worksite or on campus. These are prestigious community colleges, public school systems and community-based organizations with a tradition of providing high-quality ESL instruction in their communities, such as Lone Star College (Texas), Westchester Community College (New York), La Guardia Community College (New York), Bergen County Technical Schools (New Jersey), Baker Ripley (Texas), City Colleges of Chicago (Illinois) and Highline College (Washington).
Curriculum Design
The training is an industry-specific blended worksite course, consisting of 40% in-person instruction and 60% self-paced online modules accessible on desktop and mobile devices. With expert practitioners from Miami Dade College and the CCCIE, the Forum developed an original curriculum, online learning tools and an instruction infrastructure to support the integration of LEP employees into a company’s workforce, aided by sector-specific vocabulary related to customer service, safety, technology and critical soft skills such as team communication. The initial curriculum was designed to serve front-line LEP retail workers.
With a goal of scaling the project to be a solution for the nation’s LEP workers, the project team made important training enhancements following the pilot year that include streamlining and further customizing the curriculum to better serve learners and operations, and optimizing online learning for mobile platforms. The curriculum has since been adapted, with input from partner curriculum experts from World Education and CCCIE, to be a solution for workers in the sectors included in the chart below.
Plan for Scale
Encouraged by the outstanding training results in the first two years of the project, the Forum conducted a planning process to look at the viability of taking SONAW to scale, providing more employers across the nation and various industries with a cutting-edge English language learning program that increases skills and improves operational productivity. The plan for scale included:
- Assessing our project evaluation and creating a business plan.
- Adapting our instruction infrastructure and developing new models of training delivery to better serve the needs of employers. Our blended instruction models include:
- Community College — In-person instruction is offered by community college instructors (at worksite or campus) with parallel self-paced online learning.
- Corporate Trainer — supported and trained by Forum educators to offer in-person instruction, alongside self-paced online learning.
- Remote Digital — a fully remote model in which the face-to-face component is delivered by a community college instructor via live webinar integrated with our learning management system (LMS), in addition to self-paced online learning.
- Refining and expanding our curriculum portfolio to include new industry contexts and new modes of delivery.
- Determining and implementing improvements needed for our technology tools. This includes creating our own highly customized LMS hosted on the open-source LMS platform Moodle, setup and implementation of a virtual classroom for the fully remote model and migrating our online learning to the new platform.
- Expanding our instruction network by adding new community college partners across the country.
- Creating a diversified funding model, including exploring additional revenue streams, across public and private sector opportunities and models to ensure project growth and sustainability. These include expanded employer investment and philanthropic support.
Evaluation
In order to ensure the validity of SONAW as a tool that provides a solution for LEP employees of the nation’s largest companies, the project team developed a well-rounded plan to assess the success of the project from the concept design phase to the implementation and completion of training.
The Forum leads internal formative and summative evaluations of our processes, guided by our project work plan. The Forum collects quantitative and qualitative project data, which we record in a database customizable to tracking and reporting needs — crucial to maintaining accurate and up-to-date information. The database is being accessed by all project partners, who have specific roles in maintaining it. The Forum has full control over the database and provides training to others who need to access it.
Most important, the Forum contracted RTI International, which since the launch of this project has served as our external evaluator and has issued a final, independent evaluation report after each round of training. Our external evaluation process includes:
- Measurement of learning outcomes through standard and custom pre- and post-testing, a rubric for assessing industry-specific soft skills and tracking course completion, grades, in-person attendance and participation in online learning.
- Measurement of employment outcomes and business ROI through manager surveys (productivity, communication on the job, confidence and morale, and customer satisfaction) and participant surveys (job retention, promotion, wage increases and employee satisfaction).
- Analysis of the pedagogical models and curriculum suitability through weekly instructor surveys, classroom observation and developer and partner interviews.
Lessons Learned
Employer Engagement
The Forum brings a depth of experience to this project in the area of onsite services to support immigrant workers. This experience includes partnering with more than 400 employers to offer tailored services that meet the demands of operations and individual workers. With the knowledge acquired in this effort, we approached the development and implementation of this contextualized training mindful of the needs of not only workers, but also employers and their operational considerations.
Contextualized and Customized Curricula
We have learned that the best way to ensure that we meet the operational needs of our company partner is to find key ways to incorporate their engagement while not straining their capacity. One of the most important ways that we incorporated employer engagement in this training was through the process of customizing the curricula and training for a specific company partner. Our blended curricula are customized for each industry and job function, but also for each employer. As we customize the training for a particular work site, we partner with the company leadership to incorporate key training materials and company-specific tools and vocabulary, so that employers are able to solve challenges they encounter with LEP staff.
Whether the challenges occur with delivering customer service, following safety procedures or engaging company technology tools, our customization process allows for a tailored training experience that meets both individual and company needs, and the employer participation in the process is invaluable in achieving this result.
Since our partners focus on enhancing the skills of their existing workforce, they consistently emphasize that contextualized language training plays a crucial role in increasing the pool of candidates for supervisory or managerial positions. Moreover, these programs strengthen their recruitment strategy by letting them present a more attractive benefits package that includes language training for employees.
“If an associate has the ability to improve their English skills through this program, can they be considered for the next level promotion? Absolutely! […] It’s a goal for us to promote from within.” — Deanna David, Talent Manager with the Kroger Houston Division
Logistics Lead to Success
Another important way that we incorporate employer engagement is through planning training logistics specific to the work site. We work from the base of our experience to offer a training plan that takes into account industry-specific needs, but also ask the employer to advise us on the specific details that will ensure the training is successful. These considerations include the timing and scheduling of training, in-person versus remote instruction needs and location accessibility. The more that operational needs can be considered in the design and implementation of the training, the more likely the training will be a success.
In addition to considering the operational needs of the employer, we educate the employer on the specific needs of the immigrant workforce and encourage them to take measures to ensure these employees feel supported and can succeed in the training. These recommendations include measures such as widely publicizing the opportunity in employees’ first language, providing multiple schedule and location opportunities and ensuring that working schedules are maintained for the entire course of the training, among others.
Evaluation
One of the most important aspects of employer engagement we incorporated as we developed and scaled the training was including the participants’ supervising managers in the project evaluation process. We prepared the managers by creating resources for them, such as a video presentation of the program that emphasized the importance of encouraging, supporting and constantly assessing employee participants during their training. The video also laid out the areas of competence the training sought to offer in improving employees’ performance. This type of involvement by managers and other supervisory staff also ensures that employees’ English language learning is reinforced on the job.
At the conclusion of the training, managers are surveyed on the gains they observed in their staff and in their operational outcomes as a result of the training. We are very pleased that over 90% of the managers surveyed observed moderate or significant progress with employees’ understanding of safety guidelines and processes, morale and job satisfaction, and confidence on the job.
Motivation, Accountability, Recognition and Celebration
Perhaps one of the greatest keys to training success, however, is the way that we have incorporated employer leadership in the process of motivating, engaging and recognizing employees throughout the training process. Nationally, ESL classes have a course retention rate of 30%, on average.[5] We have been able consistently to achieve a course retention rate of over 80% across employers, industries and instructional models. We believe that a major factor is that employer leadership is involved in encouraging employees toward success. We invite leadership to welcome students at the beginning of training and to recognize them with a work site-based graduation, in front of an audience of their peers.
In addition to ensuring the community college instruction partner awards them with a certificate, we encourage companies to do the same. Perhaps most important, we encourage employers to recognize this accomplishment when the employee seeks or is considered for a promotion. By adopting these practices, employers build much-needed employee job retention that translates to increases in productivity and business competitiveness.
In the words of our employer partners…
“Our industry can have a high turnover rate but in the two years of the project, we haven’t lost a single associate who participated. We’ve even seen many associates gain the confidence and skills needed to apply for and receive promotions.” — Mark Wilkins, Regional Trainer with Whole Foods Market
Curriculum design
One of the most important lessons from our pilot program was that the curriculum design process, as well as its adaptation to new sectors and modes of delivery, requires a close collaboration of the Forum’s project team with experts in curriculum development and education technology. We created a work flow that allowed us to leverage each other’s expertise and have a strong peer-review process illustrated by the chart below.
This highly collaborative process guarantees that the curriculum is highly customized, responds to the main learner challenges and meets the training goals.
Another critical lesson learned from the curriculum design process was the need to develop a strong curriculum base and set of tools that can be easily adapted to different sectors and modalities of instruction, particularly in the scale context. For example, in the past year we were able to contextualize our curriculum for four new industries, with another two on the horizon. We were also able to adjust our face-to-face instruction materials for delivery both in person and virtually, via a live webinar.
Instruction Delivery
Contributed by CCCIE
Community College Partnerships: Key Lessons
The Forum’s partnership with community colleges has been key to the successful launch and evolution of SONAW. Many of these partnerships were facilitated by our partner CCCIE, a national network comprising more than 50 exceptional community colleges and other organizations that have joined forces to increase educational opportunities for immigrant students. The CCCIE is a national voice and vehicle for exchange of best practices in immigrant education, and its Blue Ribbon Panel leverages the technical expertise of educators, administrators and researchers to inform the development of programs serving immigrants in various sectors. The Forum’s collaboration with these partners has provided a diversity of perspectives, competencies and a shared commitment to the project’s goals that translated into training excellence and provided the specialized expertise and the validity of accreditation that employers value.[6]
To learn more about the Forum’s partnership with community colleges supporting this initiative, read: Working Together: How Community Colleges and Their Partners Help Immigrants Succeed, a book edited by our partners at CCCIE Teresita Wisell and Jill Casner-Lotto.
These assets were used throughout the process of designing an industry-contextualized English curriculum using online, classroom and work site training methods; in developing technology tools and a user-friendly interface for the online learning platform, and in building a strong instruction infrastructure. CCCIE has played an important role in disseminating the lessons we learned in this process and the results of the project with educator stakeholders and the community college sector as a whole.
The project has successfully leveraged community college resources, including — most important — the participation of highly qualified and trained ESL instructors. The value of having a trained instructor in the room is critical in a work site-based program where students are fitting classes in between their work schedules.
The online component is a valuable complement but not a substitute for classroom instruction. The students accessed the modules on their own time and at their own pace during the training period, which increased their interaction with the job-related language content and their exposure to technology. But even with this virtual, self-paced work, the instructors’ role has been critical: They provide students with guidance and monitor their progress in the completion of online exercises and quizzes. Additionally, community college and adult education campuses offer valuable resources, including use of computer labs as well as administrative expertise and support in program implementation and troubleshooting.
Beyond improving students’ English language and job-related skills, an important goal of the SONAW model has been to help students build self-confidence and self-advocacy skills as their English language proficiency improves. A related and somewhat unexpected outcome of our pilot was whetting the learners’ appetite for continued learning of English and other training at the colleges or adult schools, which our employer partners found valuable.
Recognizing the need for expanding our network of instruction partners, as part of the planning for scale, the Forum worked with CCCIE and consulted with the League for Innovation in the Community College to map member colleges who could potentially join the project and deliver training in their areas. When tapping into the community college networks, in addition to experience with blended workplace ESL instruction, we look for demonstrated executive-level commitment and innovation, an interest in lending expertise to pilot and implement our various instruction models, entrepreneurial leadership and a strong infrastructure that supports contract training services with business and industry. As the project continues to expand to serve new sectors and employers in various parts of the country, the partners will work with a larger pool of community colleges ready to deliver this groundbreaking, high-quality programming. This could help colleges expand their influence by adopting these innovative, technology-based models that will allow them to serve a dramatically larger number of LEP workers.
Training and Instruction Delivery
The instructor “Community of Practice” meetings are a highly effective forum for discussing instructional challenges, sharing best and promising practices and providing timely instructor feedback. These regularly scheduled calls (in some cases combined with visual presentations) allowed instructors and facilitators to discuss a wide range of issues as they emerged and provide feedback in a timely manner, allowing for continuous improvement. Issues included training delivery, curriculum, teaching techniques, technology tools and administrative processes. The meetings also helped build a sense of camaraderie among instructors as they shared common challenges and brainstormed solutions.
Qualified ESL instructors who embrace new teaching models are key to successful training. The project team looked to recruit creative and nimble instructors who are comfortable teaching in a nontraditional off-campus classroom environment that entails highly dynamic, multilevel lessons with a focus on oral communication and job competencies. We were pleased to find many across the country ready to use their resourcefulness, creativity and enthusiasm to address the challenges that emerge in this kind of classroom environment, especially in responding to learning needs and strengths of students at various proficiency levels. Our highly engaged instructors use “outside-the-box” teaching strategies to adapt the curriculum to fit learners’ needs, including offering extra help for lower-level learners while challenging higher-level learners. Examples of such strategies include peer-led activities, role play and workplace-inspired dialogs, listening exercises, brainstorming solutions to real-life challenges on the job and communication etiquette.
Flexibility in community college partners with planning course logistics is imperative for training success. Colleges typically plan a course over a full semester, while employers schedule their employees’ shifts a week or two out, which makes coordinating schedules difficult. The partners have demonstrated creativity in reallocating resources to plan SONAW courses in nontraditional settings, presenting them at a pace based on the employer’s needs.
As the project expanded and served more workers and employers, the team worked to fully digitize our instruction repository, including the train-the-trainer module. This process made it highly portable, so it can be easily shared and uploaded to various learning platforms with minimal use of resources. Going forward, our instructor training is offered exclusively via webinar to increase efficiency and minimize costs.
Curriculum Design, Learning Challenges and Solutions
Online learning is on students’ own time, but “behind-the-scenes” instructor guidance and assistance is essential in increasing students’ success with the technology. Instructors assisted students, many of whom are not digitally literate, with logging on to the LMS platform and learning to navigate the online modules. This was especially true in cases where trainees didn’t own a computer and participated in online learning exclusively on a smartphone. This kind of assistance continued throughout the course, with instructors monitoring students’ weekly activities online, tracking quiz scores and topics that needed reinforcement. When campus computer labs were available, instructors utilized them before or after class to help students troubleshoot.
The curriculum design had to take into consideration the learners’ mix of English proficiency levels, including various abilities in listening, understanding, reading and speaking, and encouraged instructors to employ their own strategies to help learners hone their language skills based on their needs. In streamlining the curriculum to better serve the needs of our trainees, the developers included a fair amount of work in pairs or groups, allowing instructors to team up lower-level students with higher-level ones. There is a strong sense of camaraderie and collegiality among students, and heterogeneous groups allow more advanced English speakers to serve as role models for those at beginning levels, highly benefiting both groups.
The remote class pilots demonstrated success while allowing for insights on how best to incorporate this instructional model. The instructors followed a redesigned curriculum to fit the virtual classroom, while ensuring that learners with no experience in using a webinar tool felt comfortable and fully participated. They were able to create a positive learning environment and establish good rapport with their students who connected remotely. One of the most important lessons from the pilot was to be proactive about providing support to remote students. In addition to a user guide and technology orientation, having live support available before, during and after the remote sessions is critical for successfully bringing learners on board and retaining them. In spite of the students’ learning curves observed when they used the technology tools, their commitment to learning and growth in language in the fully remote pilot were comparable to those of fellow trainees using our original model with 40% in-person instruction, demonstrating that we retained training efficacy with this new mode of delivery.
Participation in Program Evaluation
The project offers multiple sources for program evaluation, such as the ongoing instructor feedback, the customized pre- and post-program student assessment test, various standardized tests used by the colleges and adult schools, and employer surveys.
In this type of work-based, contextualized ESL program, in addition to measuring students’ gains in English language skills, ultimately one of the most valuable measures of success is the students’ improvement in “soft skills,” such as the ability to interact with customers, team members and their supervisors. The SONAW project partners developed a robust rubric, the Participant’s Progress Assessment, used by instructors to track students’ attendance and measure their improvement in various work-related competencies as demonstrated in class through role plays, conversations and responses to instructors’ prompts. For example, instructors measured gains in students’ confidence, comfort and ease in speaking English, and the ability to use English in various work-related conversations, such as “using language to greet and assist customers” and “ask[ing] for information, details and explanations during a conversation.”
Along the way, we encountered the need to select one standardized assessment tool to measure students’ English language gains, since different community colleges and adult education schools use different tests. We consulted with our external evaluator and several experts in the field and learned that it is most effective to work with various standardized tests and assessment processes that community college partners already use, rather than require all to implement a certain standardized test. This significantly reduces costs and represents a reliable, consistent way to measure results across the program sites.
Benefits Outweigh Challenges
The key lessons we’ve shared demonstrate the value of community colleges’ participation in the SONAW project. In turn, community colleges on our Blue Ribbon Panel[7] and others have cited the benefits from their participation in the project, with an opportunity to expand their instructional services beyond their geographic areas through remote classrooms and strengthen their curricula by exploring new and innovative ESL hybrid models. For example, both City Colleges of Chicago and Westchester Community College, SONAW partners, have expressed interest in ways to incorporate ESL courses that blend online and classroom learning into their continuing education curriculum, as well as learning how to better engage employer partners and increase employer awareness of the benefits of contextualized ESL classes. Noncredit continuing education departments offer a particularly nimble space for exploring new, nontraditional, technology-based ESL models. Additionally, the SONAW initiative comes at a time of heightened interest and movement toward industry-recognized, shorter-term credentials that will help address the growing middle-skills gap.
By offering employees the opportunity to take free classes during the employers’ hours of operation or at times adjacent to work schedules while continuing to access convenient online learning on their own time, anywhere, the model also helped employee-students solve challenges ESL students typically face, including availability, affordability and scheduling of classes, work schedules, unfamiliarity with college systems and enrollment processes, and lack of child care and transportation.
To ensure sustainability, the SONAW project has moved from a fully philanthropically subsidized model to one that combines employer partner investment with philanthropic grants.
As the partners continue to scale and adapt the curriculum, instructor training and repository of tools to work with employers in a wider pool of retail sectors, as well as new employers in different sectors — and as the funding model evolves — a critical challenge ahead for community colleges will be balancing the necessary flexibility, creativity and fast-paced planning required, while also continuing to ensure the highest quality in the content and delivery of contextualized ESL instruction and achieve the desired outcomes.
Nonetheless, the novel and exciting aspects of the SONAW project outweigh the challenges. The SONAW model has improved employees’ English language proficiency, communication, customer service and technology skills and has led to increased employee promotions, job retention and store productivity. The partners have demonstrated their shared commitment to the project goals and have developed the communication and planning processes to work strategically with each other and within their own organizations.
Technology Assessment and Development
Contributed by World Education EdTech Center
In 2015, the Forum partnered with Miami Dade College to develop and implement the online learning component for SONAW. In the pilot year, the digital modules created were made available to learners through the Miami Dade College Blackboard Learning Management System (LMS).
The original design offered only a desktop-accessible platform for online learning, which resulted in lower-than-expected participation online. A key lesson from our pilot was the need to increase accessibility of online learning by making the platform accessible on smartphones and tablets, in addition to computers. The project’s evaluation showed that almost half of our participants (46%) had low or no computer literacy, but 100% indicated they use a smartphone or tablet to communicate with friends through text messaging and social media. In addition, abundant research suggested that Hispanics are the most avid smartphone users in the U.S. Among Hispanics, smartphone ownership is 71%,[8] higher than any other racial or ethnic group. They are on their phones for more than 14 hours a week[9] and close to half have used a mobile device to take an online class or look up education content.[10] Therefore, the Forum set out to redesign our digital modules to be smartphone- and tablet-accessible and added a digital literacy module to our curriculum. The online learning modules feature multimedia content that bundles together instructional activities, learning games and quizzes. By integrating digital literacy with contextualized English instruction, the Forum adopted a best practice in adult education that builds vocabulary, provides a natural context to practice language and develops an essential job skill. As a result, in 2019, 56% of learners working in retail and 100% of learners in ride-sharing felt more comfortable using technology on the job. [11]
As the project team set out to adapt the technology for implementation at scale, the Forum collected and analyzed participant feedback from course evaluations, participation reports and other program data, which indicated that the digital learning platform and experience for users could improve. As the Forum looked to expand the initiative to new industries, pilot new blended learning models and scale its program nationwide, the team took an intentional, strategic approach to identifying needs by:
- Conducting more research to better identify learners’ access to technology, including use of smartphones and related behaviors, and
- Commissioning an independent audit of the digital learning content and technology used to host and deliver that content.
The Forum selected the World Education EdTech Center to conduct the audit of the current LMS and online content used in the project. The audit examined a number of areas and documented key observations, recommendations and opportunities to enhance the online learning component of the program. Project staff from the EdTech Center and the Forum team then worked closely together in 2018 to identify key requirements, considerations and other criteria for the new platform to host, deliver and manage its blended learning programs. Additional evaluation criteria[12] and resources were also used to facilitate the process. The team identified the following main requirements:
- Ease of management: Ability for the Forum to transition to fully manage and oversee the platform themselves. Full access to student progress and assessment reporting data that can then be used to inform future adaptations, as well as to provide on-demand tech support and outreach to support learner persistence.
- Adaptability: Ability to easily customize interface, site content and navigation, and overall look and feel of LMS through plug-ins. Robust and secure integration options, specifically for enabling interaction in virtual classrooms through video conferencing.
- Affordability: Flexible pricing and cost structure based on usage and tech support needs and training affordability at large scale.
- Mobile-optimized, user-friendly platform: Easily accessible for adult learners with limited digital literacy to learn to use as well as a wide array of online tutorials, onboarding support, guidelines and other resources for new administrators.
- Security and longevity: Preference for an open-source solution for greater assurance on protection and ownership of data as well as longevity of tool.
Based on these key LMS requirements, the team selected Moodle to deliver the training using the project’s different instruction models featured in Table 1, above. The new platform is now used to host the online learning, launch the virtual classroom, share instructor tools and collect participant data.
Concurrent to selecting Moodle, the team evaluated different platforms and options for web conferencing to be used in a learning setting. Ease of accessing and launching the live, virtual classroom for learners on both computer and mobile was one of the most important factors in deciding which video conferencing platform to use. Furthermore, the team prioritized solutions that supported full integration with Moodle. Big Blue Button (BBB), an open-source web conferencing platform developed by Blindside Networks, best incorporated these characteristics and was, therefore, the favored option.
Lessons from Developing a Customized, Open-source LMS
During implementation, there were additional lessons learned from the development and operation of the customized LMS and delivery of training content, namely:
- Scaling smart. The team was able to build capacity in house step-by-step through on-demand tech support. With guidance from World Education, the team was able to identify its own capacity, gaps and needs, and refine educational technology strategy for expansion.
- Versatility. The ability to create course pages that are easy to navigate and to make quick, on-the-go iterations to the platform and course content enabled the team to deliver high-quality training that serves the needs of working learners. Both open-source creators were eager to support the project by innovating platform features to meet the needs of the project.
- Mobile access. As we learned from the pilot, the ability to access online learning on smartphone and tablet is key to successful participation in online learning and course completion. After redesigning our digital modules to be mobile accessible, online participation doubled from an average of 2.5 modules completed in Year 1 to an average of 5 (out of a total of 6) in Year 2. In our most recent round of training in 2019, after implementing the new LMS, participants in online learning completed an average of 5.5 modules of 6 total.
Findings from the Digital Learning Design Process
The pilot revealed lessons learned and promising practices across three areas: training design, content development and program implementation areas.
Adapting Content for Remote, Mobile Instruction
When adapting content for remote instruction and to teach front-line workers in different sectors, the team had essentially two mediums of training content to develop and adapt for:
- Classroom instruction visual aids, optimized for both in-person and virtual delivery.
- Self-paced digital modules with learning activities, games, listening comprehension exercises and videos.
The first stage of work involved taking classroom instruction delivered in person and converting it to be used in the digital learning models shown in Table 1 above. When designing for synchronous, virtual learning for mobile environments, the team first identified key learning objectives for the lesson and core activities. Then the team evaluated how the platform features could support their effective delivery and what modifications and/or substitutions were needed. Finally, the team tested modified activities with teachers and program staff to determine ease of use and feasibility.
Many classroom activities could be adapted for virtual live instruction. There were several practices that contributed to successful adaptation and implementation, particularly in the remote virtual classroom:
- Capturing, documenting and integrating user feedback on content, navigation, bugs and more.
- Clean navigation, user-friendly interface with fewer steps and less text.
- Proactive tech support for remote learners.
- Integrated digital literacy instruction throughout modules.
- Clear guidance and instructions for how to facilitate the activities in different settings, including ample introductory resources and support.
A separate stage of work involved adapting existing, blended learning content for new contexts. For this process, the team used a number of resources to contextualize activities and instructional materials; these resources included employer feedback, job descriptions, industry standards, online forums and employee feedback. The adaptation process was agile and iterative and involved multiple review and revision stages.
Agile, User-centered Design
The team harnessed user-centered design practices to inform instructional design and content adaptations. As noted above, the approach to content adaptations was iterative; the team used tools to capture, document and track all user feedback on platform, tools and content, and then evaluate, qualify and integrate user feedback. The team used every opportunity to do rapid user testing and integrate feedback as quickly as possible, including demonstrations with employers. Being able to make on-the-go modifications was a huge benefit and an added value for the program.
Less Text, Fewer Steps
To best serve LEP and limited-digital-proficiency end users, the project team constantly checks that the tools and the digital content continue to be easy to digest and navigate throughout the adaptation process. Much of the user experience feedback was on cleaning up the homepage and removing text or features that were not of use and could confuse English learners. In addition to removing unnecessary text, important text and features were made more prominent in size and/or color. Similarly, additional steps mean additional hurdles, which is why tool integration was critical for user success. Managing and toggling between different applications on a mobile device requires a higher level of digital literacy skills which learners may not have yet acquired.
Supporting Digital Literacy Skills Development
For all models, but especially the remote model, it was particularly important for the project team and end users to understand platform vocabulary and functions so they could successfully navigate, independently troubleshoot common issues and actively participate in the virtual classroom. Course designers integrated digital literacy skills development into instruction and activities, including key vocabulary such as log in/log out, mute/unmute, microphone, headset, chat and poll. For example, students would practice logging in and logging out, muting and unmuting, using the chat box to share comments or questions and responding to poll prompts. This practice was supported with real-time troubleshooting and coaching by the help desk, as needed. Research shows that such intentional practice, when coupled with activities to build awareness and proficiency in using the vocabulary that describes the skills and tools required, can build digital literacy and comfort in online learning.[13]
One promising practice was peer-to-peer tech support and collaborative problem-solving in technology-rich environments. In the remote model, there were some instances when a learner had an issue with audio or unmuting, and another learner would jump in and try to help. Often this happened when learners knew each other through work. The other learner would quickly identify the problem and provide steps to troubleshoot, sometimes in the users’ first language.
Proactive, Not Reactive, Tech Support
Another practice was the incredibly proactive tech support initiated by program staff. Although the project staff provided user guides and help-desk information, the team did not wait for tech support requests to come in, but did active individual outreach to get students on board. This simple step and initial outreach allowed the team to catch any issues in advance of the first day of class and build a communication line/relationship with learners so that they would feel comfortable to ask for help in the future if needed and knew whom to reach out to.
Bringing Teachers on Board
Several Moodle and BBB orientation sessions gave teachers the time to test it out and practice, and they received handbooks and clear, step-by-step directions in an instructor manual. These sessions were recorded for future reference and were also used by program staff to review and reflect on what went well and on possible improvements.
The Forum’s work to incorporate technology in delivering industry-contextualized English training does not stop here. The Forum joined the EdTech Center @ World Education and more than 20 organizations in a national coalition called Digital US that will ensure that all of us have foundational digital skills to thrive in work and life by 2030.
To learn more visit: JoinDigitalUS.org
Expanding for Scale
SONAW, in its original state as a blended, contextualized English language training for the retail sector, exceeded our expectations and allowed us to realize we had created a potential solution for LEP workers. However, the original model had some limiting features that could hinder it from achieving our mission, to be able to scale to assist the large numbers of immigrant workers.
As we realized this challenge, we launched the process of creating a business plan based on labor-market research and external program evaluation. This process, assisted by external research firms such as Impact Catalysts, helped us create a plan for our needed improvements to three areas of the program: instructional model, technology tools and funding model.
In order to be the solution for the nation’s largest employers, with reach to large numbers of LEP workers, we realized we had to expand our instructional model from just an in-person option to include a fully remote model. Adding this instructional model, still taught live by accredited community college instructors, meant that we not only had to adapt the curricula for a remote setting, but also that we would have to implement a webinar tool for remote students to access their instruction.
In addition, at the time we set out to expand our instructional model for remote students, our existing learning management system (LMS) hosts decided no longer to host the program, due to a lack of confidence in their LMS’s capability for substantial scale. While we had hoped eventually to create and administer our own LMS, we were not prepared at that moment to undertake the task. Therefore, we solicited the support of expert educational technology partner World Education Inc. With their knowledge and technical assistance, we were able to customize Moodle and migrate our training modules to the open-source platform. Together, we were also able to customize and integrate the Big Blue Button webinar tool, especially mindful that we were customizing for LEP workers who also often struggle with digital-literacy challenges. Simultaneously, we partnered with World Education and CCCIE to adapt the in-person curricula to the remote training platform.
Overall, to scale the number of participants we are engaging, creating the remote instruction model and moving the LMS to open-source gives us more flexibility, more management and the ability to customize reporting, while allowing us to lower costs and provide competitive pricing for our product.
While these changes did help to lower the training costs, and to streamline and scale instruction, we knew that we had to look at a strategic plan to diversify our funding model. In this process, we assessed labor market potential and viable funding options to diversify, scale and sustain the program. We also assessed the ROI case to develop a plan for employer investment. Lastly, we researched and made decisions about pursuing long-range funding models, such as partnering with social finance, workforce boards or state adult education programs. Through this analysis, we also created a tiered pricing model that would ensure a plan to incorporate employer investment, after making the ROI case to employers, and considering what the market would bear.
These are the pivotal adjustments made to mitigate challenges and prepare for scaling the program as a national solution for LEP workers. However, nimbleness and the ability to make adjustments as needs arose also were woven into the entire process of designing and implementing SONAW. We are especially proud of how we seamlessly fine-tuned the training program while consistently pulling in the expertise of our collaborating partners across the educational technology, community college and evaluation fields.
Evaluation
Contributed by RTI International
Methodology: Training Evaluation at Scale
RTI International has served as the independent evaluator for Skills and Opportunity for the New American Workforce since the pilot program launched in January 2016 with the first cohort of employees. RTI has used the following research questions to guide the ongoing assessment of the instructional and employer/business value of the program:
- What were the English language gains for employees as a result of the program?
- What were the participants’ experiences with the program, and what benefits did they perceive for skill development, job performance, confidence and motivation for career advancement?
- What were the benefits perceived by managers?
- What lessons were learned to inform future efforts?
To address these research questions, RTI designed evaluation tools to gather data from administrative and test records, instructors, participants (employees) and managers. Data for the evaluation included (1) administrative and test data provided by the project managers (e.g., attendance, online module completion, 30-day retention, promotion, employee demographics, test scores, grades and instructors’ ratings of students); (2) instructor surveys (at the end of the program) and an instructor log (weekly) administered online by RTI; (3) employee surveys before and after the program; and (4) two-part (workforce background and employee ratings) manager surveys implemented with managers following the program. RTI has also conducted visits to each site, observed classrooms and conducted interviews with instructors and instructor partner staff.
As the program has evolved over the past three years, the processes and tools for data collection have been adapted accordingly. The 2016 pilot study was the first opportunity for RTI to test instruments and procedures for the evaluation. Following the pilot study, RTI revised surveys to clarify questions and to eliminate or add questions as needed. A scannable response form for the Employee Survey was created to allow for more accurate and efficient processing of survey responses for larger groups of participants. With the expansion of sites to more diverse locations, there was also a need to translate the survey into Spanish and Haitian Creole to accommodate students’ native languages. Finally, with each new employer partner, RTI customizes the Employee Survey to take into consideration the employee’s specific work environment.
Most recently, with the development of the fully remote model, all surveys were digitized and are now administered online, through the program platform. The online data collection has facilitated monitoring of responses, enabled more efficient and accurate data collection and facilitated data cleaning and analysis.
Using Employer Perspectives to Validate and Fine-Tune Training Models
Gathering manager/employer perspectives has been essential to the development and refinement of the program. The feedback provided by managers represents a firsthand account of the impact of the program, based on direct observations of improvements in employees’ language skills, communication and other aspects of their job roles. In the first part of the Manager Survey, managers give an overview of the workforce, and in the second part, managers are asked to rate each of their employees on different job-related dimensions.
When asked what impact an employee’s limited English proficiency status may have on the workplace, managers in Year 2 of the project said they were most concerned about it negatively affecting customer services (84%), causing communication difficulties among staff (73%) and limiting their employees’ growth potential in the workplace (70%).
Nearly all managers agreed that contextualized English classes are beneficial to the employee and to the company; nearly all would recommend these classes to other employers.
Managers were asked to assess the progress made by their employees on different job-related dimensions. Over 90% of the ratings indicated moderate or significant progress with employees’ understanding of safety guidelines and processes (safety on the job), morale and job satisfaction and confidence on the job. Between 80 and 90% of the ratings indicated moderate or significant progress with increased self-esteem; working through language challenges; interactions with customers; participation in worksite orientations, training and workshops; and communication with colleagues and supervisors. About 90% of the ratings indicated progress was made in employee productivity, defined as having demonstrated an increase in productivity as a result of increased language skills, increased quality of work and reduced time per task. Similar results have been observed each year of the evaluation.
These data suggest that the program has benefits for both employees and companies, especially those companies with a customer-oriented focus where verbal communication is critical. Based on these data and feedback from employees, the curriculum was further customized to offer class activities for specific job functions and to place greater emphasis on day-to-day oral communication in the workplace and job competencies, rather than grammar.
Challenges: Past and Present
As part of the ongoing evaluation of SONAW, RTI gathers input from instructors on the successes and challenges of program implementation. Collecting these data following each round of data collection has informed implementation planning and curriculum refinements for the next round. RTI has used the instructors’ weekly log and an end-of-course instructor survey to capture information about immediate/daily issues and broader reflections from instructors. RTI obtained additional feedback through in-person interviews with instructors and instructor partner staff. These data have proved invaluable for addressing ongoing issues and concerns and for strengthening the program overall. While many of the challenges have been resolved successfully, a few remain; the project team continues to look for ways to address these in collaboration with the project partners. The importance of a formal evaluation for the purpose of continued program improvement cannot be overstated.
An example of a challenge that was addressed successfully is accessibility of online learning. In past years when students were offered both online and in-person training, the participation level for the online portion of the program lagged behind the in-person attendance. The issue stemmed from lack of access to a computer, which varied depending on the training location, e.g. store training rooms or community college classrooms. Sites that partnered with community colleges often provided students access to computers or computer labs while cohorts who received classes in store buildings did not have consistent access to computers or the internet. Responses to questions on the Employee Survey revealed that the majority of participants reported using their cellphones for text messaging and social media every day. The project team used this information to develop an application that could be accessible both on desktop computers and on mobile devices, thereby increasing access to online learning. The application was made available at all times to accommodate participants’ work schedules and was left open through the remainder of the calendar year, beyond the end of the course, so participants could continue practicing their skills. Findings reported in 2017 indicated that approximately half the students accessed the online learning tool successfully using both their cellphone and a computer. More important, as noted above, the average number of modules completed on the online learning tool increased from 2.5 in 2016 to 5 (of 6 total) in 2017.
One challenge that the project continues to face is coordination with managers with regard to employees’ work and class schedules so students do not miss class for work-related issues. Although the training recorded high retention compared with the average for ESL classes, in 2017, the issue cited most often — by 13 of 17 instructors — was absent or tardy students. Data revealed that the most common reasons for being tardy or absent, and for dropping out, were work-related. Students most often reported missing class because their work schedule for the week did not allow them to attend class. Others were promoted or moved to another job with a different schedule. Students sometimes had to forgo participation in the class altogether when the work-related issues could not be resolved. Related to the evaluation, the concern is that instructional time per student (or dosage) could fall below the recommended hours of instruction needed to demonstrate gains on standardized language tests. To mitigate the impact of work-related issues on attendance and retention, which ultimately affect dosage, the project team continues to work with managers to find ways to implement the program while allowing for maximum participation by employees. For example, recordings of each virtual session are made available for 7 days so that students who miss class may watch the recording and still get credit for attending. In the current project model, the cost of the training program has been passed on to the employers. In addition to increasing sustainability for the program, the shift in financial responsibility for the program to the employers may also bring a greater commitment from managers to support their employees in this regard.
Integrating Evaluation Instruments with New Technology Tools and Training Models
Over the past several years the SONAW program has expanded to include larger numbers of participants across more sites. While the original implementation model — 40% in-person/60% online — is still available to clients, delivery options now include fully remote programming and virtual classrooms. The model is selected based on the needs of the employer and the availability of instructors. The fully remote model continues to be blended (40% face-to-face instruction via a live webinar/virtual classroom and 60% self-paced online learning). These updates to program implementation have prompted appropriate changes in evaluation tools and processes in order to maintain a close fit between the evaluation design and the program being evaluated.
For example, the use of technology for program delivery has allowed RTI to deploy all surveys and data collection tools through the same platform as the curriculum. This has resulted in lowered costs, more efficient collection of data and faster turnaround time for analysis and results. This is most evident in the pre- and post-program Employee Survey, which was previously administered by site staff using paper-and-pencil forms. The forms were then shipped to RTI from each of the sites, and scanned and uploaded to the database before data could be cleaned and analyzed. In the most recent collection of the same data, participants entered the web-based survey through the same portal as the curriculum and completed the survey during a specified window of about two weeks. As before, surveys were offered in multiple languages. RTI monitored the survey completion progress remotely and sent weekly (or more frequent) status reports to the project team. The project team worked with the virtual instructors to track nonrespondents and to address any issues with user IDs or other anomalies flagged by the RTI reports.
Adjustments were also made to other aspects of data collection. Classroom observations, previously conducted in person by RTI, were completed by watching live or recorded sessions with the virtual instructor. The Instructor Survey and Instructor Weekly Log were both deployed through the curriculum portal instead of online servers. The Manager Survey, on the other hand, was accessed through a link sent via email rather than through the curriculum portal. Throughout data collection, RTI provided the project team with status reports as well as data summaries for all instruments.
With the curriculum now designed to fit both a physical and a virtual classroom with fully remote delivery, RTI will incorporate new metrics using data captured by the system. For example, in addition to tracking program participation, module completion and scores on knowledge checks, RTI may collect time spent by participants in each module and in total, calculate dosage as time spent engaged online and potentially use quick polls to collect feedback on particular modules or sessions. These and other metrics may help the project team better understand where participants spend the most time, which modules take the longest to complete and what the participants think of particular sessions or modules.
Key Lessons Supporting our Models and Strategy for Scale
A formal evaluation has been in place since the program’s inception in 2016. Through extensive data gathered from multiple sources, including administrative/test records, instructors, managers and program participants, the evaluation has yielded a number of lessons learned and best practices that helped shape the current training model. These have been critical to the refinement and evolution of the program, from the original training delivery model (40/60 blended learning) to the current model for scaling up (remote), including the following:
The overall level of digital (computer) skills was lower than needed for the blended learning program. The online learning component in the original training model required that students have basic computer literacy skills and be able to manage the online work independently. For many students, this was a challenge. Instructors recommended adding more computer instruction, providing better support for students, and making changes to the online curriculum presentation and platform. In response to this feedback, the curriculum and the online learning program underwent substantial revisions for Year 2 that included: the addition of a computer literacy component for the curriculum, development of a new application that was accessible on desktop computers and on mobile devices and improved and simplified online learning experiences through a more user-friendly online learning platform. As the technical requirements for participants have increased with each new iteration of the program delivery model, the program has continued to adopt technical support features to ensure that participants’ needs for digital skills are being met.
Participants were most engaged during class when it was relevant to their own job role, indicating that the curriculum could be further customized according to job functions. The contextualized English language classes, which use vocabulary and scenarios relevant to employees in their jobs, allow participants to take what they learn in class and apply it to their jobs daily. According to instructors interviewed, the opportunity to practice English relevant to their jobs helped students gain self-confidence, which motivated them to use English with customers and managers. This suggested that class activities could be further customized to specific job functions for the benefit of students. As a result, the curriculum was revised to include more customized class activities and to focus on oral communication rather than grammar. In-class activities emphasized grammar used in real worksite situations (e.g., a customer placing an order for a cake at the bake shop). As the program diversifies to new business segments beyond the original retail industry, the program will continue to be tailored to the specific workplace environments in order to maximize the impact of the program on employee language skills, communication, productivity and career development. To date, the training offers solutions for LEP workers in retail, manufacturing, ride-sharing, gaming and casino resorts and amusement parks, with several new contexts to be added to the curriculum portfolio in the next year.
Evaluation data suggest that English language learning is critical to opening career pathways. Employees have consistently reported very high 30-day and 60-day retention rates following the SONAW training program. In most training cohorts to date, 100% reported they were still employed at the 60-day mark. Moreover, 37% of participants trained in Year 2 received a promotion following the course, and 73% reported they were “on track” to receive one. Employees also report personal gains in job performance as well as increased self-confidence using English with customers and other staff. Over the past years, participants have reported considerably less difficulty with work activities associated with English language skills at the end of the program than at the beginning of the program with respect to daily communication with customers, daily communication with coworkers, following instructions and understanding safety guidelines.
Strong English skills can also serve as a catalyst for continued workforce development, including participation in workplace skills training, credentialing, building digital and soft skills and continuing education. SONAW instructors report that participants often find the experience of being a student to be empowering and a great source of pride, as well as a motivator for taking other classes or furthering their education.
Evaluation findings have not only served to support continuous program improvement efforts but also to validate the training models at each phase of the project. Conclusions drawn in the early phases using the 40/60 blended learning model supported the notion that this is a pedagogically sound model that generated highly successful results. In order to reach more students in more locations while keeping the essential components of the original model intact, the project team moved to a fully remote, mobile-accessible model that incorporated virtual classrooms. With the adaptation to a fully remote model, the evaluation will assess whether the program retains its training efficacy as it scales up to reach more employees across diverse geographical regions. Importantly, these adaptations have been informed by evaluation findings and recommendations at each phase of the project.
Training Impact: A Word for Practitioners
In alignment with the Forum’s mission, with this project the Forum deepened its work in engaging with the role of immigrants in the workforce, with an expanded emphasis on ensuring that immigrant employees have the tools they need to succeed. Our results prove that increased English language skills have a lasting and multiplying impact on career mobility, helping workers improve their livelihoods, while boosting local and national economies. For Julia, Gerardo, William, Mariano and Teresa, along with millions of other LEP workers, this training not only helps improve language proficiency, but unlocks the potential to continue to enhance their skills by taking part in additional training opportunities.
With support from the Walmart Foundation and JPB Foundation, and counsel from various education and technology experts, the Forum, a nonprofit organization with deep connection to and knowledge about the recipient population, is well positioned to successfully create and integrate this innovative program for the private sector. The project has demonstrated that it fulfills company operational needs while offering excellent pedagogy and proven results, with the potential for serving companies with large workforces in diverse geographic regions, which allows the training to compete with for-profit ventures.
1. Impact for Individual Workers
SONAW sits at the nexus of workplace English training and workforce development by serving as a bridge for workers to fulfill their goals:
- Boost English language skills.
- Improve communication with colleagues and supervisors.
- Build digital skills.
- Participate in workplace skills training.
- Continue their education and pursue credentialing or licensing programs.
- Pursue career advancement opportunities and increase economic mobility.
2. Impact for Companies
SONAW also supports companies in retaining employees by:
- Improving work-related language and technology skills.
- Boosting morale, confidence on the job and loyalty.
- Facilitating promotions to more skilled and higher-paid jobs.
- Increasing productivity and business revenue.
3. Impact for Customers
Last, but not least, SONAW offers an enhanced customer experience by:
- Developing contextualized communication and customer service skills.
- Increasing employee confidence and other soft skills needed to satisfy customers.
- Building brand loyalty and expertise.
By blending face-to-face instruction with technology-based learning our training has maintained high efficacy in addressing learners’ needs. The excellent results achieved in the past three years are due in part to solving many of the barriers faced by LEP working adults across the country: availability, affordability and schedule of classes, work schedules, child care and reliable transportation and access to educational and training opportunities. The robust online learning supported by weekly, convenient classes taught by qualified instructors is key to the student persistence recorded by the program and attaining language gains at rates double the national average for similar training programs. The project team was very intentional in adapting all digital tools for the LEP learners to ensure their success, while helping them increase digital literacy skills at the worksite. This includes developing and customizing our own LMS, integrated with custom webinar tools for delivering the course remotely, all accessible on computer, smartphone and tablet.
The impact of this English language training positions it as an innovative and highly effective workforce development solution for integrating foreign-born workers into the company’s workforce, by accelerating skills development, improving job performance and opening career pathways. And because of its high potential to be easily customized, our training can become a solution for several other fields and entities that support improving the skills of immigrant workers, including government, government workforce development systems, educators and immigration organizations.
Acknowledgments
Special appreciation is extended to the Walmart Foundation* for funding the formation and expansion of this project. In addition, we appreciate The JPB Foundation for joining this work as a supporting funder, as the project scales.
We also extend appreciation to the additional consultation offered by Impact Catalysts, and advisory support from the Barbara Bush Foundation for Family Literacy and Graham Holdings.
*The research included in this report was made possible through funding by the Walmart Foundation. The findings, conclusions and recommendations presented in this report are those of the National Immigration Forum alone, and do not necessarily reflect the opinions of Walmart or the Walmart Foundation.
Authors and Contributors
Authors:
Jennie Murray
Director of Integration Programs
National Immigration Forum
Ana Negoescu
Manager of Integration Programs
National Immigration Forum
Contributors:
Suyapa Silvia
RTI International
Teresita Wisell
Community College Consortium for Immigrant Education
Jill Casner-Lotto
Community College Consortium for Immigrant Education
Alison Ascher-Webber
World Education
Victoria Neff
World Education
Notes
[1] Wilson, J. H. (2014). Investing in English Skills: The Limited English Proficient Workforce in U.S. Metropolitan Areas. The Brookings Institution. Retrieved on 9/25/2019 from https://www.brookings.edu/wp-content/uploads/2014/09/metro_20140924_investing_in_english_skils_report.pdf
[2] Wilson, J. H. Investing in English Skills, 13-14.
[3] National Skills Coalition, Middle-Skill Job Fact Sheets. (2017). United States Middle-Skill Fact Sheet. Retrieved on 10/29/2019 from https://www.nationalskillscoalition.org/resources/publications/2017-middle-skills-fact-sheets/file/United-States-MiddleSkills.pdf
[4] Boushey, H. and Glynn, S. J. (2012). There Are Significant Business Costs to Replacing Employees. Center for American Progress. Retrieved on 10/29/2019 from https://cdn.americanprogress.org/wp-content/uploads/2012/11/16084443/CostofTurnover0815.pdf
[5] Roberts, M. (2006). Student persistence in the adult ESOL classroom. Pearson Education, Inc. Retrieved 10/29/2019 from http://www.pearsonlongman.com/ae/download/adulted/persistence.pdf
[6] Murray, J and Negoescu, A. (2019). Building Retail Career Pathways for the New American Workforce (pp. 29-39). In Working Together: How Community Colleges and Their Partners Help Immigrants Succeed. Lanham, MD: Rowman & Littlefield
[7] For a list of the CCCIE Blue Ribbon Panel members visit: https://www.cccie.org/about/our-blue-ribbon-panel/
[8] Smith, A. (2012). U.S. Smartphone Use in 2015. Pew Research Center. Retrieved on 10/29/2019 from https://www.pewinternet.org/2015/04/01/us-smartphone-use-in-2015/
[9] Nielsen Holdings (2015). The Total Audience Report Q1 2015. Retrieved on 10/29/2019 from https://www.nielsen.com/wp-content/uploads/sites/3/2019/04/total-audience-report-q1-2015.pdf
[10] Anderson, M. (2015). Racial and ethnic differences in how people use mobile technology. Pew Research Center. Retrieved on 10/29/2019 from https://www.pewresearch.org/fact-tank/2015/04/30/racial-and-ethnic-differences-in-how-people-use-mobile-technology/ft_15-04-30_smartphonerace/
[11] Ascher Webber, A. (2017). Integrating Digital Literacy and Language Instruction. EdTech Center. Retrieved on 10/29/2019 from https://edtech.worlded.org/digital-literacy-second-language-instruction/
[12] Criteria for evaluating workforce education technology tools can be found here: https://workforceedtech.org/tool-evaluation-criteria/
[13] Vanek, J. (2017). Second Language Proficiency, Academic Language, and Digital Literacy for LESLLA. In J. Sacklin & D. McParland (Eds.), Literacy Education and Second Language Learning for Adults (LESLLA). Portland. Retrieved from http://www.leslla.org/