Tuesday, August 01, 2017

Job-Oriented Online Programs (JOOPs) -- Part 2

Last update: Thursday 8/3/17
This note provides an introduction to packages of online courses for professionals who want to acquire new job-related skills that will enable them to change their career paths. Part 1 presents concepts and background information; Part 2 presents detailed profiles of the programs offered by a few of the major providers of these courses plus some conclusions.




IV. Profiles of Programs

The following profiles summarize the programs offered by each provider, but include enough details to show that Udacity offers substantially more support for students in its JOOPs than the other providers and generally charges substantially higher tuition and other fees. Udacity seems to assume that potential students will understand that its more extensive support services funded by its higher tuition and fees will increase the likelihood that they will acquire the skills they need to find new jobs after they graduate.

How can we determine the involvement of industry partners in the development of JOOPs? I suggest that the acid test should be the public commitment of industry partners to hire the program's best graduates. Any partner who made this commitment would be heavily involved in the program's development in order to be sure that its graduates were the kind of professionals the partner really wanted to hire. On the other hand, programs that were developed without the benefit of extensive input from industry partners would be less likely to produce graduates who would be hired by anyone. Unfortunately, as the profiles will show, none of the four providers has obtained this kind of iron-clad commitment from any of its industry partners.

Finally, JOOPs are collections of short courses, shorter than the standard 3 credit 15 week semester courses with which most potential students are familiar.  The following profiles include rough estimates of the number of credit hours that would be equivalent to each short course as per the process described in the "conversation" between Dr. B. and his former student in Job-Oriented Courseloads (above). In other words, given the amount of study time required for a course in a JOOP, the profiles provide rough estimates of the number of credit hours the student would expect to earn if he or she invested the same amount of time in an on-campus course that was offered for credit: 
  • Readers will recall that a 1 credit hour face-to-face course = 45 hours student effort ... because 1 semester = 15 weeks, and students attend a 1 hour lecture per week + study 2 hours each week = 3 hours effort each week. Total effort per semester = 3 hours/week * 15 weeks =  45 hours per semester.
     
  • Given the total number of weeks in all the short courses in a package and the number of estimated hours of study required per week, weeks * hours/week = total hours of study for the package. Then divide total hours by 45 hours per credit hour = estimated credit hours.
     
  • If the lengths of the courses are given in months, first convert months to weeks by dividing months by 12 months per year * 52 weeks in a year = weeks in the course.
These rough estimates should be good enough to make plausible comparisons of the courses offered in JOOPs with the courses measured in credit hours offered by universities in their on-campus degree programs.

Apologies to the reader -- I have included the tedious step-by-step calculations of tuition and equivalent credit hours in the following profiles in order to make it easier for me to identify/correct any errors (and to update the estimates when the providers change the input values), and also to make it easier for readers to see how I derived these estimates. 


A. Udacity

Package Name -- "nanodegrees"

Number -- Udacity currently offers 16 nanodegrees

Levels -- Udacity's programs are classified as "beginner", "intermediate", and "advanced" 


Open vs. Closed -- Required backgrounds are suggested for "beginner" and "intermediate" programs, but they are open to anyone. "Machine Learning" is open, but the other "advanced" programs are closed ==> students apply for admission and are admitted on the basis of their qualifications.

Projects -- Students progress through the programs by completing projects, i..e, assignments  that are comparable to the real assignments that professionals in the students' intended field must deal with. 

    Programs, estimated hours of study time required, tuition for programs, partners (cites first 2 or 3 partners listed on Udacity's Web pages if there are more than one) ... and equivalent credit hours

    Self-paced MOOCs offered by Udacity provide the knowledge required to complete each project. Most of Udacity's online courses are open to anyone for free; but non-paying students do not have access to nanodegree projects. Some nano degrees estimate higher weekly study times for their courses than others, but all estimates lie between 8 and 12 hours per week. The higher estimate, 12 hours per week, is used for all estimates of equivalent credit hours in the following lists of programs:
    • Beginner 
      -- Digital Marketing (8 projects, 3 months, $999 -- Facebook, Google, Hootsuite) ... credits hours = (3 months / 12) * 52 weeks  * 12 hours per week / 45  = 3.5 credits
      -- Business Analyst (8 projects, 16 wks, $200/month, tuition = 16 weeks /(52 weeks/12  months)* $200 per month = $738 -- Alteryx, Tableau) ... credit hours = 16 weeks * 12 hours per week / 45 = 4.3
      -- Android Basics (10 projects, 6 mos, $199/month, tuition = 6 * $199 = $1,194 -- Google ... Note: This nanodegree is not a "JOOP" because it does not prepare students for a career change; it just prepares them to enroll in nanodegrees that are JOOPs ... credit hours = (6 months / 12) * 52 weeks * 12 hours per week / 45 = 6.9
      -- Intro to Programming (5 projects, 5 months, $399... Note: This nanodegree is not a "JOOP" because it does not prepare students for a career change; it just prepares them to enroll in nanodegrees that are JOOPs ... credit hours = (5 months / 12) * 52 weeks * 12 hours per week  / 45 = 5.8

    • Intermediate 
      -- React (3 projects, 4 months, $499 -- React Training) ... credit hours = (4 months/12 ) * 52 weeks * 12 / 45 = 4.6

      -- VR Developer (13 projects, 3 two-month terms, $400/term, tuition = 3 * $400 = $1,200 -- Google VR, Vive, Upload) ... credit hours =(2 * 3 months /12) * 52 weeks * 12 hours per week /45 = 6.9 
      -- Android Developer (7 projects, 9 months, $199/month, tuition 9 * $199 = $1,791 -- Google) ... credit hours = (9 months/12) * 52 weeks* 12 hours per week  45 = 10.4
      -- Front End Web Developer (7 projects, 6 months, $199/month, tuition = 6 * $199 = $1,194 -- AT&T, Google, GitHub) ... credit hours = (6 months /12) * 52 weeks * 12 hours per week /45 = 6.9 
      -- Full Stack Web Developer (7 projects, 6 months, $199/month, tuition = 6 * $199 = $1,194 -- Amazon, GitHub, AT&T)  ... credit hours = (6 months/ 12) * 52 * 12 / 45 = 6.9
      -- Data Analyst (7 projects, 260 hours, $199/month, tuition = (260 hours / 12 hours/week) / (52 weeks/12 months) * $199 per month= $1,990 -- Facebook, Tableau)  ... credit hours = 260 hours / 45 = 5.7
      -- iOS Developer (14 projects, 6 mos, $199/month, tuition = $1,194 -- AT&T, Lyft, Google) ... credit hours = (6 months/ 12) * 52 * 12 hours per week   45 = 6.9
      -- Deep Learning Foundations (5 projects, 6 months, $399 [obtained from Reddit])  ... Note: This nanodegree is not a "JOOP" because it does not prepare students for a career change; it just prepares students for & guarantees their admission into Udacity's Robotics, Self-Driving Car, and AI advanced nanodegree programs) ... credit hours = (6 months/12 * 52) * 12 hours per week /45 = 6.9

       
    • Advanced 
      Robotics (4 projects,  2 three-month terms, $1200 per term, tuition = 2 * $1200 = $2400 -- Bosch, Electric Movement, iRobot)  ... credit hours = (2 * 3 months / 12 ) * 52 * 12 hours per week / 45 = 6.9
      Artificial Intelligence (4 projects, 2 three-month terms, $800 per term, tuition = 2 * $800 = $1,600 -- IBM Watson, Amazon Alexa, DiDi) ... credit hours = (2 * 3 months / 12) * 52 * 12 hours per week / 45 = 6.9
      Self-Driving Car Engineer (5 "challenges", three 12-week terms, $800/term, tuition = 3 * $800 = $2400 -- Mercedes Benz, Nvidia, Uber ATG) ... credit hours = 3 terms * 12 weeks/term * 12 hours per week  / 45 = 9.6
      Machine Learning Engineer (11 projects,  $199/month, 6 months, tuition  = 6 * 199 = $1,194 -- Kaggle (acquired by Google)) ... credit hours = (6 months/12) * 52 * 12 hours per week / 45 = 6.9
    As the reader can see from the highlighted numbers at the end of each program line, Udacity expects its students to invest a substantial amount of time studying for the courses in its nanodegrees. Most of its courses require study time that would have earned at least 6.9 credits if the students were enrolled in degree programs. Indeed, students who prepare for its Robotics (6.9 credits), AI (6.9 credits), and Self-Driving Car Engineer (9.6 credits) programs by first taking its Deep Learning Foundations (6.9 credits) would have earned 13.8, 13.8, and 16.5 credits from degree programs from degree programs, i.e., four or five 3 credit courses.

    Industry Partners -- How involved were Udacity's partners in the development of its nanodegrees? None of its programs passed the "acid test" that I proposed earlier, i.e., none of its industry partners posted firm commitments on their nanodegree Web pages that they would hire the programs' best graduates. So I began with Sebastian Thrun's position as a former member of Google's senior technical staff, a position made him a certified member of Silicon Valley's technical elite, a status that facilitated his capacity to obtain well publicized support for Udacity's programs from Google and other leading firms in the Valley. Then I looked for other indicators of his parters' involvement, especially their reputations.
    • Google "owns" the Android OS and is deeply committed to virtual reality (VR) via its cardboard initiative. Therefore it seems likely that Google would closely scrutinize Udacity's Android and VR courses to be sure that they provided the required skill sets for Android and VR developers before adding its name as an industry partner for these three nanodegrees. 
       
    • Google recently purchased Kaggle, Udacity's industry partner for machine learning and the most prominent organizer of competitions for data science/machine learning experts. Therefore it seems likely that Google would closely scrutinize Udacity's machine learning engineer nanodegree to be sure that it provided graduates with the kinds of skills required to do well in Kaggles
       
    • Google and Facebook owe their massive success to their mastery of digital marketing and to advertisers' continuing faith in their mastery. It therefore seems likely that both companies would closely scrutinize Udacity's digital marketing nanodegree to ensure that this program would enhance their brands. 
       
    • IBM recently positioned its Watson AI system as the cornerstone for a wide array of new services that it will provide to businesses and governments in the near future. AI is a broad field, so it seems likely that IBM would have closely scrutinized Udacity's advanced AI nanodegree to be sure that its graduates acquired the particular skills required to appreciate potential applications of Watson's particular embodiment of AI technologies. 
       
    • And then there's the "pipeline" problem. Responding to pressure from the Rev. Jesse Jackson, Sr. and other activists a few years ago, the most prominent tech firms in Silicon Valley have released annual reports that document their continuing underemployment of female, Black, and Hispanic employees on their tech staffs. Their continuing "explanation" for their lack of diversity has been the low percentage of women, Black, and Hispanic students in their traditional pipelines, i.e, the colleges and universities from which they recruit most of their tech employees.  

      Udacity's nanodegrees provide their industry partners with new pipelines. Therefore it will be highly embarrassing to Google, Facebook, GitHub, and Udacity's other Silicon Valley partners if they decline to hire women, Black, and Hispanic graduates of Udacity's programs. So the best way for them to avoid this debacle is to continuously audit Udacity's programs to be sure that the programs are teaching the skills the partners are really hiring.
    Support staff are experienced experts who answer students' questions, evaluate their projects, and act as mentors. 

    50% tuition refunds are awarded to students who complete the Android Basics, Android Developer, Data Analyst, Front End Developer, Full Stack Web Developer, Become an iOS Developer, and Machine Learning programs in 12 months.

    Face-to-face weekly meetings are open to students who subscribe to the "Udacity Connect" service. During these sessions students can discuss their courses and projects with staff and other students, and enhance their professional networks. At this time the UConnect service is only available in San Francisco, Los Angeles, and New York. UConnect costs $99 per month.

    Job Guarantees are provided to students who subscribe to Udacity's "Nanodegree Plus" service. Udacity's slogan for this service is 
    "Get a job or your money back". Subscribers receive help finding jobs in their career tracks within 6 months after graduation and receive 100% refunds of their paid tuition if they don't find jobs. The detailed "Terms and Conditions" for this service should leave no doubt that Udacity expects students to invest substantial time and effort to earn Udacity's money back guarantee.  At the present time this service is only available for the following nanodegrees: Machine Learning Engineer, Full Stack Web Developer, Android Developer, Become an iOS Developer, and Data Analyst. As noted on Thrun's blog, the Nanodegree Plus service costs $299 per month, but this subscription fee includes tuition.
    B. DataCamp

    Package Name -- "Career Tracks"

    Number -- DataCamp currently offers 7 career tracks

    Open vs. Closed -- All courses and tracks are open to all students.


    Programs, estimated hours of study time required, tuition for programs ... and equivalent credit hours

    • DataCamp students pay $29 per month subscription fees for access to all DataCamp courses. The fee per week = 29 /(52 weeks/12 months) = $6.70
    • The following list assumes that serious students will invest at least 12 hours per week in their self-paced courses, the same as was used for Udacity's profile (above)
    DataCamp's short lecture videos are usually accompanied by online labs in which students write scripts and functions to perform the calculations discussed in the short videos. DataCamp provides is own estimates of the time students should expect to view the videos and complete the labs. 

    My own experience in a few DataCamp's courses was that mastery of the content of the videos plus labs required that I also spend substantial additional time playing around with similar problems in the IDE on my own computer (R Studio) in order to be sure I could write comparable scripts and programs outside of the browser-based IDE provided by DataCamp. How much additional time? 

    Rather than provide estimated times based on my personal experience, I think it's better to reframe DataCamp's videos + labs as an example of a teaching style that some instructors in traditional face-to-face degree courses have used. Following John Dewey's classic maxim -- "Learn by doing" -- these instructors spend class time making brief presentations of concepts and techniques. Then they engage their students in hands-on applications of these ideas for the remainder of the class. Homework and other outside class activities provide opportunities for students to deepen their understanding of what they learned in class. 

    Accordingly, in the following list of programs I interpret DataCamp's estimates as "lecture" time, then add twice that time, as usual, to obtain rough estimates of the total time students spend on their courses, i.e., total time = 3 * lecture time.
    • R Developer (10 courses) ... total hours = 3 * 40 DC estimate = 120 hours ... weeks = 120 hours / 12 hours per week = 10 weeks ... tuition = 10 * $6.70/week = $67 ... credit hours = 120 hours / 45 = 2.6
    • Data Analyst with R (16 courses) ... total hours = 3 * 64 DC estimate = 192 hours ... weeks = 192 hours / 12 hours per week = 16 weeks ... tuition = 16 * $6.70/week = $107... credit hours = 192 hours/ 45 = 4.3 credits
    • Data Scientist with R (23 courses) ... total hours = 3 * 95 DC estimate = 285 hours ... weeks = 285 / 12 = 23.75 ... tuition = 23.75 * $6.70 = $159 ... credit hours = 285 / 45  = 6.3
    • Quantitative Analyst with R (12 courses) ... total hours  = 3 * 51 DC estimate = 153 ... weeks = 153 / 12 hours per week = 12.75 ... tuition = 12.75 * $6.70 = $85 ... credit = 153/ 45 = 3.4 
    • Python Developer (10 courses) ... total hours = 3 * 36 DC estimate = 108 ... weeks = 108 / 12 = 9 ... tuition =9 * $6.70 = $63 ... credit hours = 108 / 45 = 2.4 
    • Data Analyst with Python (13 courses) ... total hours = 3 * 47 DC estimate = 141 ... weeks = 141 / 12 hours per week = 11.75 ... tuition = 11.75 * $6.70 = $79 ... credits =  141 / 45 = 3.1
    • Data Scientist with Python (20 courses) ... total hours = 3 * 67 DC estimate = 201 ... weeks = 201 / 12 hours per week = 16.75 = $112 ... tuition = 16.75 * $6.70 ... credit hours= 201 / 45 = 4.5
    "Industry partners" -- Whereas Udacity and edX reference the reputations of prominent corporate partners, DataCamp references the eminence of its instructors. Its students learn new skills from instructors who are renowned practitioners of those skills in various industries. If I modified my acid test for DataCamp to require that its instructors commit to writing strong letters of recommendation for its best graduates, the programs would still not pass my test because their Web pages offer no commitments from the instructors to helping their best grads find employment.
    C. edX

    Package name -- "Professional Certificate Programs"

    Number -- edX currently offers 18 Professional Certificate Programs  

    Levels -- edX courses are classified as "beginner", "intermediate", and "advanced"

    Open vs. closed -- All certificate programs are open to all students.


    Programs, estimated hours of study time required, tuition for programs ... and equivalent credit hours
    Note 1  -- Tuition is the price students pay for obtaining verified certificates that they passed the courses in each program; students who don't want a verified certificate can take the courses for free.

    Note 2 -- Many edX programs suggest a range of study hours per week, e.g., 3 to 4; when calculating the study times required for all of the courses in a program, I consistently used the upper end of the study range for each course in the program.

    • Introductory Programs 
      -- Inclusive Leadership (4 intro courses) ... total hours =  4*1.5 + 4*1.5 + 4*2 + 1*1 = 21 hours, tuition = 50 + 50 +50 + 25 = $175) ... credit hours = 21 / 45 = 0.50.5
      -- Data Science for Executives (3 intro courses) ... total hours = 5*10 + 5*10 + 5*10 = 150 hours, tuition+ 99 + 99 + 149 = $347... credit hours = 150 / 45 = 3.3
      -- Java and Android Foundation (3 intro courses) ... total hours = 5*5 + 5*5 + 6*5 = 80 hours, tuition= 99 + 99 + 99 = $297) ... credit hours = 80/45 = 1.8
      -- Gestion Publica para el Desarrollo (3 intro courses) ... total hours = 6*6 + 5*6 + 7*7 = 135 hours, 25 + 25 + 25 = $75) ... credit hours = 135 / 45 = 3.0
      -- Project Finance and Public Private Partnerships (7 intro courses) ... total hours = 6 * 3*2 + 2 = 38 hours, tuition = 99 + 189 + 189 + 189 + 189 + 189 + 355 = $1399) ... credit hours = 38 / 45 = 0.8
      -- Essentials of Cybersecurity (4 intro courses) ... total hours = 4 * 4*5 = 80 hours, 79 + 79 + 79 + 79 = $316) ... credit hours = 80 / 45 = 1.8
      -- Introduction to Java Programming (3 intro courses) ... total hours =  5*10 + 5*7 + 5*7  = 120 hours, 99 + 99 + 99 = $297) ... credit hours = 120 / 45 = 2.7
      -- Six Sigma and Lean: Quantitative Tools for Quality and Productivity (3 intro courses) ... time =  8*4 + 8*4 + 8*4= 96 hours, 88 + 88 + 88 = $264 ... credit hours = 96 / 45 = 2.1
    • Intermediate Programs 
      -- Data Science (9 courses = 2 intro + 6 inter + 1 adv ) + capstone [adv] -- Microsoft) ... total hours = 6*4 + 6*5 + 6*4 + 6*4 + 4*2 + 6*4 + 6*4 +6*8 + 4*4 + 4*4 = 238 hours, 10 * 99 = $990 ) ... credit hours = 238 / 45 = 5.3
      -- Computer Science Essentials for Software Development (4 inter courses) ... total hours = 4*8 * 4 = 128 hours, 149 + 149 + 149 + 149 = $596)  ... credit hours = 128 / 45 = 2.8
      -- Front-End Web Developer (5 courses = 3 intro + 1 inter + 2 adv) ... total hours = 5*8 + 6*6 + 6*8 + 5*8 + 4*8 = 196 hours, 129 + 129 + 99 + 49 + 99 = $505) ... credit hours = 196 / 45 = 4.3
      -- Virtual Reality App Development (3 courses = 1 intro + 2 inter) ... total hours = 6*7 + 6*7 + 6*7 = 112 hours,  99 + 99 + 99 = $297) ... credit hours = 112 / 45 = 2.5
      -- Retail and Omnichannel Management (2 inter courses) ... time = 4*4 + 4*4 = 32 hours, 99 + 99 = $198) ... credit hours = 32 / 45 = 0.7
      -- Mergers and Acquisitions (M&A) (6 courses = 5 int + 1 adv) ... total hours =   5*4*2 + 1 = 41 hours,  50 + 100 + 200 + 300 + 400 + 500 = $1550) ... credit hours = 41 / 45 = 0.9
      -- Digital Marketing (4 inter courses) ... total hours = 6*4 + 6*3 + 6*4 + 6*3 = 84 hours, 585 + 585 + 585 + 585 = $2340) ... credit hours = 84 / 45 =  1.9
      -- Risk Management (6 inter courses) ... total hours = 5*4*2 + 1= 41 hours, 99 + 249 + 349 + 349 + 349 + 500 = $1895) ... credit hours = 0.9
      -- Agile Development Using Ruby on Rails (2 inter courses) ... total hours = 6*12 + 6*12 = 192 hours, 99 + 99 = $198) ... credit hours = 192 / 45 = 4.3
      -- Mortgage Backed Securities (MBS) (3 inter courses) ... total hours = 2*4 + 2*4 + 2 = 18 hours, 450 + 550 + 399 = $1399) ... credit hours =  18 / 45 = 0.4
    Industry Partners -- Although the description of the edX Professional Certificate Programs quoted in Part II of this note declared that "industry leaders" were involved in the development of these programs, only one program provided tangible indication of such involvement -- the Data Science program that is taught by Microsoft, instead of by an edX academic partner. By contrast, the partners for the other edX programs are only cited as sources of favorable comments about the programs, comments that read more like snippets from favorable book reviews than as commitments by the partners to hiring the program's best graduates.

    back-to-menu
    D. Coursera

    Package Name -- "Specializations"

    Number -- For the reasons that I noted in Providers section, I can't determine the number of JOOPs offered by Coursera. The Data Science program offered by professors from Johns Hopkins University is definitely a JOOP, but Coursera does not identify which of its other programs are job-oriented. 

    Open vs. Closed -- Required backgrounds are suggested for all courses in all specializations, but all courses and specializations are open to all students.


    Programs, estimated hours of study time required, tuition for programs ... and equivalent credit hours
    • Data Science (9 courses + capstone) ... total hours = 1 course * 5w * 4h/w + 8 courses * 5w * 9h/w + 1 course * 8w * 9h/w = 452 hours, weeks= 5w + 8 * 5w + 8w = 53 weeks, tuition = 53 /52 * 12 =  12.2 months @ $49/mo = $593 ... Industry partners = Yelp and SwiftKey ... credit hours = 452 hours/ 45 hours =  10.0

    V. Conclusions

    When I decided to write this note, I did not anticipate coming to any "conclusions". I just wanted to provide detailed descriptions of the programs offered by a few prominent providers of JOOPs. Having recently retired from a long career as a member of the tenured faculty and administrative staff of a prominent university, I had decades of hands-on experience dealing with the nuts and bolts of programs in higher ed. So I knew what to notice when I looked under the hoods of these new job-oriented operations. 

    My decision to enroll in a job-oriented program as part of my personal effort to make a final career change meant that I could not claim to be a disinterested observer.  But I would argue that my status as a participant-observer gave me a keener appreciation of the limits of what these programs could or could not achieve. So that's what my conclusions are all about -- the limits of JOOPs, i.e., what I think they can and can't do for professionals looking to change their career paths.
    • Assessment
      As a retired professor and a current student, my biggest concern about JOOPs is assessment. In face-to-face and online courses offered for credit  by accredited institutions, the size of enrollments in course sections is limited so that there are enough faculty or teams of faculty plus teaching assistants to assess every student's submissions. While multiple choice questions and other types of closed assignments can be graded by software, human assessment is required for open-ended assignments like essays, reports, and projects -- especially in programs that provide professional training.

      Courses in Masters degree programs and other traditional gateways to various professions are often taught by part-time instructors who are practicing professionals so that students can obtain the benefits of expert opinions with regards to current best practices in the field. Peer evaluations from other students are useful, but they are insufficient substitutes for expert evaluations. Indeed, many masters degree programs include internships, practicums, and other field assignments wherein students work under the direct supervision of practicing professionals. 

      Question: How can JOOPs that do not provide expert assessments of realistic assignments enable students to acquire professional level skills in a field?

      My answer: I doubt that most students can attain professional level skills without the benefit of constructive critiques of their projects from experienced mentors.
       
    • Jumps vs. Pivots
      OK, let's trot out the mantra: "Jump with degrees; pivot with JOOPs." Now let's try that again in English ... :-)
       
      In times past, degrees provided entry into a wide range of professions. For example, MBAs provided entry into management; LLBs opened doors to legal careers; PhDs were required for academic positions and/or research; bachelors degrees became the minimum requirements for entry into an increasing number of non-specialized white collar occupations. If someone wanted to jump into a new field, they usually had to obtain another degree. 

      JOOPs can now be used by professionals who only want to "pivot", i.e. to change directions, but stay in the same field. Why pivot? Many reasons come to mind, but an increasing number of professionals in an increasing number of fields face slower career advances and/or loss of employment as result of the disruptive impact of information technology.  

      Professionals who want to pivot might only need to enhance their skills to include some computer-based methods that could be applied to old problems and/or address new problems that were inaccessible to their old skills. New skills might be acquired through relatively inexpensive JOOPs that were roughly equivalent to, say, 12 credits of new knowledge, rather than the expensive 48 or more credits for a Masters degree. 

      But how does a student know that he or she has learned their new skills well enough to find new employment? This brings me back to my primary concern -- assessment. In my own case, I did well in all of the courses I took in a JOOP; but all of my projects were assessed by other students; none were assessed by faculty or by practicing professionals. I passed ten courses and earned ten certificates ... but did I really pivot??? No one has ever accused me of being shy. Nevertheless, I didn't feel my usual confidence that I had learned my new skills well enough to meet the expectations of potential clients. So I did what conscientious students usually do when they don't think they have learned enough ==> I enrolled in more job-oriented courses ... :-(
       
    • Job search
      Colleges and universities have a long history of supporting the efforts of their degree graduates to find jobs. Letters of recommendation from faculty for their best students are probably the most widespread forms of this support. But many institutions host "Job Fairs" that enable recruiters from corporations and government agencies to conduct on-campus interviews with large numbers of students. And many institutions have established "Career Centers" wherein students can receive guidance on a wide range "Do's and Don'ts", including how to prepare more effective resumes, how to organize portfolios of their projects and other course submissions, and how to answer questions during job interviews. Taken together these traditional supports for the graduates of degree programs provide a useful reference model for the kinds of services that might be offered by the providers of JOOPs for their graduates.

      Whereas academic institutions rightly claim that obtaining jobs is just one of the many benefits that graduates of their degree programs derive from their education, jobs are the only benefits that graduates of JOOPs expect to receive.  Indeed, their students' success in obtaining jobs is the raison d'être for JOOPs and the acid test of their effectiveness. That's why I think that support for job search should be a mandatory component of all JOOPs. Of course, providers of JOOPs won't have the funds to pay for these services if their tuition is too low.
       
    • Data vs. Anecdotes
      In my opinion none of the providers of JOOPs discussed in this note have proven their effectiveness as enablers of career change. Their Websites contain glowing anecdotal testimonies from their industry partners and from their graduates, but none of the providers offers systematic data about their enrollments, graduations, time-to-completions, gainful employment after graduation, and the number of graduates who were hired by industry partners; nor do they identify which kinds of students have been the most/least successful in completing their programs. To be more specific, how are Black, Hispanic, and female students doing in their programs?

      These caveats notwithstanding, I hope that the details that I provided in the Profiles section of this note leave no doubt in the reader's mind that Udacity is far ahead of the other three providers in its efforts to address the limitations that I identified in my other conclusions. Udacity's experts assess students' assignments; it offers pivotable programs worth two to four 3 credit hour courses; and it provides vigorous support for its graduates' job search.  Furthermore, Udacity has more industry partners than the other providers; it displays more testimonials from satisfied students on its Web pages; and market-savy investors have seen enough promise in its business plans to pay prices for its stock that are high enough for Udacity to be valued at $1 billion dollars, making it the first "unicorn" in the MOOC universe. Nevertheless, Udacity can't prove that its programs are effective ... yet.

      A few years ago, coding bootcamps that showed a similar kind of early promise burst on the scene; but two of the most prominent bootcamps (Dev Bootcamp and Iron Yard) are folding -- reminders that early promise does not guarantee future success. If Udacity fails, it might be due to unforeseen flaws in its business plans. For example, some of the most talented students might find that the higher tuition and fees that Udacity charges in order to fund its support services were not worth the greater benefits that these services provided. A particularly interesting possibility is that DataCamp (or some other low cost /no frills provider) might disrupt Udacity's dominant position by adding just enough support services to entice substantial numbers of the most ambitious/talented students away from Udacity, students who wouldn't need as much support as most students. This loss would jeopardize Udacity's capacity to deliver the kind of top-rated students its partners most wanted to hire and would thereby weaken Udacity's partnerships.

      I also have reservations about a few of Udacity's current nanodegrees. Most nanodegrees facilitate pivots. For example, a developer who currently works on personal computers or mainframes might use the iOS Developer or Android developer nanodegrees to pivot to developing mobile apps for iPhones or Android phones. Or a JavaScript developer might execute an easy pivot via the React nanodegree, whose projects and courses would add Facebook's important React JavaScript library to the developer's toolkit. But I am skeptical that substantial numbers of non-PhDs in computer science will be able to pivot their way onto the high ground of AI/machine learning currently dominated by computer science PhDs from the best universities in the world. Some career changes still require full jumps, i.e., additional degrees, because they require more additional knowledge than can be gained through pivots.

      For the foreseeable future, nanodegrees might pose riskier options for career changers than degree programs. But for those who only want to pivot rather than jump, the desired payoff may be worth the greater risk. As Bret Maverick, one of the twentieth century's most esteemed existential philosophers, was fond of saying: "Faint heart never filled a flush." ... :-)
    Full disclosure #2 ... Yes, I am seriously thinking about enrolling in one of Udacity's nanodegree programs ... :-)