Fact Checking the Myths of Silicon Valley

Last update:  5/31/18

Continuous repetition and elaboration convert tall tales into myths. Three years ago Silicon Valley's leaders furrowed their brows over "pipelines"; recently they have been nodding sagely about "leaky pipelines." And next year, will they be singing their own praises as the "plumbers" who are best suited to fix the "leaky pipelines" in some distant future???

What galls me about unfounded myths is that the more often they are repeated, the more people tend to believe they provide credible explanations. The myths become folklore, conventional wisdom, conceptual frameworks that exclude the real data from most discussions. 

A. Findings
In 2016/2017 I posted three reports on this blog that addressed the diversity of the U.S. information technology sector in the nation's fifty states based on data for 2015, the latest available at the time. My findings are summarized in the following paragraphs. Links to my reports are provided at the end of this note. 

Most of my findings were based on data downloaded from U.S. government Websites, mostly from the U.S. Census Bureau. However my reports also reflected data that appeared in the diversity reports published by many of the leading firms in Silicon Valley in 2015, 2016, and 2017. 

Although I chose to gather data for states rather than for counties or cities, the annual corporate reports confirmed that the Valley's tech sector was less diverse than the tech sector for the entire state of California, which, in turn, was far less diverse than the tech sectors in many Eastern states. So the reader is hereby alerted to my sometimes making statements about "Silicon Valley", then supporting these statements with facts about the entire state of California.

(As interested readers will discover, my reports assembled diversity data for Asian American, Hispanic Americans, and female Americans as well as for Black Americans in tech. However this summary only addresses diversity issues facing Black Americans.)

Myth #1: The Valley recruits its techs from the best colleges and universities in the world

• Fact: Historically, the Valley has recruited most of its American staff from California, which contains a small but impressive share of the best colleges and universities in the US. However it should be noted that at least one institution with a large number of alums in the Valley is San Jose State University which is not on any list of the top 100 universities in the U.S. that I have ever seen. Nevertheless San Jose's Website proudly proclaims that it is "powering Silicon Valley."
  
  In the last few years, the Valley has allocated an increasing share of its new technical positions to graduates from universities outside of California, a shift that reflects the increased importance of artificial intelligence, more specifically, of machine learning. But there are no financial incentives for the Valley to hire their other staff -- marketing, human resources, accounting, etc -- from out-of-state

Myth #2: The Valley imports highly skilled immigrants as temporary employees because they have skills that American techs don't have.

• Fact: California recruits a large and increasing share of its techs from Asia via H1-B visas and other temporary immigrant worker programs. In 2010, it employed 88,122 foreign techs, but employed 127,671 foreign techs in 2015 (mostly from Asia), a 44.9 percent increase in just five years.
   
• Fact: The vast majority of the Valley's hundred thousand Asian-Asian recruits (not to be confused with its Asian-American recruits) were not trained at the best universities in the world because Asia only contains a tiny handful of colleges and universities that are ranked on anyone's list of the world's top 100 institutions of higher learning. So there is no reason to believe that they have special skills that are missing from the American workforce.
   
• Fact: The Valley only pays its imported Asian-Asian techs 50 to 60 percent of what it pays comparably skilled and experienced American techs. The Valley could easily find qualified American techs to fill these positions; what the Valley can't find is American techs who are willing to work in America for sub-American wages.
   
• Fact: The only tech skills for which there is solid evidence of a shortage of American personnel is machine learning. This shortfall is global, is increasing, and is most clearly manifest in the outsized starting salaries (upwards of $300,000 per year) awarded to newly minted PhDs in machine learning. This increasing shortage suggests that at the margins the Valley is now hiring low pay Java and Python grunts from Asia, and hiring high pay machine learning specialists from all over the U.S., Canada, and the U.K.

Myth #3: The Valley's low percentage of Black techs reflects the low percentage of Black Americans in the tech employment pipelines of the nation as a whole

• Fact: In 2015, Black Americans represented a small share of the population of California (6.0 percent) and an even smaller share of its higher ed enrollments (4.0 percent). Therefore California's low percentage of Black employment in its tech sector (3.7 percent) was about what one would expect from tech companies that have until recently recruited most of their American techs from California. In other words, California's lack of diversity merely reflects the lack of diversity in California's pipelines
   
• Fact: By contrast, tech employment in many Eastern states was far more diverse than tech employment in California. According to 2015 Census data, 9.6 percent of tech employees in Texas were Black, 24.0 percent in Georgia, 13.3 percent in North Carolina, 19.3 percent in Maryland, 32.9 percent in Washington, DC, 15.2 percent in Virginia, and 7.1 percent in New York. It should also be noted that Texas, Georgia, and Virginia employed 1.5 times as many Black techs as California, i.e., 31,437, 31,211, and 28,281 vs. California's 19,968.

B. Conclusions 

1. The Valley pursues classic profit maximizing strategies 
It pays low salaries to temporary tech immigrants from low income countries in Asia who possess commodity skills sets, but pays premium salaries to techs from any country who posses globally scarce skill sets. Paying low salaries to tech temps is a win-win: employers save money on the temps' low salaries, but the temps' low salaries (low by American standards) are substantially higher than what they would have earned back home.

Better still, the skill sets of tech staffs are no longer the Valleys' most valuable assets. Its most valuable assets are the petabytes of information that consumers are providing for free via browsers and posts on social media. Previous generations of corporate and government staffs with minimal training in statistics were able to apply the powerful statistical procedures in the SPSS, SAS, and MatLab packages (accessible for substantial fees) to data collected by their employers.

So the Valley's titans are now racing to develop comparable machine learning packages that will be usable by the world's corporate and government staffs with minimal training in machine learning. Access to these packages will be free or cost minimal fees. But access to the titans' Big Data will cost substantial fees. Of course the titans will continue to generate huge profits by selling ads based on their own analysis of their Big Data.

2. The Valley's "pipeline" myth was a strategic misapprehension
The Valley's "pipeline" theories briefly diverted attention from the Valley's lack of diversity to the Valley's misperception of a lack of diversity in the information technology sector of the entire nation. But most of us who looked more closely at the Census data quickly recognized that tech employment in Eastern states was far more diverse than in California.

This should not surprise anyone. Most people attend college no more than a few hundred miles from where they grew up. Given the large Black population centers in Eastern states, Black Americans also represent higher percentages of the enrollments in the colleges and universities in Eastern states than in Western states. The additional fact that most people launch their careers within a few hundred miles of where they attended college is, therefore, the biggest reason why tech is far more diverse in Eastern states than in California and in other Western states. 

"Yes, but our pipelines are far more selective than Eastern pipelines because we only hire the best of the best of the best of the best of the best of the ..." which brings me back to myths #1 and #2 (see "Findings", above). When I said that more and more people believed the Valley's myths, I should have noted that many of the myths' most enthusiastic evangelists also run some of the Valley's most successful corporations ... :-(

3. A modest proposal ... 
As I see it, three factors make it difficult (impossible?) for Western states (including California, including Silicon Valley) to substantially increase the number of Black techs on their employment rosters in the near future:

• There aren't many Black machine learning specialists anywhere; this critical expertise is a globally scarce resource, regardless of color.
   
• An increasing share of its lower level tech positions are being awarded by California and other Western states to imported low pay Asian temps, positions that might otherwise be offered to young Black techs ... for higher pay, of course.
   
• The vast majority of Black techs were born, educated, and still reside in Eastern states.

A long time ago, I learned the "secret" that every consultant has to learn if he or she is to enjoy any success. You have to forget about ideal solutions that your clients can't do or won't pursue; instead, you have to encourage your clients to do more of the "good" things they are already doing or at least thinking of doing. 

Hiring thousands of Black machine learning specialists is an ideal solution, but it can't be done because they don't exist. Hiring fewer Asian temps is another ideal solution, but the temps' low wages make substantial contributions to the West Coast's bottom lines, so the biggest companies won't stop hiring these temps unless they are compelled to do so by drastic reductions in the quotas for H1-B and other visas issued by federal agencies. 

This leaves a third option, one that some West Coast tech companies are already pursuing and/or exploring ==> Relocate a substantial portion of their most important tech operations to Eastern states. This would enable West Coast firms to recruit more talent produced by the top ranked universities in Eastern states and also benefit from the more diverse employment pipelines in those states. 

• We all know the story of Pittsburgh's recent turnaround. It used to be a declining steel town; now it's a thriving urban center for machine learning and other tech. So when Uber wanted to beef up its AI capabilities, it persuaded more than 40 experts from Carnegie-Mellon University to come to work for Uber in Pittsburgh. Whatever the costs, the salaries Uber would have had to to pay these experts to persuade them to uproot their families to the sky-high real estate in San Francisco or the Valley would have been substantially higher. 
   
• Relocations would also create more favorable policy environments for the Valley's biggest companies. Congressmen, senators, and governors of the relocation states (and nearby states) would act to "protect" the new employment opportunities for the graduates of their universities, not to mention the new tax streams their states would be receiving.
   
• Amazon is currently conducting a highly publicized competition that will select a city for its second headquarters. Will anyone really be surprised if the winner is in an Eastern state?
   
• Finally, as a former consultant for the U.S. Department of Defense I am concerned that tech has become one of the most important components, if not THE most important component of the nation's industrial base, or to use the term preferred by the Trump administration: its "national security innovation base". The present concentration of the nation's most advanced tech in California and a couple of other Western states makes it more vulnerable to cyberattacks and/or physical attacks than if these operations were more widely dispersed. This vulnerability poses a substantial threat to our national security.

Roy L Beasley, PhD
DLL Editor

++++++++++++++++++++
Related reports on this blog:

• Applying data science to Black achievements in higher ed ... May 2018 
• Foreign Professionals: U.S. Tech’s Other Pipelines – Version 1.0 ... January 2017
• Best States for American Black, Hispanic, Asian, and Female Tech – Vers 1.0f ... December 2016
• The California Pipelines of Silicon Valley ... July 2016