Applying data science to Black achievements in higher ed

Last update: Thursday 31 May 2018

From time to time the denial of the achievements of Black Americans in higher education becomes so pervasive as to almost make me believe in "conspiracies." Given my personal history, I am particularly concerned by the recent "conspiracy" that denies the existence of hundreds of thousands of Black Americans like me who obtained higher ed degrees in STEM fields because the "pipelines" that would have enabled our existence haven't been created yet, according to the "conspirators". This kind of racist nonsense isn't new, but its persistence in a world wherein the most arcane facts are accessible to anyone who has access to the Internet is baffling. Herbert A. Simon, one of the great pioneers of our Information Age, anticipated the roots of this paradox.

A. Black Failure
More than fifty years ago Simon recognized that the scarce resource in the coming age of information abundance would be attention. Each of us can only attend to smaller and smaller percentages of the world's rapidly expanding spheres of knowledge. Whereas knowledge is expanding at exponential rates, our individual capacity to absorb these expansions remains unchanged. Now more than in any previous era, each of us can be profiled by the particular chunks of truth that we carry in our brains. "Of all the gin joints in all the world, why has my brain retained these particular chunks??? ... hmmmmmm"

While it has always been true that nobody could be well informed about everything, nowadays all of us can only stay well-informed about a few things. That's why I put "conspiracies" in quotes in this note's opening paragraph. Many deniers of Black achievement aren't racists; they are just misinformed. Given the scarcity of attention, getting more of these good folk to become more aware of Black achievement requires commitment to a zero sum game. More attention to Black success means less attention to something else. As I see it, the most obvious "something else" is Black failure.

Focusing on Black failure was a politically productive strategy for many decades. President Johnson initiated affirmative action programs in the 1960s because he recognized that outlawing discrimination was not enough. Prejudicial attitudes still prevailed in many fields. Therefore low Black participation in higher education and subsequent employment became the benchmarks that triggered affirmative action, the presumption being that the failure of Black Americans to obtain higher levels of participation was the consequence of residual individual and/or institutional prejudice in those fields.

B. Black Success
Now flash forward sixty years. The U.S. higher education system has maintained a more sustained effort to integrate its rosters than any other component of our society, with the notable exception of the U.S. Department of Defense. With what results? After sixty years of non-discriminatory recruitment and affirmative action enrollments, have we achieved parity? Not yet, but the good news is that the glasses are no longer empty; the better news is that they are way beyond half-full.

Unfortunately, progress has varied widely from state to state and from region to region. Because of these wide variations, we have to treat national percentages and other national statistics with considerable care. Low national percentages may obscure regional percentages that are much higher than the national percentages. They may also obscure regional percentages that are much lower than the national percentages.

• Case in point: contrast the low percentage of Black participation in higher education and subsequent employment in information technology in California (especially Silicon Valley) and other Western states with the much higher levels of tech education and subsequent employment in the Southern states (especially Georgia and Texas). From an historical perspective, there is considerable irony in the fact that Black Americans now enjoy far more tech success in the New South than in the Old West. Will serious observers really be surprised by a sudden upsurge in the hiring of Black techs by Apple if it establishes a new campus in North Carolina or Virginia? 

C. Data Science to the rescue
So what does all this have to do with data science? Recall that I suggested that the challenge was to get more people to become more aware of Black success. In other words more people should be motivated to invest the time required to incorporate data about Black success during the last 60 years into their long term memories. Most of the required data is already out there on government Websites. So the trick is to reformat this data into chunks that are more readily assimilated by more people. ... Enter data science, stage right ...

By "Data science" I don't mean the newer branches that focus on AI techniques, like deep learning or reinforcement learning. I mean the older branches that are rooted in statistical learning, branches that include an array of tools for acquiring, munging, analyzing, and visualizing data that's accessible from Websites and other sources. These branches also maintain a lively tradition of data scientists as the tellers of persuasive "stories" that explain the meaning of complex data. Indeed, the data scientists in these branches also developed interactive user interfaces to enhance the persuasiveness of their presentations.

Regular readers of this blog may be aware that I myself completed lots of data science MOOCs during the last three years and have earned two data science certificates for my efforts.

• Question:  Have you applied your new data science skills to spreading the good word about Black achievements in tech yet?
  My answer:  I tried.
   
• Question: Were you successful?
  My Answer: No.
   
• Question: How did you measure your failure?
  My answer:  Less than 200 people viewed my report
   
• Question: That's bad. Why do you think you failed?
  My answer: For a long time I was perplexed, even angry. Then for other reasons, I took Barbara Oakly's famous MOOC "Learning How to Learn"... and all became clear. My report contained all the facts that I needed to make my case, but it failed to persuade because it overloaded my readers' brains. Indeed, the last time I reread it I fell asleep midway through page five. Yes, it was late at night, and I was tired, but I fell asleep. If I was bored, lots of my readers must have been bored.
   
• Question: Why was it boring? Was it too long?
  My answer: No and yes. No, it was not too long because it contained all of the tables and maps that were needed to support its most important assertions; but that was also the problem. Most readers weren't interested in all of its "most important" assertions. Most readers were more interested in learning about what happened in their home state and a few other states; that's all. So yes, it was way, way, way too long because it covered all fifty states.
   
• Question: When you revise your report, what will you change?
  Answer: I'll change the format. The next version will be interactive. The New York Times published an interactive op-ed last week that simulated the consequences of trying to denuclearize North Korea by military strategies instead of diplomacy. 
  
  The op-ed presented its readers with a series of binary decision choices. After the reader selected one of the two options, the op-ed presented a discussion of the immediate consequences of their choice. Each discussion ended with another binary choice. The op-ed presented four choices, so there were sixteen decision scenarios in all. I had fun spending ten or fifteen minutes running five or six. Other readers may have run more scenarios in more time; still others may have run fewer in less time. But the name of the game was always "reader's choice".
  
  By unhappy contrast, the readers of the first edition of my report were given "writer's choice", i.e., they had to slog through discussions of tables and maps that covered all fifty states. Some readers got bored and skipped out before they reached what they might have regarded as the "good stuff". Other readers read the entire report but their brains were probably so numbed by data overload that they retained little or nothing from what they read. Neither group was likely to recommend my report to friends and colleagues. So my next version will be interactive, i.e., "reader's choice".
   
• Question: What else will you change?
  My answer: My attitude!!! ... :-)
  
  When I started my first draft back in 2016, I was almost 100 percent sure that racism explained the failure of Facebook, Google, and other leading employers in Silicon Valley to mount effective campaigns for recruiting top-notch Black technical talent from the Southern and Eastern states.
  
  But then came the Valley's shocking display of massive ignorance when confronted with evidence that the Russians had exploited social media to influence the outcome of the 2016 presidential election via the purchase and/or hacking of confidential information about social media's millions of users. If the Valley was this ignorant about the potential abuses of social media, perhaps its continuing failure to mobilize effective recruitment campaigns for Black techs is also motivated by massive ignorance, rather than racism. 

Roy L. Beasley, PhD
DLL Editor

++++++++++++++++++++
Related reports on this blog:

• Fact Checking the Myths of Silicon Valley ... March 2018
• Foreign Professionals: U.S. Tech’s Other Pipelines – Version 1.0 ... January 2017
• Best States for American Black, Hispanic, Asian, and Female Tech – Vers 1.0f ... December 2016
• The California Pipelines of Silicon Valley ... July 2016