The AI Literacy Gap
How should we measure AI literacy?
A recent Business Insider article highlighted comments from Deloitte executive Rob Hillard, who argued that too many university graduates enter the workforce viewing artificial intelligence as a form of cheating rather than as a professional tool. The article cited surveys showing that more than half of college students use AI at least weekly and that roughly one in five use it daily.
Those numbers seem encouraging. Yet they raise a more fundamental question: what exactly do they measure?
Frequency of use is often treated as evidence of AI literacy, but that assumption deserves closer examination. A student who uses ChatGPT ten times per day is not necessarily more AI literate than a student who uses it once per week. The first may simply rely on the technology more often, whereas the second may be considerably more skilled at evaluating outputs, verifying claims, identifying weaknesses, and integrating AI into complex intellectual work. Usage is easy to count. Understanding is much harder.
The distinction matters because employers, educators, and policymakers increasingly discuss AI readiness as though it were a single measurable characteristic. Universities point to rising adoption rates as evidence that students are adapting successfully. Employers point to gaps in workplace preparedness. Both observations may be correct because they may be measuring different things.
The apparent disagreement stems from a larger problem: many discussions of AI literacy rely on metrics that measure access rather than competence. Measuring AI literacy through frequency of use is akin to measuring writing ability by counting how often someone opens a word processor or evaluating quantitative literacy by tracking calculator usage. Exposure may be a prerequisite for competence, but exposure alone does not tell us whether someone can think critically, solve problems effectively, or exercise sound judgment.
A more useful question would be whether students understand AI well enough to use it responsibly and effectively. Can they identify hallucinations? Can they distinguish reliable information from plausible misinformation? Can they recognize bias, privacy concerns, and intellectual property risks? Can they combine AI generated outputs with disciplinary expertise to produce better work than either a human or a machine could create alone? Those questions begin to move beyond adoption and toward something more meaningful: AI literacy.
One possible framework is an AI Literacy Index built around six levels of competency.
| Level | Skill | Example |
|---|---|---|
| 1 | Access | Can use ChatGPT, Claude, Gemini |
| 2 | Prompt | Can write effective prompts |
| 3 | Evaluate | Can identify hallucinations and errors |
| 4 | Integrate | Can combine AI output with domain knowledge |
| 5 | Govern | Can recognize ethical, legal, and privacy issues |
| 6 | Create | Can design workflows where humans and AI collaborate effectively |
The progression reflects a movement from simple interaction with AI toward increasingly sophisticated forms of judgment. Access and prompting are important foundational skills, but they represent only the beginning of the journey. Higher levels require students to evaluate information critically, integrate AI outputs into disciplinary contexts, understand governance implications, and ultimately design systems in which humans and AI collaborate productively.
Governor Delivers Commencement Address at the University of Maryland Graduation Ceremony, 2014. Photo by Jay L. Baker for Maryland GovPics. CC BY 2.0.
Measuring the Wrong Thing
If one applies this framework as a thought experiment to the approximately two million bachelor's degree recipients graduating from American colleges and universities each year, a revealing pattern emerges. No national assessment currently measures AI literacy in this way, so the estimates below are illustrative rather than empirical. The distribution nevertheless helps illuminate the difference between AI adoption and AI competence.
Estimated Distribution of AI Literacy Among Approximately 2 Million Annual U.S. Bachelor's Graduates
| Level | Skill | Estimated % | Estimated Graduates |
|---|---|---|---|
| 1 | Access | 95% | 1,900,000 |
| 2 | Prompt | 65% | 1,300,000 |
| 3 | Evaluate | 30% | 600,000 |
| 4 | Integrate | 15% | 300,000 |
| 5 | Govern | 8% | 160,000 |
| 6 | Create | 2% | 40,000 |
Put differently, out of every 100 bachelor's degree recipients, roughly 95 can access AI tools, 65 can write effective prompts, 30 can critically evaluate outputs, 15 can integrate AI with disciplinary expertise, 8 understand governance and risk considerations, and perhaps only 2 can design sophisticated human AI workflows. The sharp decline from access to creation suggests that the most valuable AI skills may also be the rarest.
Under this framework, the United States graduates nearly two million AI users every year but perhaps only forty thousand AI architects.
Such a pattern helps explain why discussions about AI readiness often feel contradictory. Universities observe widespread adoption and conclude that students are adapting rapidly. Employers encounter graduates who can generate content but struggle to evaluate outputs, integrate AI into professional processes, or understand organizational risks. Neither perspective is necessarily wrong. Each reflects a different point on the same competency ladder.
The framework also maps naturally onto Bloom's Taxonomy, one of the most influential models in education.
| Bloom's Taxonomy | AI Competency |
|---|---|
| Remember | Define AI concepts |
| Understand | Explain strengths and weaknesses |
| Apply | Use AI to complete tasks |
| Analyze | Compare outputs and identify flaws |
| Evaluate | Judge reliability and appropriateness |
| Create | Design human AI workflows |
Most surveys effectively measure the Apply stage. Employers increasingly care about Analyze, Evaluate, and Create. The result is a mismatch between what is easy to measure and what is economically valuable.
Viewed through this lens, the much discussed AI skills gap begins to look different. The United States may not suffer from a shortage of AI users, nor does it necessarily lack future AI specialists. The larger challenge may be that too many students remain concentrated at the lower levels of competency, where AI functions primarily as a tool for convenience rather than a tool for judgment.
History offers several useful parallels. Access to books did not create literacy. Access to calculators did not create mathematical reasoning. Access to the internet did not create information literacy. Each technological advance eventually forced educational institutions to move beyond access and develop new standards for competence. Artificial intelligence appears to be following the same trajectory.
The most important AI metric in higher education may therefore have little to do with adoption. Students have already embraced the technology at a pace that would have been difficult to imagine only a few years ago. The more pressing question is whether they have learned to evaluate AI outputs, challenge questionable conclusions, verify claims independently, recognize limitations, and integrate these systems responsibly into their work. Those capabilities, rather than simple usage rates, are likely to determine whether graduates can transfer AI skills successfully into professional settings.
Future surveys will almost certainly continue asking how often students use AI because the answers are easy to collect and compare. A more revealing question, however, would ask how often students can prove the AI wrong. The answer would tell us far more about whether graduates are prepared for the world they are entering and, perhaps more importantly, whether we are measuring AI literacy correctly in the first place.
Further Reading
- Business Insider, Graduates Are Coming into the Workforce Thinking That Using AI Is Cheating, a Top Deloitte Exec Says
- Gallup & Lumina Foundation, College Students' AI Use and Attitudes
- Bloom's Taxonomy of Educational Objectives (Wikipedia)