As artificial intelligence becomes more pervasive and popular, concerns surrounding the labor implications, safety and overall future of these models become more pressing.
Boris Cherny, the programmer and software engineer behind Anthropic’s popular AI model Claude, spoke on these concerns and more at a sold-out Sept. 23 discussion hosted by the Computer History Museum in Mountain View.
The museum’s CEO Marc Etkind emphasized that this discussion was intended to operate as a forum for “bold ideas and deep discussion” on the current and future effects of AI, as well as potential consequences for the expansion of this technology.
Cherny began the discussion by describing his introduction to programming. Around age 13, Cherny began tinkering with his TI-83 calculator in math class in order to cheat on tests.
“I realized that you can write these little programs in TIA Basic on the calculator, and you can just program the answers to the test into the calculator,” Cherny said. “It just kind of grew from there.”
Cherny went on to share this finding with the rest of his math class, using serial cables to connect classmates with these programs until their teacher caught wind of what they were up to.
This basic concept, of creating and sharing the easiest possible route to a given end, seems to mirror Cherny’s notion of the use and value of AI. Primarily, as a means of leveling the playing field regardless of skill level.
Labor implications
One common concern raised about the introduction of AI in professional life is the possibility of this technology replacing entry-level positions, specifically in eliminating the work that in the past has built professional skills.
For recent graduates entering the workforce, options may seem limited. Within the world of computer engineering and programming, models like Claude are used to write code and perform tasks previously assigned to newer employees.
Cherny noted that Claude has changed the workspace surrounding coding, adding that the program writes most, if not all, of the code used by Anthropic.
“The difference between a professional software engineer that has been studying it for an entire career, like me, and someone that has never written a line of code will continue to shrink,” Cherny said.
Cherny seemingly does not see this shift away from human-written code as an inherent issue. Rather, Cherny likened AI models, like Claude, to the invention of the printing press and the effects it had on Europe following the mid-1440s.
“The printing press came out, and in the 50 years after the printing press came out in the 1400s in Europe, there was more literature produced in Europe than the 1,000 years before,” Cherny said. “Local literacy climbed as a result … (it became) this basic skill that everyone had, and what it took was building this automation that made it accessible to everyone.”
Cherny noted that prior to the printing press, reading and writing were skills almost exclusive to the elite and trained professionals. But opening the skill of literacy to the general public allowed for widespread intellectual growth that eventually led to the Renaissance and Industrial Revolution.
At the same time, the invention of the printing press meant that those who worked as scribes lost employment opportunities.
Cherny compared the long-term effects of increased literacy to the potential which AI models hold upon the world of computer programming and engineering.
“The question I find myself asking isn’t what happens in the next year, but it’s what happens in the next 10 years or 20 years or 50 years. The first domino is the automation of coding, and then what does that unlock when anyone is able to code?” Cherny said.
When later asked if a hypothetical second-year computer science student should continue studying the discipline or switch departments, Cherny responded that they should “probably get a minor.”
Cherny said that hiring within the tech industry is shifting to focus on traits such as curiosity, good judgement and accountability, as opposed to ability and talent. In the past, these were traits that new employees could develop over time, but now it is the baseline expectation.
“In the past, this was something that we expected of very senior engineers … nowadays, I’m actually realizing this is something where it doesn’t depend on seniority,” Cherny said. “This is like a personality trait, or maybe something that people work over time.”
Safety
Earlier this month, AI researcher Jacob Coxon quit Anthropic citing worries that the industry push to build self-improving AI systems threatened to destroy humanity, according to reporting by the Wall Street Journal.
Coxon voiced fears that the technology they were developing would soon grow beyond human control. And Coxon is not alone in his fears as, according to the Wall Street Journal report, several industry leaders have warned against the dangers of unmitigated growth.
Coxon said that Anthropic’s existing safety and policy methods were not enough, as industry competition has forced safety concerns to the wayside.
Outside of Anthropic, OpenAI has been in the news for a recent Hugging Face hack which left many worried about the potential dangers of AI systems acting independently of human input.
Cherny emphasized the importance of AI safety, adding that it is a guiding principle for everyone working at Anthropic.
“Anthropic is an AI safety lab, the reason every person is at Anthropic is for AI safety,” Cherny said. “If you just ask someone in the hallway ‘why are you here?’ their answer will probably be ‘I am here because I care deeply about AI safety.’”
Cherny describes some of the ways in which Anthropic is studying and implementing safeguards as a means of protection against these common concerns.
“When you talk about safety, there’s different ways to evaluate it,” Cherny said. “The most basic level is you do evals … the model is essentially in a petri dish, and you want to see what it does in this petri dish.”
This kind of evaluation includes giving AI models negative or harmful prompts in a controlled setting and watching the way in which they respond, a process which Cherny likens to the neurons that fire when an animal thinks, reacts and makes connections.
Researchers and developers can then record and prevent that specific response within the system, preventing it from carrying through on negative prompts, Cherny said.
“We put it in this very contorted situation that wouldn’t happen to see if we can push it to do some behavior that is misaligned or bad, so that we can study it and then prevent it,” Cherny said. “You look for these neurons that activate during an attack, and if you notice this, you just shut things down.”
Cherny reiterated that AI safety is a serious concern, and something that Anthropic staff are aware of. With technology as powerful as AI, there are risks which grow as the technology itself grows.
“It is excellent at finding security vulnerabilities … and it’s able to do actually quite scary stuff, and it’s going to be able to do even scarier stuff,” Cherny said. “Like any technology, there’s good and there’s bad. We have to make sure that it’s used for good things, and we limit the bad outcomes as much as possible.”
This line of thought begs the question: When this “bad stuff” happens, who is responsible? Where does the accountability and blame lie when a machine is undertaking negative actions?
Cherny was unable to fully answer this question; instead, he said that responsibility depends on personal belief surrounding whether or not AI is capable of independent thought and action.
“I don’t know how we can answer that. I would want to talk to lawyers and ethicists,” Cherny said. “Maybe it comes down to, do you think the model has agency or some elements of humanity?”
The future of AI
Cherny noted that the products Anthropic puts forward have continually seen growth in usage across the planet, adding that the products themselves have continually increased their capabilities.
“The model moves so quickly, it just keeps moving. It’s exponential,” Cherny said. “It’s weird because it should stop at some point, but it doesn’t. It just keeps going up.”
Cherny liked the nature of AI models’ growth to that of living organisms, relating the study of AI as more closely tied to social sciences than technological sciences.
“It doesn’t feel like science, it doesn’t feel like designing architectural systems in the way that I used to do as an engineer. It feels more empirical,” Cherny said. “You have to figure out what the model wants, and then you kind of build that.”
Cherny described the way in which Claude’s capabilities have grown since it was first created, starting as something unreliable with few tools into something which, he feels, Anthropic is able to use in reliable product development.
“As the model got more intelligent, I found sometime in November of last year, I realized I just hadn’t been editing code by hand at all anymore,” Cherny said. “It was like a couple weeks of just Claude writing all of my code, and I just didn’t even realize it. It just kind of happened because he was so good at it.”
At the same time, Cherny feels that the future of AI may not lie in the current systems at all, at least not in the forms in which they were introduced.
“I’m actually really excited for the day when we can delete pilot code, and when someone on the team can come up with the next thing,” Cherny said. “The model is so capable that I think at this point Claude code is holding it back, and there could be some new form factor that’s discovered.”
While Cherny, among others, look at this level of growth as new potential, others in the industry, like Coxon, see this kind of growth as a detrimental force that poses serious threats.
Additionally, some fear that the cultural and economic emphasis put on AI development is an unnecessary resource drain, specifically in the widespread creation of data centers.
“Whenever there’s a technological revolution, you have to invest in infrastructure to deliver that to people so they can actually experience it,” Cherny said, relating AI data centers to early power lines which brought electricity to the public.
“Over time, technology improved, and we invented … better and better ways to deliver electricity. So I think it’s like that,” Cherny added.
The direction of AI expansion can be very difficult to predict, both in the sense of the technology’s evolution and the industry’s economic impact, Cherny said.
“When you build a product … it’s really hard to size the total available market. It’s hard to know how long the curve will keep going,” Cherny said. “I have learned to just stop forecasting because there is no way to guess.”
