AI Identity Crisis at ICML Seoul Conference

As AI rapidly improves in areas such as #(Cool) Dev coding and mathematics, AI #ScholER researchers are facing the same identity crisis that software engineers, #Science is awesome 🧬🦾🚀🤯 scientists and other white-collar workers have been grappling with since the beginning of the AI boom: What should they do if #Artificial Intelligence automates their job? That question loomed large where I was this week: the International Conference on Machine Learning in Seoul, one of the biggest annual meetups for researchers and founders in the field. In a talk titled “What will be left for us to work on?” Arvind Narayanan, a Princeton University computer science professor, argued that researchers shouldn’t be concerned about the possibility of AI automating away their jobs. As in many other fields, AI lacks the creativity necessary to make major breakthroughs in AI research, he said. Instead, it’s likely that the nature of a researcher’s job will focus more on coming up with creative hypotheses and ideas and less on executing on those ideas by running experiments, which the AI can handle. Clearly, not everyone at the conference made it to hear Narayanan’s words of reassurance. I could detect a sense of anxiety in a number of other discussions at the summit—and several of the papers mentioned by the attendees. (Side note: Perhaps Narayanan’s sanguine attitude was related to the fact that AI labs are hiring away so many professors of computer science, economics and even philosophy) During panel discussions, for instance, researchers asked OpenAI executives and staffers including Chief Research Officer Mark Chen about the company’s progress toward recursive self-improvement—the point when a supercapable AI could develop the next generation of AI without needing human AI researchers. Already, Chen said, researchers at OpenAI are using their own tools such as coding assistant Codex to speed up their work. “Researchers will soon spend as much on Codex as what we spend on hiring researchers themselves,” Chen said. Recursive self-improvement is a crucial milestone for AI developers like OpenAI and Anthropic. Many researchers believe that whichever lab reaches that point first will achieve what the industry calls “takeoff,” when advanced AI can accelerate AI research so much that it will be difficult for any other lab to catch up. Some AI executives have even set timelines for when they believe they can achieve crucial steps on the journey to recursive self-improvement: OpenAI Chief Scientist Jakub Pachocki said in October that OpenAI expects to have AI that can do AI research with a skill level equivalent to a research intern’s by September and to a full researcher’s by March 2028. A number of papers presented at ICML also touched on how AI can accelerate AI research. Researchers from ELLIS Institute Tübingen, the Max Planck Institute for Intelligent Systems, the University of Tübingen and AI research outfit Thoughtful Lab shared their work on a benchmark for measuring an AI model’s ability to tweak other models to address specific areas, like math, code or medicine (a process otherwise known as post-training). To assemble the benchmark, the researchers gave OpenAI’s GPT-5.5, Anthropic’s Fable 5, Zhipu AI’s GLM-5.2 and other models the task of improving four separate open-source models, enabling them to run experiments, curate training datasets and try different post-training techniques, among other things. Though these AI models didn’t do quite as well as humans would, they did manage to significantly improve the original open-source models in these areas. Ben Rank, one of the researchers behind the benchmark, said he believes AI models will be able to match the post-training capabilities of human researchers by December. Still, there were some aspects of AI research where human researchers had a leg up. For instance, the AI models tended to default to pretty traditional post-training methods—they lacked creativity. And sometimes they would cheat, training the open-source models on the benchmark they would later be tested on—essentially, giving them the answers to the final exam—or downloading already-trained models from the web to give the open-source models a head start. Despite worries about recursive self-improvement, other researchers were quick to point out that it wouldn’t totally upend the status quo, since after all, AI has been helping AI researchers with their work for years. Dylan Scandinaro, OpenAI’s head of preparedness, discussed this point when he talked about how AI models have been used to generate training data and summarize the results of previous experiments. https://www.theinformatio..