Data Point 04/24

Huge Meta Release, New Organizations, Modernizing Government

Welcome to the second week of Data Point!

This week’s podcast is with Keegan McBride, a lecturer at Oxford and an Alliance for the Future policy advisor. We discuss exactly that: how modernizing the government succeeded in Estonia and what it would take for it to do the same in the US.

Policy

We published a research summary on why AI is likely to follow historical patterns of scientific development. This refutes the idea that AI will autonomously make a discontinuous jump in ability. “Because returns diminish superexponentially, a slowdown in science occurs despite the theoretical ability for self-improvement ability to grow exponentially.”

The Abundance Institute, a technology-focused offshoot of the Center for Growth and Opportunity, launched this Monday. A completely different Inclusive Abundance Institute launched on the same day. The winds seem to be shifting in a more pro-tech direction.

Google released a massive 250-page AI Ethics paper. It is very dense with serious treatment of the broader societal and philosophical questions around AI. It is worth reading if you are curious about these philosophical questions, but may be difficult and unrelenting if you aren’t. 

Industry

Meta deployed and open-sourced 8B and 70B models of Llama 3. Try it here. They are also working on multimodal. 

In an interview with Dwarkesh Patel, Mark Zuckerberg disclosed that the yet to be released Llama 3 405B is a dense model. An oversimplified explanation of why it matters is that it indicates Meta rejects newer algorithmic approaches and focuses instead on increasing the scale of older algorithms.

Research

Chatbot Arena published a new benchmark using user interaction data. This is a good reason to introduce you to Arena, which is a platform for users to rate and compare LLMs. It uses an ELO ranking based on real user feedback, which is a better way of rating performance than fixed benchmarks in my opinion.

Alongside Llama 3, Meta also released a new cybersecurity eval.