Old talks, 2000 - 2020 period.

A list of talks that happened in 2000 - 2020 period, for the future people to just look up or pop up in search results.

The links are collected at the start of 2026.

Many of those links took a lot of effort to dig up, but then I abandoned this writeup and just posting it now to preserve it I guess.

It would be nice of you to point some AI at this to restructure it and assemble more comprehensive archive, with independently stored videos also.

Singularity Summitshttps://intelligence.org/singularitysummit/

A lot of link rotted links, but not to majority of vimeo recordings thankfully.

https://nickbostrom.com/media.pdf

Nick Bostrom lists 504 interviews on 1992 - 2012, 493 over 13 of those years, one interview per 13 days. That's a lot of interviews.

Hard AI Future Salon, Eliezer Yudkowsky 2006https://archive.org/details/FutureSalon_02_20066/10Very good actually, and the questions were too, good job secondary characters.Some questions I was not there to ask:

If you paint like 100 digit random number on Corolla, does it become a narrower target? Do you make a distinction of narrowness and utility?

It's really weird that population size doesn't mater for information gain? Obviously if you have larger pop in evolutionary algorithm, you get higher reward? I should probably just check out the paper referenced.

People do copy parts of the companies, approaches, methods that proved to work in real world. So, in some sense there is similar mechanism that makes random picks, and argmax picks, iteratively and in different proportions. Maybe humans are just a better substrate for evolution, so that the same (few) bits of ground truth obtained from failed / successful tests, get utilized better.

Eliezer Yudkowsky: The Challenge of Friendly AI 20071/10https://www.youtube.com/watch?v=nkB1e-JCgmYhttps://youtu.be/b_4fCIoUnNshttps://www.youtube.com/watch?v=Zfjo_hdNXik

Mostly the previous one? but with more talk about what exactly AIs should do about all of this

Paul Saffo: Machines of Loving Grace: Anticipating Advanced AI 20071/10https://vimeo.com/showcase/1783010?video=33960673

GCR 2008: Elizer Yudkowsky - (Artificial) Intelligence: The Wild Card 20084/10https://vimeo.com/4425926

Pretty interesting, goes in bit of detail on strategic implications.

GCR 2008: Elizer Yudkowsky: Rationally Considering the End of the Worldhttps://vimeo.com/4352818

Anders Sandberg, Global Catatrophic Risks: An Overview, and Caution about Risk Assessments 2008https://vimeo.com/2510525?fl=pl&fe=cm

GCR 2008: Robin Hanson - Catastrophe, Social Collapse and Human Extinction 2008https://vimeo.com/4390513?fl=pl&fe=cm

Eliezer Yudkowsky and Scott Aaronson 20091/10https://bloggingheads.tv/videos/2220https://www.youtube.com/watch?v=EoqpN_Ejqfk

Eliezer premoved like in chess, with "detached lever" against "reprodictive urges analogy", but it was a miss imo.

Scott says thousands of years to AGI, which is insane and has similar good old insane motivation as the people who said similar things about heavier than air flight.

Mostly a discussion of Many Worlds.

Shane Legg at The Singularity Summit 2010 -- Universal measures of intelligence3/10https://vimeo.com/17553536

Somewhat interesting.

Nobody asked a question about like, how hardcoded environment distribution was.

Also the usual stuff, like, how do you point it? IF you succeed at your task of amping up its intelligence, what then. Do you have just like a superintelligent reward function written in code optimizer? Yeah, you are dead.

Shane Legg and Dwarkesh Patel 2023https://youtu.be/Kc1atfJkiJU

https://vimeo.com/17513355

Good things are good. etc etc

Eliezer Yudkowsky at Singularity Summit 2011 - Open Problems in Friendly Artificial Intelligence2/10https://vimeo.com/158653748

Mostly about introduction to decision theory hobby horses.

"So, do I think you ought to wait? I think that realistically speaking, they won't wait. I mean, talked to these people... laughter And so I just like to go run off and solve the problems ourselves, very quickly, before they go ahead and do things anyway. Which they will. [...] And that's what Singularity Institute all about." c Eliezer v2011

Eliezer Yudkowsky "Friendly AI" 2011https://www.youtube.com/watch?v=Uoda5BSj_6o

Vague memory of something turning out to be wrong about group selection points.

Random Schmidhuber dunk, I approve, after seeing a few his old talks.

The outcome pump, notoriously counterproductive as an intuition pump.

Eliezer Yudkowsky on Friendly Artificial Intelligence 20131/10https://www.youtube.com/watch?v=7c1WSwqwMOE

Recommends A Modern Approach 3rd edition.

Basically a worse version of 2006 talk.

Some interesting thought on evolution again, although a bit shaky on its own grounds.

Sound is borked, slightly.

Intelligence Explosion Microeconomicshttps://intelligence.org/files/IEM.pdf (did not read)

(2) no, I mean that GPT-3 looks like a village idiot but it's actually dumber than a village idiot but has an extra million years of practice. 4o maybe begins to be as deep-smart as a village idiot, but it looks like a PhD because it has a million years of added practice pretending to be a PhD. These are not confident analogies and may be eyeballing the levels completely wrongly, since these abstractions are not ones that EA-level intelligences can operate and accordingly there is no work on inventing or measuring metrics for them.I did not, in my previous visualization of the future, predict the part about the extra million years of added practice being impactful. In 2005, I expected conversational intelligence to operate closer to Mu Zero on the algorithmic spectrum running from GPT3 to Mu Zero.The basic LLM paradigm already switched to add RL and inference. Asking "how would the old thing scale without new discoveries?" is and always has been a dumb question because there are always new discoveries, and that was the case since before AlphaGo.(c) Eliezer v2025

(2) no, I mean that GPT-3 looks like a village idiot but it's actually dumber than a village idiot but has an extra million years of practice. 4o maybe begins to be as deep-smart as a village idiot, but it looks like a PhD because it has a million years of added practice pretending to be a PhD. These are not confident analogies and may be eyeballing the levels completely wrongly, since these abstractions are not ones that EA-level intelligences can operate and accordingly there is no work on inventing or measuring metrics for them.

I did not, in my previous visualization of the future, predict the part about the extra million years of added practice being impactful. In 2005, I expected conversational intelligence to operate closer to Mu Zero on the algorithmic spectrum running from GPT3 to Mu Zero.

The basic LLM paradigm already switched to add RL and inference. Asking "how would the old thing scale without new discoveries?" is and always has been a dumb question because there are always new discoveries, and that was the case since before AlphaGo.(c) Eliezer v2025

Eliezer Yudkowsky, given at New York University on October 14, 2016https://youtu.be/93sYbHDtv9Mhttps://youtu.be/YicCAgjsky83/10

"Nobody going to put frontier research AI in charge of paperclip factory!" - Eliezer v2017

Eliezer Yudkowsky – AI Alignment: Why It's Hard, and Where to Start 20166/10https://www.youtube.com/watch?v=EUjc1WuyPT8

Gives pretty good pointers at some papers in the end, check this out, future me. Especially open source game theory, sounds fun.

That's the video YouTube shoves at you randomly or when you google his name. Canonical in some sense.

I think what happened with cauldron isn't "prefer it being filled" because then it would be full and then guarded, not cycling through as much volume per unit of time as possible. "Preferring it being filled as an action"

"You might ordinarily expect that whoever has taken on the job of building an AI is just naturally going to try to point it in a relatively nice direction. They are not going to make evil AI, they are like, not cackling villains. [...]"

The example with compressing-decompressing is probably wrong? You'd need to define utility on your observations, not world model, but then you'd better to see nothing at all?

Yeah Altman definitely wants to be emperor. How awful would that be. IMO asi killing us all would be a million times better. (c) @MemoMiiVRPossibly! It would be a hard call rather than an easy call in real life! But since Altman has already betrayed the people he worked with a few times, I think it makes sense for society to treat him as his worst-case scenario. If one doesn't like that, don't betray. (c) Eliezer v2023

Yeah Altman definitely wants to be emperor. How awful would that be. IMO asi killing us all would be a million times better. (c) @MemoMiiVR

Possibly! It would be a hard call rather than an easy call in real life! But since Altman has already betrayed the people he worked with a few times, I think it makes sense for society to treat him as his worst-case scenario. If one doesn't like that, don't betray. (c) Eliezer v2023

Heuristics and Biases, Skepticon 4 Eliezer Yudkowsky 2017https://www.youtube.com/watch?v=Q0YB5wxtODA

Yudkowsky vs Hanson — Singularity Debate 2017https://youtu.be/TuXl-iidnFY

Paul Christiano – Training an Aligned RL Agent – CSRBAI 20161/10https://www.youtube.com/watch?v=rperZzDssDE

Jessica Taylor – Alignment for Advanced Machine Learning Systems – CSRBAI 20161/10https://www.youtube.com/watch?v=_sGTqI5qdD4(watched a bit, and skimmed the transcript)

https://intelligence.org/files/AlignmentMachineLearning.pdf(did not read)

Jessica Taylor – Using Machine Learning to Address AI Risk – EAG 2016https://www.youtube.com/watch?v=TSe3p1zIvVI

Current work in AI alignment | Paul Christiano | EA Global: San Francisco 20192/10https://www.youtube.com/watch?v=-vsYtevJ2bc

Superhuman AI: If Anyone Builds It ... Everyone Dies? Eliezer Yudkowsky and Nate Soares 2025https://www.youtube.com/watch?v=WdXYQbT6ueo

Demis Hassabis "Systems neuroscience and AGI" 2011https://www.youtube.com/watch?v=IjG_Fx3D0o0

Eliezer Yudkowsky, lost GT talk 2013https://scottaaronson.blog/?p=1558

Andrew Critch – Robust Cooperation of Bounded Agents – CSRBAI 2016https://www.youtube.com/watch?v=WG_Krd-wGM4

Demis Hassabis "Systems neuroscience and AGI" 2011https://www.youtube.com/watch?v=IjG_Fx3D0o0

Stuart Armstrong – Reduced Impact AI and Other Alternatives to Friendliness – CSRBAI 2016https://youtu.be/3wsiUkmC6dI

Positive Outcomes for AI | Nate Soares | Talks at Google 20173/10https://youtu.be/dY3zDvoLoao

Same slides with Cauldron, but goes more extensively on strategic implication at the Q&A.

Random panelhttps://youtu.be/fWBBe13rAPU

1/10 but Russel is based here.

https://www.youtube.com/watch?v=PwhFU_YAASo

Yudkowsky vs Hanson — Singularity Debate 2017https://www.youtube.com/watch?v=TuXl-iidnFY

How I think students should orient to AI safety | Buck Shlegeris | EA Student Summit 2020https://www.youtube.com/watch?v=R6Mzt4GwQnQ

The current alignment plan, and how we might improve it | Buck Shlegeris | EAG Bay Area 2023https://youtu.be/YTlrPeikoyw

3/10 Informative about his perspective I guess. Very low expectations. Dubious strategic reasoning, e.g. "We should make it easier for Frontier AI Company to do alignment, as if it surely will spend the counterfactually same resources on alignment, instead of racing faster." Not addressing the "RL will imbue them with weird preferences that will expand only once they are smart and powerful enough to make more option for themselves, and will not show on training and tests", countered with iterative progress, implicitly.

0/10 Irrelevant.

Percontations: The Nature of Probability | Eliezer Yudkowsky & Andrew Gelman 2018https://www.youtube.com/watch?v=Kuzfkyaj1oQ

https://www.youtube.com/watch?v=tUAXCFvvODI

https://www.youtube.com/watch?v=-mXVKLrgBwYAre Super Intelligent Computers Really A Threat to Humanity? 2015https://youtu.be/fWBBe13rAPUScience Saturday: Dreaming of an Artificial Intelligence | Eliezer Yudkowsky & Jaron Lanier 2018https://www.youtube.com/watch?v=Ff15lbI1V9M

did not watch, sounds dreadfulHalfway through, Jaron is so fundamentally wrong about everything, he's one of the most painful people I've listened to in years. (c) random YouTube comment

Halfway through, Jaron is so fundamentally wrong about everything, he's one of the most painful people I've listened to in years. (c) random YouTube comment

Halfway through, Jaron is so fundamentally wrong about everything, he's one of the most painful people I've listened to in years. (c) random YouTube comment

Science Saturday: The Great Singularity Debate | Eliezer Yudkowsky & Massimo Pigliucci 2018https://www.youtube.com/watch?v=onvAl4SQ5-Q

https://arxiv.org/pdf/1401.5577

AI Creation and the Cosmic Host | Nick Bostrom | AIADM London 20254/10https://www.youtube.com/watch?v=43tpDbfBKWs

Demis Hassabis: creativity and AI – The Rothschild Foundation Lecture 2018https://youtu.be/d-bvsJWmqlc

Safety evaluations and standards for AI | Beth Barnes | EAG Bay Area 2023https://www.youtube.com/watch?v=Vb5g7jlNzOk

3/10 "METR, the time horizons graph company?" Also it's 2023 they worked on very dumb AIs. Interesting strategic stuff. Didn't mention capability acceleration pathway of providing a good metric to optimize over.

Vernor Vinge and Bob Pisani: Conversation on the Singularityhttps://vimeo.com/showcase/1796901?video=34694288

A Brief Summary of Recent Happenings at the Singularity Institutehttps://www.lesswrong.com/posts/uJDpRFycS3Qp89cxz/a-brief-summary-of-recent-happenings-at-the-singularity

I googled this author, as I had no memory of him, and instantly got like 10 psychic damage points from looking at his twitter profile.

https://www.reddit.com/r/lectures/comments/2dxh6n/jaan_tallinn_why_now_a_quest_in_metaphysics/

Singularity Institute Executive Director Q&A #2 by lukeproghttps://www.lesswrong.com/posts/DWDA33qGTMX9ZJejp/singularity-institute-executive-director-q-and-a-2

https://www.lesswrong.com/posts/ugqkxS7jdoYXR9reg/introductory-short-videos

Scott Aaronson | How Much Math Is Knowable?https://youtu.be/VplMHWSZf5c

"Study science, not just me!" is probably the most important piece of advice Ayn Rand should've given her followers and didn't. (c) Eliezer v2007It's plausible I should've just gone full Kelthorkarnen. (c) plausible deniable Eliezer v2021

"Study science, not just me!" is probably the most important piece of advice Ayn Rand should've given her followers and didn't. (c) Eliezer v2007

It's plausible I should've just gone full Kelthorkarnen. (c) plausible deniable Eliezer v2021

https://arxiv.org/pdf/1606.06565

https://futureoflife.org/newsletters/

https://intelligence.org/workshops/#april-2016-tt

https://www.lesswrong.com/posts/rCZ9fruWriD6uGNLp/who-are-some-prominent-reasonable-people-who-are-confident

https://ai-alignment.com/directions-and-desiderata-for-ai-control-b60fca0da8f4

https://agentfoundations.org/item?id=596https://agentfoundations.org/item?id=582https://agentfoundations.org/item?id=613

https://intelligence.org/2016/07/27/alignment-machine-learning/

https://intelligence.org/nyu-talk/

https://intelligence.org/files/ValueLearningProblem.pdf

https://web.archive.org/web/20241213125154/https://intelligence.org/technical-agenda/

https://forum.effectivealtruism.org/posts/GA7ytcMeRQe5b27ge/ask-miri-anything-ama

https://intelligence.org/2017/04/07/decisions-are-for-making-bad-outcomes-inconsistent/

! video 2017https://intelligence.org/2017/04/12/ensuring/

https://intelligence.org/2017/11/25/security-mindset-ordinary-paranoia/https://intelligence.org/2017/11/26/security-mindset-and-the-logistic-success-curve/

https://intelligence.org/2016/08/05/miri-strategy-update-2016/https://intelligence.org/2015/12/04/new-paper-proof-producing-reflection-for-hol/https://intelligence.org/2013/10/25/bas-steunebrink-on-sleight/https://intelligence.org/files/ValueLearningProblem.pdfhttps://intelligence.org/files/TilingAgentsDraft.pdf

Rob Wiblin & Ajeya Cotra 2026https://youtu.be/Z19UEZHJzAg

1/10 uninformative.

Garrabrant and Shah on Human Models in AGI 2021https://www.youtube.com/watch?v=tSQWMNQBtAg&t=2s

187 Stuart Armstrong and Scott Garrabrant: If I were a Well-intentioned AI --2020https://www.reddit.com/r/ControlProblem/comments/h1094d/187_stuart_armstrong_and_scott_garrabrant_if_i/