ai alignment

Paul Christiano is the world’s leading AI safety researcher. My full episode with him is out!

We discuss:

- Does he regret inventing RLHF, and is alignment necessarily dual-use?

- Why he has relatively modest timelines (40% by 2040, 15% by 2030),

- What do we want post-AGI world to look like (do we want to keep gods enslaved forever)?

- Why he’s leading the push to get to labs develop responsible scaling policies, and what it would take to prevent an AI coup or bioweapon,

- His current research into a new proof system, and how this could solve alignment by explaining model's behavior

- and much more.

Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.

Open Philanthropy

Open Philanthropy is currently hiring for twenty-two different roles to reduce catastrophic risks from fast-moving advances in AI and biotechnology, including grantmaking, research, and operations.

For more information and to apply, please see the application: https://www.openphilanthropy.org/research/new-roles-on-our-gcr-team/

The deadline to apply is November 9th; make sure to check out those roles before they close.

Timestamps

(00:00:00) - What do we want post-AGI world to look like?

(00:24:25) - Timelines

(00:45:28) - Evolution vs gradient descent

(00:54:53) - Misalignment and takeover

(01:17:23) - Is alignment dual-use?

(01:31:38) - Responsible scaling policies

(01:58:25) - Paul’s alignment research

(02:35:01) - Will this revolutionize theoretical CS and math?

(02:46:11) - How Paul invented RLHF

(02:55:10) - Disagreements with Carl Shulman

(03:01:53) - Long TSMC but not NVIDIA

Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe

AI advocate Marc Andreessen joins us to clear up misconceptions about AI and discuss its potential impact on job creation, creativity, and moral reasoning.

What We Discuss with Marc Andreessen:

• Will AI create new jobs, take our old ones outright, or amplify our ability to perform them better?
• What role will AI play in current and future US-China relations?
• How might AI be used to shape (or manipulate) public opinion and the economy?
• Does AI belong in creative industries, or does it challenge (and perhaps cheapen) what it means to be human?
• How can we safeguard our future against the possibility that AI could get smart enough to remove humanity from the board entirely?
• And much more...

Full show notes and resources can be found here: jordanharbinger.com/888

This Episode Is Brought To You By Our Fine Sponsors: jordanharbinger.com/deals

Sign up for Six-Minute Networking — our free networking and relationship development mini course — at jordanharbinger.com/course!

Like this show? Please leave us a review here — even one sentence helps! Consider including your Twitter handle so we can thank you personally!

In this episode, we delve into the frontier of AI and the challenges surrounding AI alignment. The AI / Crypto overlap at Zuzalu sparked discussions on topics like ZKML, MEV bots, and the integration of AI agents into the Ethereum landscape. 

However, the focal point was the alignment conversation, which showcased both pessimistic and resigned optimistic perspectives. We hear from Nate Sores of MIRI, who offers a downstream view on AI risk, and Deger Turan, who emphasizes the importance of human alignment as a prerequisite for aligning AI. Their discussions touch on epistemology, individual preferences, and the potential of AI to assist in personal and societal growth.

------
🚀 Join Ryan & David at Permissionless in September. Bankless Citizens get 30% off. 🚀
https://bankless.cc/GoToPermissionless

------
BANKLESS SPONSOR TOOLS:

🐙KRAKEN | MOST-TRUSTED CRYPTO EXCHANGE
⁠https://k.xyz/bankless-pod-q2⁠ 

🦊METAMASK PORTFOLIO | TRACK & MANAGE YOUR WEB3 EVERYTHING
⁠https://bankless.cc/MetaMask 

⚖️ ARBITRUM | SCALING ETHEREUM
⁠https://bankless.cc/Arbitrum⁠ 

🛞MANTLE | MODULAR LAYER 2 NETWORK
⁠https://bankless.cc/Mantle⁠ 

👾POLYGON | VALUE LAYER OF THE INTERNET
https://polygon.technology/roadmap 

------

Timestamps

0:00 Intro
1:50 Guests

5:30 NATE SOARES
7:25 MIRI
13:30 Human Coordination
17:00 Dangers of Superintelligence
21:00 AI’s Big Moment
24:45 Chances of Doom
35:35 A Serious Threat
42:45 Talent is Scarce
48:20 Solving the Alignment Problem
59:35 Dealing with Pessimism
1:03:45 The Sliver of Utopia

1:14:00 DEGER TURAN
1:17:00 Solving Human Alignment
1:22:40 Using AI to Solve Problems
1:26:30 AI Objectives Institute
1:31:30 Epistemic Security
1:36:18 Curating AI Content
1:41:00 Scalable Coordination
1:47:15 Building Evolving Systems
1:54:00 Independent Flexible Systems
1:58:30 The Problem is the Solution
2:03:30 A Better Future

-----
Resources

Nate Soares
https://twitter.com/So8res?s=20 

Deger Turan
https://twitter.com/degerturann?s=20 

MIRI
https://intelligence.org/ 

Less Wrong AI Alignment
xhttps://www.lesswrong.com/tag/ai-alignment-intro-materials 

AI Objectives Institute
https://aiobjectives.org/ 

------

Not financial or tax advice. This channel is strictly educational and is not investment advice or a solicitation to buy or sell any assets or to make any financial decisions. This video is not tax advice. Talk to your accountant. Do your own research.

Disclosure. From time-to-time I may add links in this newsletter to products I use. I may receive commission if you make a purchase through one of these links. Additionally, the Bankless writers hold crypto assets. See our investment disclosures here:
 https://www.bankless.com/disclosures⁠ 

Paul Christiano runs the Alignment Research Center, a non-profit research organization whose mission is to align future machine learning systems with human interests. Paul previously ran the language model alignment team at OpenAI, the creators of ChatGPT. 

Today, we’re hoping to explore the solution-landscape to the AI Alignment problem, and hoping Paul can guide us on that journey. 

------
✨ DEBRIEF | Unpacking the episode: 
https://www.bankless.com/debrief-paul-christiano 
 
------
✨ COLLECTIBLES | Collect this episode: 
https://collectibles.bankless.com/mint 

------
✨ Always wanted to become a Token Analyst? Bankless Citizens get exclusive access to Token Hub. Join Them.
https://bankless.cc/TokenHubRSS  

------
In today’s episode, Paul answers many questions, but the overarching ones are: 
1) How BIG is the AI Alignment problem? 
2) How HARD is the AI Alighment problem?
3) How SOLVABLE is the AI Alignment problem? 

Does humanity have a chance? Tune in to hear Paul’s thoughts. 

------
BANKLESS SPONSOR TOOLS: 

⚖️ ARBITRUM | SCALING ETHEREUM
https://bankless.cc/Arbitrum 

🐙KRAKEN | MOST-TRUSTED CRYPTO EXCHANGE
https://bankless.cc/kraken 

🦄UNISWAP | ON-CHAIN MARKETPLACE
https://bankless.cc/uniswap 

👻 PHANTOM | FRIENDLY MULTICHAIN WALLET
https://bankless.cc/phantom-waitlist 

🦊METAMASK LEARN | HELPFUL WEB3 RESOURCE
https://bankless.cc/MetaMask 

------
Topics Covered

0:00 Intro
9:20 Percentage Likelihood of Death by AI
11:24 Timing 
19:15 Chimps to Human Jump
21:55 Thoughts on ChatGPT
27:51 LLMs & AGI
32:49 Time to React?
38:29 AI Takeover 
41:51 AI Agency 
49:35 Loopholes 
51:14 Training AIs to Be Honest 
58:00 Psychology 
59:36 How Solvable Is the AI Alignment Problem?
1:03:48 The Technical Solutions (Scalable Oversight) 
1:16:14 Training AIs to be Bad?! 
1:18:22 More Solutions
1:21:36 Stabby AIs 
1:26:03 Public vs. Private (Lab) AIs
1:28:31 Inside Neural Nets
1:32:11 4th Solution 
1:35:00 Manpower & Funding 
1:38:15 Pause AI?
1:43:29 Resources & Education on AI Safety 
1:46:13 Talent  
1:49:00 Paul’s Day Job
1:50:15 Nobel Prize 
1:52:35 Treating AIs with Respect 
1:53:41 Uptopia Scenario 
1:55:50 Closing & Disclaimers 

------
Resources:

Alignment Research Center
https://www.alignment.org/ 

Paul Christiano’s Website
https://paulfchristiano.com/ai/ 

-----
Not financial or tax advice. This channel is strictly educational and is not investment advice or a solicitation to buy or sell any assets or to make any financial decisions. This video is not tax advice. Talk to your accountant. Do your own research.

Disclosure. From time-to-time I may add links in this newsletter to products I use. I may receive commission if you make a purchase through one of these links. Additionally, the Bankless writers hold crypto assets. See our investment disclosures here:
https://www.bankless.com/disclosures 

Debriefing the episode with Eliezer Yudkowsky. This one was so good, we had to share. The fate of humanity might depend on it.

The Debrief Episode goes out EVERY MONDAY for Bankless Citizens. Want the Debrief Episode? Get the Premium RSS feed by Subscribing to Bankless!

WATCH THE FULL EPISODE HERE:
https://youtu.be/gA1sNLL6yg4 

------
🚀 SUBSCRIBE TO NEWSLETTER:          https://newsletter.banklesshq.com/ 

-----
Not financial or tax advice. This channel is strictly educational and is not investment advice or a solicitation to buy or sell any assets or to make any financial decisions. This video is not tax advice. Talk to your accountant. Do your own research.

Disclosure. From time-to-time I may add links in this newsletter to products I use. I may receive commission if you make a purchase through one of these links. Additionally, the Bankless writers hold crypto assets. See our investment disclosures here:
https://www.bankless.com/disclosures 

Eliezer Yudkowsky is an author, founder, and leading thinker in the AI space.

------
✨ DEBRIEF | Unpacking the episode: 
https://shows.banklesshq.com/p/debrief-eliezer 
 
------
✨ COLLECTIBLES | Collect this episode: 
https://collectibles.bankless.com/mint 

------
We wanted to do an episode on AI… and we went deep down the rabbit hole. As we went down, we discussed ChatGPT and the new generation of AI, digital superintelligence, the end of humanity, and if there’s anything we can do to survive.

This conversation with Eliezer Yudkowsky sent us into an existential crisis, with the primary claim that we are on the cusp of developing AI that will destroy humanity. 

Be warned before diving into this episode, dear listener. Once you dive in, there’s no going back.

------
📣 MetaMask Learn | Learn Web3 with the Leading Web3 Wallet https://bankless.cc/

------
🚀 JOIN BANKLESS PREMIUM: 
https://newsletter.banklesshq.com/subscribe 

------
BANKLESS SPONSOR TOOLS: 

🐙KRAKEN | MOST-TRUSTED CRYPTO EXCHANGE
https://bankless.cc/kraken 

🦄UNISWAP | ON-CHAIN MARKETPLACE
https://bankless.cc/uniswap 

⚖️ ARBITRUM | SCALING ETHEREUM
https://bankless.cc/Arbitrum 

👻 PHANTOM | #1 SOLANA WALLET
https://bankless.cc/phantom-waitlist 

------
Topics Covered

0:00 Intro
10:00 ChatGPT
16:30 AGI
21:00 More Efficient than You
24:45 Modeling Intelligence
32:50 AI Alignment
36:55 Benevolent AI
46:00 AI Goals
49:10 Consensus
55:45 God Mode and Aliens
1:03:15 Good Outcomes
1:08:00 Ryan’s Childhood Questions
1:18:00 Orders of Magnitude
1:23:15 Trying to Resist
1:30:45 Miri and Education
1:34:00 How Long Do We Have?
1:38:15 Bearish Hope
1:43:50 The End Goal

------
Resources:

Eliezer Yudkowsky
https://twitter.com/ESYudkowsky 

MIRI
https://intelligence.org/

Reply to Francois Chollet
https://intelligence.org/2017/12/06/chollet/ 

Grabby Aliens
https://grabbyaliens.com/ 

-----
Not financial or tax advice. This channel is strictly educational and is not investment advice or a solicitation to buy or sell any assets or to make any financial decisions. This video is not tax advice. Talk to your accountant. Do your own research.

Disclosure. From time-to-time I may add links in this newsletter to products I use. I may receive commission if you make a purchase through one of these links. Additionally, the Bankless writers hold crypto assets. See our investment disclosures here:
https://www.bankless.com/disclosures