When ChatGPT launched in November 2022, it became clear that Large Language Models (‘LLMs’) will cause huge disruption. Now Bing has incorporated GPT-4 and Google has semi-launched Bard, it’s clear that disruption will include SEO.

It’s impossible to predict how this will play out, maybe:

• The web will be flooded with mediocre LLM content and search engines will aggessivley filter it
• Bing will grab huge market share
• Google won’t drive traffic to the sites it scrapes for it’s index / corpus and the search ecosystem breaks
• Searchers won’t like LLM or chat as a UI, and it’s just another productivity tool

🤷

Whatever happens it’s going to be imporant to understand the technology.

Books 📚

Make Your Own Neural Network: Learn how to build the kind of neural network that underlie the ChatGPT, Bard etc from scratch. Requires basic Python knowlege, but is very clearly written.

Key papers 📝

Attention Is All You Need (2017): Introduced the transformer archtecture that all recent LLM progress has been buit on.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2019): Seminal paper from Google introducing bidirectional transformers. Useful for SEOs as it has been used by Google to improve query understanding.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020): Introduced using a neural network to retreive data from external sources e.g. a search engine index. Very likley how Bing is providing sources for it’s GPT-4 integration.

TALM: Tool Augmented Language Models (2022): Adding ‘pluginss - capabilities beyond language to LLMs.

GPT-4 Technical Report (2023): Describes the development of GPT-4, as used by Bing.

Podcasts 🎙️

Machine Learning Street Talk

Courses 🎓

Machine Learning Specialization by Andrew Ng

Newsletters 🗞️

Import AI: The latest AI news from industry and academia. Comes with a free short story inspired by AI, which is very cool.

TLDR > AI A daily email summary of AI launches, news, papers etc.

Social 😄

Front page of Hacker News: I’m going to waste time on social media anyway, so I may as well learn something.

Header image generated by Dalle-2.