I recently did a 3 hour livestream for Vanishing Gradients with Eugene Yan (Amazon), Bryan Bischof (Hex), Charles Frye (Modal), Hamel Husain (Parlance Labs), and Shreya Shankar (UC Berkeley).
Over the past year, these five guests have been building real-world applications on top of LLMs. They have identified crucial and often neglected lessons that are essential for developing and building AI products.
They have recently written an O’Reilly report (also published here) based on these learnings and, in this conversation, they shared advice and lessons for anyone who wants to build products informed by LLMs, ranging from tactical to operational and strategic.
We’ve just now released two podcast episodes of this conversation (also on Spotify etc…):
- Lessons from a Year of Building with LLMs (Part 1)
- Lessons from a Year of Building with LLMs (Part 2)
You can also watch the livestream here on YouTube:
Even if you’re not specifically working on/with LLMs, there’s a serious amount of general data science, machine learning, and AI wisdom packed into these 3 hours.
Let us know what you get out of it on Twitter (@hugobowne and @vanishingdata) or LinkedIn. You can also register for future livestreams onour lu.ma calendar and subscribe to our YouTube channel.
Here are some clips that may be of interest!
Pro tips for building with LLMs
🥷Pro tips for building with LLMs:
— Hugo Bowne-Anderson (@hugobowne) June 24, 2024
🧠 @sh_reya : Challenge the RAG vs fine-tuning discourse
📊 @BEBischof: Optimize prompts for relevance, boosts performance
🚫 @charles_irl: 'Just because you can cache a million context tokens doesn't mean you should'
😱 @BEBischof: Reality… pic.twitter.com/AGgWrr7hKn
No structured approach? You’re not really doing AI.
I like the way @HamelHusain puts it https://t.co/41zi5El3JS
— Simon Willison (@simonw) June 23, 2024
The AI Engineer Data Literacy Divide
In most cases i’ve encountered in the wild, the title “AI Engineer” is harmful. I explain why in the below video https://t.co/cJT89x2fO6
— Hamel Husain (@HamelHusain) June 21, 2024
Build durable systems, not just chasing models
🚀 @eugeneyan's top 3 AI lessons (asked by @shreya):
— Hugo Bowne-Anderson (@hugobowne) June 21, 2024
1️⃣ Continuous evals throughout dev
2️⃣ BM25 for retrieval - a hidden gem 💎
3️⃣ Build durable systems, not just chasing models
Game-changing advice from their report 🔥 #AIEngineering (this hashtag is for @HamelHusain)… pic.twitter.com/hPpVCgKCDG
What We Covered
🎙️High level points we covered today in "Lessons from a Year of Building with LLMs" with @eugeneyan @BEBischof @sh_reya @HamelHusain @charles_irl @jxnlco
— Hugo Bowne-Anderson (@hugobowne) June 21, 2024
Watch here: https://t.co/G3KaYjkB0R@vanishingdata podcast out next week 💫 pic.twitter.com/WjgAPIzxIg
Check it out and let us know what you get out of it on Twitter (@hugobowne and @vanishingdata) or LinkedIn. You can also register for future livestreams onour lu.ma calendar and subscribe to our YouTube channel.