This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
The ETLS Connect livestream will begin at 10am EDT, and will broadcast here: https://members.itrevolution.com/live/watch/?stream=on We will activate the page 10 minutes prior to starting, so please reload at that time. Also, you must create or log in using the email you registered with here: http://members.itrevolution.com/login — if you are properly logged in, you will see this page.
This is the channel where all the discussion will occur.
Please update your Slack profile to include more information about yourself. Name, image, organization, title, pronouns… whatever you feel comfortable sharing with this community to help us learn a little more about you. Here’s an example… https://devopsenterprise.slack.com/files/UATE4LJ94/F07HBK48RV3/image.png
Fun fact: if you tag a post with a helpful link, reference, or resource with 🔖, that post will show up in #C04ED43AQAC. Then we all have handy place to go to see all the helpful resources shared by this community!
It's 1000 here in NYC @documentally so I'm afraid we're on breakfast bagels and juice here
Fair enough, a bit early, but I bet you’ll have time to catch up :-)
FYI the slack doesn’t seem to be updating on the tab on the screen. Not sure if it’s just us in Amsterdam that’ it’s affecting
It's filtering results for engagement, and unfortunately can't load posts with images despite engagement.
We are also still stuck in a bad paradigm for Web Development, because we are using the poor abstractions such as HTML. Even though better abstractions exist (and have existed for quite a while (ex: Lively Kernel from Smalltalk Dan Ingals)
I have been loving practicing conversations with chatbots. Setting up the bot to take a particularly hard stance and then practicing my arguments with her. Absolutely amazing value.
could you share some excerpts from your prompts to get it set up for the “particularly hard stance”?
"I want to argue the pros and cons of using ReactNative for my app that spans iOS, Android and Web. You argue the pro side and I'll argue the cons side. You start."
1Have folks tried sharing pictures (ex: web design, probability distribution, …) and seeing what code the AI generates?
> Systems work best when their consumers fall into the “pit of success,” a term coined by software engineer Rico Mariani
The Berlin watch party is filling up and we have confetti....
"Used the interface, how it rolled off the factory line", I know that one :rolling_on_the_floor_laughing:
Thank you so much, @idan!! ---------- 🔥 Next up is @steve.yegge, Head of Engineering, Sourcegraph. At ETLS Las Vegas, Steve shared his thoughts on GenAI, coding assistants, and the "death of the junior developer." He explored how coding assistants are transforming the work of senior developers, making them more like master chefs overseeing a team of robots doing the heavy lifting.
Small talk leads to big talk! The chat prior to the stream here was vibrant, can’t wait to capture some thoughts and feedback after the stream. If you’re at the Amsterdam streaming party please come find me.
3Really looking forward to this one! Death of the Junior Developer. :thinking_face:
Whew thank you all! https://githubnext.com is where you can check out the various prototypes from GitHub Next and https://gh.io/next-discord is the discord if you want to engage with our prototypes
AI can help “junior programmers” IF they learn how to use it. Which begs the questions: - What do they need to learn - How do we help them learn
My two cents: - Ask them questions like @michael.s.winslow asked on have an argument with me (ie: challenge my mental models) - Use it for understanding not for getting homework done - Explain this code to me line by line - How else could I do this? (“You don’t really know something until you know it more than one way”) - Use it to learn how to explain the problem better (analogous to solve a problem for yourself, by posting a question on StackOverflow)
Y'all should check out https://githubuniverse.com next week for some exciting work in this space… 🤐
2Agree. As junior developers are learning from scratch AI might be a huge boost for them to learn working differently and be more productive in a long term
Is Tech Leadership becoming a "Tour of Duty"??? Man the allure is strong to go back and do such cool things! Our good friend Jon Moore made the move back to IC! None of us are immune! 😄
lol! @genek thank you for asking about Steve about the transition from leadership to IC. I've been feeling a similar pull, there's so much cool stuff going on these days, it's hard not to spend more time hands on.
Totally. I've never done as much coding as now, and I'm building so many cool tools I'm using almost every day!!!! I'll post some examples!!!!
If this is a time when everything is changing, I wonder if technologists need to do tour of duty as ICs to get actual hands on experience.
i find it so interesting that it took an experienced engineer with a beginner’s mind (by necessity… because he’d been out of it for a while) to see the issues that might lead to something like the “death of the junior developer”.
“Imagine if I was so good that you could get an instant Answer to any question you wanted to ask. Or instant output for ny request you made. What would be worth teaching in a world like that?” - Cassi Kozrkov Strawberry Paradox - When Perfect Answers Aren’t Enough”.
Article https://towardsdatascience.com/strawberrys-paradox-when-perfect-answers-aren-t-enough-a4e3414eadc0
@kboth_does agreed. > Questions are taken for granted rather than given a starring role in the human drama. Yet all my teaching and consulting experience has taught me that what builds a relationship, what solves problems, what moves things forward is asking the right questions. > - Edgar Schien “Humble Inquiry”
One question I stole from @genek and have been asking a lot is: > How much would you pay for independence of action?
"Companies that are preventing their employees from leveraging AI are putting themselves at serious risk." - Steve Yegge
On helping young programmers (6-60) learn to use the tools … Too often, students are taught to navigate fixed problems with clear solutions. Given Excercises with clear boundaries do the right answer is a known destination We need to help them do the equivalent of Learning Math (ie: how to create your own maths) NOT how to calculate • Ex: Had kids come up with their own terms and definitions before giving them the commonly known ones ◦ Give them problems and tools to learn with ◦ We learn to write by reading good literature, kids should be reading good code
At Triton, we presented the session that @steve.yegge gave in Vegas a couple of weeks back with different industry trends, introduction and workshop about GitHub Copilot to have our developers understand the why, and the use of it. We got very good feedback and our journey just started. 🙂
Ethan Mollick discusses the concept/problem of dulling the AI knife: https://www.linkedin.com/posts/nickjac_ai-aiineducation-activity-7222819821447438336-3zlj?utm_source=share&utm_medium=member_desktop
1https://www.linkedin.com/in/emollick/: "The more you dull the knife of AI so that it can't cause any possible issues, the less useful it is as a tool." Just let that one sit for a while
I actually monitor dashboards to see how many of my engineers are using our LLMs. Not acceptable for any of my engineers to have not even played with one of our internal products.
What about measuring “How much they are sharing what they are learning about how to use LLMs”
#MakeTheRoomSmarter
Thanks, we are experimenting with bi-weekly “let’s learn how to use AI together” sessions to share ideas and ask questions.
The learning occurs in conversations. But I like the idea of a Quicksights (Quick Insights?) dashboard to share learning more broadly
Young programmers also need good mentors/coaches. Think of how long you have struggled with coding issues on your own only to show it to someone else and they find a type or other problem you have. Unfortunately this does not scale. Perhaps AI mentors/coaches could help
seeding this in advance… https://www.amazon.com/Skill-Code-Ability-Intelligent-Machines/dp/0063337797
What we see now is that junior developers just use various tools to generate code and then they just copy-paste it without actually understanding it. If it works it works.. but this unfortunately opens the door to various security issues, and we will see more and more serious problems in the future.
Thank you so much, @steve.yegge!!! ---------- 📚 Please welcome @matt746, author of The Skill Code. He and Gene will discuss why the death of the junior developer that might be happening in software is happening in so many professions because of what he calls the "novice optional" problem.
Book https://www.harpercollins.com/products/the-skill-code-matt-beane?variant=41108953006114
one of the things i love so much about this community is the “boundary spanning” that happens -- things we learn from other domains that we can apply to the world of technology. so many examples of that over the years. this talk highlights yet another example of that.
also the motto of Isidore Newman High School in New Orleans
This is so amazing meeting someone who is literally the world expert on the “novice optional” problem, which we’re observing happening in our profession as we speak!!!
1Previously, the surgeon needed three hands — and so needed a novice! But after the advent of the surgical robot, no novice needed!
Incidentally, here’s how surgical robots are made! https://videos.itrevolution.com/watch/941691966
Dr. Daniel Rock at Wharton! He spoke to this community 1.5 years ago — it’s time to bring him back. His work will be brought again by Dr. Joe Davis, Chief Economist at Vanguard later today.
AI can augment our intelligence or it can make us more ignorant. Choose wisely my friends
PS: Dr. Rock’s paper that Dr. Beane just mentioned is often called the “OpenAI Jobs Report” by people (e.g., YouTubers)
So what will happen when the “experts” retire? We will maybe see a 360 shift in the way we teach and onboard juniors today? #BusinessIdea
Either you share your skills and learning and coach others or you hoard and do it yourself.
I love the discussion about the more we outsource to ai the more ignorant we will become in those tasks. Interested to hear about accountability when AI inevitably makes mistakes.
I wonder if there are any parallels to be drawn from other ‘abstraction layers’ we’ve built. IE - we don’t write code using assembly language anymore, right, so we are ‘ok’ with trusting some stack of other code to handle that. but to your point - the scope of what we are ‘outsourcing’ seems so much larger and less specific here.
What we do and how we do it will change. I think it’s going to be less about coding and more about understanding the problem spaces and jobs to be done. What am I missing here? (I like to be corrected :D)
@stevesargon Excellent comment. I believe we must become better in problem solving. I like your comment "jobs to be done".
1How can we make it safe, quickly detectable and easily reversible to deliver code that is “bad”?
LimitTheBlastRadius
I always wondered about that saying. Putting toothpaste back in the tube does not feel like the most impossible task in the world! 😄
this connection theme reminds me of the workshop from @jmrichardson1 and @e.klauser.bassett a couple years back. building connection takes work and intention. https://videos.itrevolution.com/watch/687309828
To the comment on building bonds (aka relationships) we can learn from Edgar Shein
> Why is it so important to learn to ask better questions that help to build positive relationships? Because in an increasingly complex, interdependent, and culturally diverse world, we cannot hope to understand and work with people from different occupational, professional, and national cultures if we do not know how to ask questions and build relationships that are based on mutual respect and the recognition that others know things that we may need to know in order to get a job done.”
We need to all learn to ask better questions FavorQuestionsOverAnswers
> Questions are taken for granted rather than given a starring role in the human drama. Yet all my teaching and consulting experience has taught me that what builds a relationship, what solves problems, what moves things forward is asking the right questions.”
I have seen another reason why people keep it to themselves! Because the powers that be in the organization do not feed the idea. They do not amplify the idea. So the individual says, "well, I guess I'll stop trying to convince everyone how great this is. I'll just use it myself."
if llm use is massive for knowledge work because individuals feel (and are) more productive… but organizations aren’t seeing those productivity gains… then maybe something else needs to happen. maybe a little “wiring the winning organization” here and addressing social circuity?
Thank you so much, @matt746!! To learn more and reach out, here's the link: https://www.mattbeane.com/ ---------- 💥 And now, say hello to @webpaige, GenAI Developer Experience Lead at Google. In August at ETLS Las Vegas, Paige shared her journey, which began at Google DeepMind and led to the creation of Gemini. She also spoke about how Gemini is replacing custom AI models that power Google properties like Search, YouTube, Sheets, and Docs.
Thanks, Ann, and again, thanks @genek! If you want a SkillBench demo, please reach out to <mailto:matt@skillbench.com|matt@skillbench.com>
Good book on how to do Failure right. https://www.hbs.edu/faculty/Pages/item.aspx?num=64186
"Keeping the human connections alive!", that's what we need, everyone with their own AI is the ultimate nightmare
Agreed, We are experimenting with bi-weekly “Let’s learn how to use AI together” meetings.
#MakeTheRoomSmarter
"For code migrations you can really stand to wait a minute!", I'm going to tell my bosses right now :rolling_on_the_floor_laughing:
"For code migrations you can really stand to wait a minute!", I'm going to tell my bosses right now :rolling_on_the_floor_laughing:
please share back the response :rolling_on_the_floor_laughing:
Will there be any discussion on the costs and ROI of using AI? Where would rules based engines and/or simpler models be just fine. Or using simply using Regression
For me this is the human feedback aspect, what is really helping us get our jobs done better.
Agreed, the focus should be on how is this “Augmenting Our Intelligence” Doug Engelbart and as a “bicycle for the mind” My question is more around the question of how do we decide which tools to use as a lot of money is spent on AI w/o good ROI
I agree, but that happened before AI, a lot of money got spent on tools that didn't help people do very much, so for me it's to have a way of measuring what work's and that's when colleagues say "this is really helping me"
That's literally the purpose of the platform/metric/data we're working on at SkillBench (and in my academic research). Key is from our POV ROI is a joint optimization of short-term productivity across roles/workflows (NOT just individuals) and the effect of new workflows on the capability of the people performing them (i.e., upskilling/deskilling). Current ROI metrics, if left untouched, will center ONLY on productivity.
Thanks @matt746, sent you an email for a demo. Looking forward to continuing the conversation
Good point @matt746, I should have said "this is really helping us", and I do admit having metrics that clearly back this up and a proper framework for obtaining them is important.
It sounds like clustering is essentially pattern matching which is what LLMs are good at.
Read the article I posted earlier https://towardsdatascience.com/strawberrys-paradox-when-perfect-answers-aren-t-enough-a4e3414eadc0
Yes: Share Use Cases (ones that work and ones that don’t)
SharingIsCaring
Woooo!!! First segment of amazing sessions are done! 🎉🎉🎉🎉 How was it? What were the high points? TY!!!!
I'm glad to stay connected this way. I like the quarterly energy, learning, sharing... silliness and irreverence and brilliance of this crew. 🙂
Hearing Steve Yegge reminding us that the AI we want is still 2+ years out. "I want to talk to my computer like Star Trek :spock-hand:"
We’re heading to a break now. Recharge, refuel, and get ready for an amazing second half of ETLS Connect. https://devopsenterprise.slack.com/files/UATE4LJ94/F07HT1XHTPT/image.png
Links I meant to post while speakers were talking: For @idan : https://www.linkedin.com/pulse/amazing-excerpts-from-idan-gazit-github-cambrian-explosion-gene-kim-1b6oc/?trackingId=AOGLhq1PRAOqT7eLda0Ulw%3D%3D This post was generated by the program I wrote — the video excerpts were generated in the 2 hour pairing session with @steve.yegge. And the prose was assisted by a “fact extractor” I wrote to help non-fiction writing. I’m absolutely loving it. (And I was amazed by how GitHub Copilot Workspace could help me prototype using the Slate.js React framework to see if I could write a quick outliner tool!)
For @steve.yegge’s talk: Here’s the first tweet stream I wrote on Dr. Erik Meijer’s amazing lecture, that I referenced a couple of times: https://twitter.com/RealGeneKim/status/1833291950726033453
The best part? Having Dr. Meier respond to me about how much he liked it!!! https://twitter.com/headinthebox/status/1833304124127121883
Here’s a summary of Yegge’s August ETLS talk (written using the same tools) https://twitter.com/RealGeneKim/status/1846643444221202839
One of my aha moments while pairing with @steve.yegge — two classes of problems it can solve: one-turn owl problems, multi-turn owl problems
My reflections on CHOP: https://twitter.com/RealGeneKim/status/1833298959890321503
Here’s the summary of @webpaige’s talk I wrote (using both tools I mentioned, which I’ve had so much fun writing) — a high point was having the famous Dr Jeff Dean retweet it. 🙂 https://twitter.com/RealGeneKim/status/1846260251894731230
The talks are starting again in 5 minutes. Start navigating your way back into your browser. https://members.itrevolution.com/live/watch/?stream=on https://devopsenterprise.slack.com/files/UATE4LJ94/F07H7N10D62/image.png
Speaking of coding assistants, I think “SlackMind” is something that @jeff.gallimore wrote in the last couple of weeks. Amiright?
that is correct. with my own creativity, very dated coding experience, and github copilot.
a few of our speakers mentioned “terrible code that works” — i resemble that remark :rolling_on_the_floor_laughing:
🌟 Welcome back! Please say hello to Dr. Joe Davis, Global Chief Economist and head of the Investment Strategy Group at Vanguard. Dr. Davis believes AI has the potential to greatly enhance global productivity and tackle long-standing structural economic challenges. In his discussion with Gene, he’ll explore why economists have much to say about AI, the historical role of technology in driving productivity, and offer valuable advice for technology leaders assessing AI’s impact.
Yes https://www.dougengelbart.org/pubs/augment-3906.html that is the challenge and opportunity.
Related to the impact of automation: “Jobs change more than they disappear.”
The challenge is the individual Economic impacts. Most people worry about: How will this impact the ability to provide for my family.
I went in a Waymo in SF, after ETLS Las Vegas this year. It was AMAZING!!!! Driverless car!!! If you are an Uber driver, seek another career now.
“AI will more transformative than the personal computer and the Internet.” 😮
We need to prepare our kids (and ourselves) to learn how to learn. For example learn how to create Maths for themselves, not learn how to calculate.
I feel like the US isn't as good at exploratory learning as some other countries...there are some pockets of great things going on of course. But would love to see us make a big leap forward. Sheesh. I would love to make a big leap forward, too. 🙂
Bob Davis (MIT Math Professor who created one of the main New Math movements in the late 50’s/early 60’s) did some great work in this way. He also ran a bunch of workshops for teachers.
Sadly his work was Cargo Culted rather than understood. I had him as a professor and his comment when asked why did it fail was: “It doesn’t scale. You need subject matter expertise in order to facilitate learning. Even if you took all the Mathematics graduates and made them teachers there would not be enough teachers”
Perhaps AI as a good tutor teaching principles, where/how to use and not use them
Any links to good research artilcles on “What jobs will AI (and tech) take over?”
also here’s an excerpt from @matt746’s book: > Let’s take two comparable, recent research papers to flesh this out. In mid-2023, Rob Seamans, an economist at New York University, and colleagues published an analysis of the potential impact of AI like ChatGPT on all known jobs, as registered in a US-government-curated database on work called O*NET.5 O*NET covers the work activities for 1,016 occupations, breaking these down into 19,265 tasks, like “immunize a patient” or “operate welding equipment.” At the same time, Daniel Rock, an economist at UPenn’s Wharton School, and colleagues from OpenAI published similar analysis.6 Both papers aimed to show how much each and every job was “exposed” to the automation that a general-purpose technology like GPT-4 represents. Exposure here means how many of the fine-grained tasks in a job could get a 50 percent productivity boost if the worker used GPT-style technology. > > Beane, Matt. The Skill Code: How to Save Human Ability in an Age of Intelligent Machines (p. 94). HarperCollins. Kindle Edition.
The https://sourcegraph.com/blog/the-death-of-the-junior-developer? 😈 Too soon?
No “junior developers” won’t die off. What they need to learn needs to change. My daughter who is now a CSB (CS and Business) major at Lehigh has Business professors encouraging them to use AI and CS professors saying don’t use it. 😢
All serious researchers agree on this - job elimination is a red herring. Job change is the thing we all need to be paying attention to. Many roles will be extensively impacted but that doesn’t mean the role goes away. It just means the retraining and learning costs are higher. oh and no big deal we’re all counting on informal vicarious learning on the job (via expert-novice collaboration) to fill that gap right when we’re nuking it.
"How is AI going to change my job?" Every human should be asking themselves this...
Ok, we love to make fun of lawyers. But some of the most amazing coaching/training sessions I've had were with groups of lawyers where we talked about making our work visible, and implementing cross-functional, outcomes-based collaboration and strategy testing, pulling biggest risks forward...
Absolutely, but there is a large amount of cruft that has jammed down the legal system/contracts and the billable hour has become a driving force at most large firms, excited about a bunch of that getting disrupted that's not to say let's get rid of all lawyers
If I was the future police I’d be suspicious of all that paper in Joe’s room. Offline back-up or secret stuff?
Love Gene asking “Tell me more” after being called “Myopic” We all need to learn to LOVE being corrected.
One of the things I love hate about these conferences is the “pile of books on my night stand keeps growing.
Thank you, Dr. Joe Davis!! ----- 🔩 And now, say hello to @jchyip, Senior Manager at Grainger. In August, he outlined his mission at Grainger to drive technology-enabled market share growth through a unified Target Operating Model focused on safety, responsiveness, trustworthiness, and coherence. He will share insights on the new skills required as technology shifts from being seen as "just a cost center" to being organized into products aimed at driving market share growth.
I forget where I got this, may have been Asimov “read three things before you go to bed, Some Non-Fiction, Some Fiction and Some Poetry”. I try and read at least a page of each before falling asleep.
for transformations: “What does it look like on paper and what actually has to happen in practice to get results” you mean they aren’t the same???!!! 🤯 :rolling_on_the_floor_laughing:
Lol. Who expects it to look like what we put on paper? That's a best guess when we know the least. Gotta test your strategy, keep doing, practicing, learning, steering.
I think I missed the context. Sometimes deep domain context is useful, sometimes not...
You can’t be a good coach, by just asking questions. You need domain expertise and deep understanding to know what questions to ask, when of whom.
If you would like Domain Expertise you can’t just take certification like https://www.youtube.com/watch?v=cwbiSCgiZNA
To be fair to @jonathansmart1 he does have some https://www.soonersaferhappier.com/traininghttps://www.soonersaferhappier.com/training
On Finding good coaches, I ask them to teach me something. Idea stolen from https://www.dojoandco.com/teamhttps://www.dojoandco.com/team
Thank you, @jchyip!! ---------- ✨ Next up is @chawthorne, Senior Director of Engineering & Product Operations at Oscar Health. At ETLS Las Vegas in August. Clare spoke about Oscar’s mission to make healthcare more accessible and affordable, how they’ve built a thriving tech organization, and the power of Product Ops in enabling their teams to succeed.
Founded in 2012, Oscar has attracted 1.6 million members with an industry-leading Net Promoter Score (NPS) of 66 — this is an astounding achievement, because as she notes, “most insurers [have NPS scores that] hover right around zero.”
Shout out, Clare is presenting local at the New York City watch party!! Have had great convos here with her and her team before the event!!
@chawthorne’s mom is the famous Dr. Carliss Baldwin, famous for many things, including her groundbreaking book “Design Rules, Vol 1” (1999). Who was also an advisor to Dr. Steve Spear!
Excited to see @chawthorne presenting again. How she examined and iterated on her delivery model was one of my favorite things from ETLS.
i heard this from a process auditor… when it comes to process, we want “just enough and not too much” (JENTM)
the trick is that everyone has different views on what “just enough” and “not too much” are 🤷 🙂
Good observation Jeff 😂 I think the trick is figuring out who/what you’re optimizing for. It’s not going to be a perfect fit for everyone, but the goal is to get the requirements met without making it painful for anyone.
Me meeting Dr. Carliss Baldwin in 2023 — @chawthorne’s mom. 😂
https://devopsenterprise.slack.com/archives/C046D51LPPF/p1724286152983479
More Faster => Value Sooner? Watch out for feature factory.
It’s a good push - when I came up with more, better, faster, it comes with the assumption that Product Managment is doing the job of ensuring we’re building the right things. Tech Ops’ role includes optimizing the delivery of those things.
"Triaging tickets", I need one of those dimly lit control rooms in my life aswell :rolling_on_the_floor_laughing:
“Transition ownership of controls to the people in the best position to do the work” 👏👏👏
This type of internal team, internal transfer of knowledge to the people who are/were doing work, in the best position to do the work. THIS is one of the ways companies are helping amazing humans learn and re-learn more quickly.
"Knowing the lingo", as I always say "it is the way you them" ...just get my coat from the cloakroom now :rolling_on_the_floor_laughing:
Whoa. "Sickest patients have the longest charts." Making the important information in there visible...
Thank you so much, everyone!!! Hope it’s been as fun for you as it has been for me!!! Also, if you want to help shape ETLS 2025, please DM me! Thank you!
5What a great day! I learned so much - thank you Gene, and all of the presenters!