Fork me on GitHub
Devops Enterprise Summit Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2022-05-10

Channels

#ask-the-speaker-track-4
<
2022-05-10
>
Slackbot11:05:26

Reminder: The breakout sessions are starting in 5 minutes. Get in front of your browser and start navigating your way to whichever session you’re attending. https://devopsenterprise.slack.com/files/UATE4LJ94/F01D34MC2KS/image.png

Molly Coyne (Sponsorship Director / ITREV)11:05:16

✨Welcome Jim Armstrong and @jamie.joshua and @christina.hobday for our next session's Q&A Thank you to our sponsor, Snyk!✨

🙌 2
1
Giulio Vian, Unum11:05:33

quoting the conference host... priceless

1
Giulio Vian, Unum11:05:58

Cloud CI/CD offering suffered from people using build resources for crypto-mining, true stories!

Jamie Joshua12:05:14

Further information on what Jim was touching upon regarding infrastructure drift and unmanaged resources: https://snyk.io/blog/detect-infrastructure-drift-unmanaged-resources-snyk-iac/

Virginia Laurenzano NSA12:05:37

thanks for that write-up!

💯 1
Molly Coyne (Sponsorship Director / ITREV)12:05:52

🐝A warm welcome to @hlynch @ldonley and Drew Piland for our next session's Q&A .Thank you CloudBees! 🐝

🙌 2
🐝 3
1
Hope12:05:12

Thank you @mollyc!

❤️ 1
Eliza Kruszelnicka12:05:49

and often Value stream mapping is seen as process mapping 😞

👆 1
🎯 1
Philip Day12:05:07

value stream = "concept to cash" - wonderfully concise

Rob Wijgers, Agile Coach at TOPdesk12:05:23

Anyone else hear people asking if they can be heard. Looks like people trying to connect not being part of the presentation

➕ 6
Rob Wijgers, Agile Coach at TOPdesk12:05:44

Andy and Martin (names called out)

Jude Wellington12:05:26

@mollyc are you hearing someone in the background?

Tino Dietel, Tradebyte, Engineering Lead12:05:41

yes - there is other audio mixed in

Rob Wijgers, Agile Coach at TOPdesk12:05:06

lots of c=licking and people asking if they can be heard (names called out: Andy and Martin)

Molly Coyne (Sponsorship Director / ITREV)12:05:51

Thanks so much! I have a feeling our VendorDome folks' microphones are hot. I'll let them know!

Anna Velasco12:05:24

Audio is all clean for me... perhaps try refreshing the page? :thinking_face:

Martin Woodward12:05:37

Andy and I will stay muted just in case - sorry about that

Gene Kim, ITREV, Program Chair12:05:43

(It sounds great to me! Loud and clear!)

Gene Kim, ITREV, Program Chair12:05:07

And hello, @martinwoodward!! So good to see you again! 🙂

Martin Woodward12:05:31

👋 - at least my language wasn’t fruity…

😆 2
Gene Kim, ITREV, Program Chair12:05:37

(and suddenly the reports of mayhem makes sense — just look for @martinwoodward. 😆

Rob Wijgers, Agile Coach at TOPdesk12:05:58

not hearing anything additional anymore, so looks like it is cleared up 👍

Molly Coyne (Sponsorship Director / ITREV)12:05:02

Thanks so much for everyone's understanding! And to @hlynch and @ldonley for the great session!

Jude Wellington12:05:31

Thank you, @mollyc

andy henderson12:05:15

Mic's gone hot again, or audio overlapping?

➕ 1
Rob Wijgers, Agile Coach at TOPdesk12:05:43

yes, can't hear the session anymore

➕ 1
Danilo12:05:06

impossible to listen with this audio ...

andy henderson12:05:13

It's like 2 audio tracks overlayed, with a couple of seconds delay between them

Gene Kim, ITREV, Program Chair12:05:44

Fixed here. And it’s back. And fixed. (Thanks for your patience, everyone!)

👍 1
Ferrix Hovi - Principal Engineering Avocado - SOK (S Group)12:05:28

somebody's mouse noises are audible on top of the audio 🙂

Ferrix Hovi - Principal Engineering Avocado - SOK (S Group)12:05:39

Well... we are a species who hasn't fixed the mute/unmute problems on teleconferencing in 20 years with more than 2 years of intensely trying... So, multi-channel audio routing is an understandable hickup.

😂 2
❤️ 1
Luke Rettig - Target, Sr Director-Global Inventory Mangement12:05:47

a true sign of success is when the dev teams can articulate the why off the cuff, without pulling up any documentation

✅ 2
Jude Wellington12:05:14

This is a really good talk so I hope everyone has a chance to listen to the on-demand recording.

Ferrix Hovi - Principal Engineering Avocado - SOK (S Group)12:05:39

Yes, I have liked the audible part very much 😄

Molly Coyne (Sponsorship Director / ITREV)12:05:05

Yes, don't forget to re-watch the recording in the video library afterwards and feel free to reach out to @hlynch @ldonley and @jwellington809 with any questions!

🐝 1
1
➕ 1
Ferrix Hovi - Principal Engineering Avocado - SOK (S Group)12:05:31

It sounds as someone is intensely clicking away with their mic on... so the problem is there but it is not actively bothering right now.

Jude Wellington12:05:39

Please stop by our https://devopsenterprise.slack.com/archives/C01TK0K6QU9 or meet up with us in Gather. We'd love to answer any questions!

🎉 2
Jessica Cregg12:05:54

We’re about to kick off in the 🔷 Vendordome!🔷 Post your questions for @martinwoodward and @abold in the thread below ⬇️

🎉 1
🙌 2
👋 1
Martin Woodward12:05:19

It’s a long way from home, but welcome to the Vendordome. Looking forward to chatting

Molly Coyne (Sponsorship Director / ITREV)12:05:28

Welcome @jcregg who will be moderating for today's VendorDome Q&A between @abold and @martinwoodward!

Molly Coyne (Sponsorship Director / ITREV)12:05:49

🌟Thank you LaunchDarkly & GitHub! 🌟

Martin Woodward12:05:13

Speed and recovery - very appropriate!!

Martin Woodward12:05:30

REBASE!!

😂 2
😱 1
Gene Kim, ITREV, Program Chair12:05:36

Hello, @martinwoodward @abold and @jcregg!!! Love this discussion — “11MM commits. 22 GB. 20m to clone.”

Gene Kim, ITREV, Program Chair12:05:21

(Reminds me of the epic stories of moving Windows OS to git — )

Jessica Cregg12:05:25

Right?! Lightening fast!

🎉 1
Gene Kim, ITREV, Program Chair12:05:46

“2-5 weeks for dev to be productive” — all too common. My fave quote: “figuring out how to have 2 versions of Python on your laptop shouldn’t have to be a core competency.” 😆

1
Gene Kim, ITREV, Program Chair12:05:41

There are legendary stories of developers at banks quitting before they complete the six weeks of compliance training. 🙂

😱 1
Phil Gadzinski Bupa13:05:34

That's why we give you 6 weeks, just in case….😂

Fokko V.13:05:59

@martinwoodward What made you so confident, when sending out your first PR?

✅ 1
Fokko V.13:05:02

Yes, so automated testing is one of the most important safety guards

Gene Kim, ITREV, Program Chair13:05:49

@jcregg I’d love to know from Andy and Martin: they have this awesome dev productivity — how many engineers do they dedicate for dev productivity, to create this amazing infrastructure?

✅ 1
Steve Pereira - Co-Author of Flow Engineering13:05:46

This is such a great question! I suspect most people would surprised how high that % is in elite performers

Gene Kim, ITREV, Program Chair13:05:08

maybe as a % of total devs?

Erik Greathouse13:05:18

As someone who works with governments, I would love to only have a 2-5 week onboarding time. One organization took 6 months to onboard before getting to the development enviroment.

😭 2
😂 1
Gene Kim, ITREV, Program Chair13:05:40

“telemetry starts nosediving” — eep

Toli13:05:56

“Do the thing that hurts, until it doesn’t hurt anymore and move on to the next thing that hurts” 👏

➕ 4
👏 2
Philip Day13:05:03

Great line... but is there a category of issues which people don't yell about because they have normalised that experience? Or aren't aware life could be better? Perhaps in an org with a particularly esoteric environment. How would you identify and tackle these?

👏 2
✅ 1
Philip Day13:05:46

Thanks for tackling this guys! 🙂 Fascinating stuff - @jcregg you mentioned someone in your org wrote a diary of an on-call engineer if I heard correctly - is that something shareable? One idea I've seen is to get new joiners to complain loudly before they become institutionalised. Especially experienced hires who might have seen relevant best practices elsewhere.

❤️ 3
Jessica Cregg14:05:03

Hey @philipday! Here’s the diary mentioned. https://thenewstack.io/diary-of-a-first-time-on-call-engineer/ It’s got some direct excerpts from chats between the engineer and her manager

Philip Day14:05:48

Thanks so much @jcregg!

Toli13:05:40

@martinwoodward how did you have confidence it worked in production? Did you have access to telemetry in production?

✅ 1
Gene Kim, ITREV, Program Chair13:05:51

(@martinwoodward expresses the joy of YOLO! 😆 )

🚀 1
Gene Kim, ITREV, Program Chair13:05:19

Martin: depends, goes in cycles; lean team: pull from feature teams to improve core product, as opposed to a dedicated dev engineering team (so interesting!) GitHub Computer Club: band of people focused on solving the big problems, getting people moved to CodeSpaces.

Gene Kim, ITREV, Program Chair13:05:58

GCC working agreement: you’d notify them when you bail out, and go back to “crummy macos dev” (my words, not theirs. 🙂

🙌 1
Eliza Kruszelnicka13:05:58

@martinwoodward do you set some standards e.g. spend 20% of time improving things? how to you protect capacity for continues improvement?

✅ 1
Gene Kim, ITREV, Program Chair13:05:39

My fave article on CodeSpaces: https://github.blog/2021-08-11-githubs-engineering-team-moved-codespaces/#:~:text=Over%20the%20past%20months%2C%20we,development%20environment%20in%20the%20cloud. My fave line: > This single log message will cause any GitHub engineer to break out in a cold sweat > (something about “dotcom gem environmetn is out of date” 🙂

📝 1
Gene Kim, ITREV, Program Chair13:05:04

Andy: at times, surged to good portion of 60 devs

David Hawes-Johnson (DevOps Enablement - BT)13:05:42

How much of your platforms are built through contributions (a la innerSource), based on how much is request based (e.g. build me this please). Does this help keep teams smaller (EDIT: and therefore the platform is more productive)

✅ 1
👀 1
Gene Kim, ITREV, Program Chair13:05:14

Andy describing how LaunchDarkly uses http://Honeycomb.io to generate production telemetry.

📝 1
Toli13:05:45

@abold do you guys use both Datadog and Honeycomb?

Andy Bold13:05:09

Yes. We’ve used Honeycomb for a long time, to the point where I think it predates Datadog adding trace support. They’re both great tools that help us understand what’s going on.

Toli13:05:22

@martinwoodward, @abold do you measure the Accelerate Metrics? If so, do you measure quantitatively or qualitatively?

👀 1
Gene Kim, ITREV, Program Chair13:05:43

Martin on NPS as satisfaction of your own devs using your tools — nice. 💯

👍 1
Jessica Cregg13:05:04

Loving these questions!

Richard Vodden13:05:31

How do you approach dealing with hero culture?

👏 1
✅ 1
Gene Kim, ITREV, Program Chair13:05:00

Fantastic panel, @jcregg — loving the insight from two great engineering orgs. Such a great example of prioritizing what you value, and it truly shows!

Gene Kim, ITREV, Program Chair13:05:27

@jcregg Is Martin saying “BURGS?” Like icebergs? 😆

😂 3
🧊 1
1
Gene Kim, ITREV, Program Chair13:05:40

@abold: “quarterly, we do a week fixing on issues; ‘focus week’ focusing on toil, tech debt; ”

🎉 1
Lloyd P13:05:52

An idea I've heard about in other organisations is 'innovation days' once a month to tackle all those little ideas that probably won't take too long to implement but have potential to really reduce toil. Stuff like "oh a slack workflow could help us solve this problem, but i'll never get time away from project work to implement it"

👍 1
Gene Kim, ITREV, Program Chair13:05:55

Martin: from MSFT, 90% of issues come from “people paid to work on that product”; 9% come from “close on org chart” (adjacent); 1% from randos adding code (“I’m going to try to add newline support in Windows” 😆

1
😂 1
Gene Kim, ITREV, Program Chair13:05:31

Martin: “further away in org chart, more likely you think they’re an idiot” — trust is lower. and issues look easy to fix for the person not immersed in the issue

✅ 1
Gene Kim, ITREV, Program Chair13:05:03

Martin: people drop by to do something, try adding something to an API interface, say. Easy case: pull data and just add to response; In contrast, at GitHub, all innersourced: “great you come with a gift of a [giant] PR, but can we actually maintain; same ratio at Microsoft: 90/9/1” (@martinwoodward did I get that stat right?)

👍 1
Gene Kim, ITREV, Program Chair13:05:50

Andy: dependency: % of completion drops as exponent of # of dependencies; 50% -> 25% -> 12% (as # of deps go from 1, 2, 3…)

Gene Kim, ITREV, Program Chair13:05:27

Andy: “here at LaunchDarkly: people who know the most share the most”

📝 2
❤️ 3
Richard Vodden13:05:35

> “If you ask the same questions, you’ll get the same answers; for a successful transformation you have to have the discipline to only ask questions which actually matter” gold dust 🙂

📝 1
👏 1
Erik Greathouse13:05:18

Interesting about dependency %. Is this because they are software-focused companies? This seems counter to an open-source philosophy?

Chris Leeworthy (he/him)13:05:21

the law of constraints says as an organisation you’re only as fast as your slowest bit, so individual team performance is only useful if you see how it fits in the whole org

Toli13:05:20

I guess that’s why we strive for autonomous cross-functional teams? To make them as autonomous as possible? It’s a local optimisation but it hopefully contributes to global optimisation?

👍 2
Chris Leeworthy (he/him)13:05:33

I think that certainly helps. but I’m not sure how widespread that practice is yet, and you can still have concerns if you’re working on a legacy monolith where individual teams can still affect each other or be dependant on each other.

💯 1
Philip Day13:05:23

That stuff about incentives from @martinwoodward was gold dust! One of the hardest and most central problems IMHO.

Philip Day13:05:57

I think Martin said, to max out your bonus you have to demonstrate: • What you did • What others did where you helped them • What you did that built on things others did ... which promotes a culture of spreading credit and collaboration I'm not just (or particularly) thinking about developers within teams... I'm thinking anyone in the org, where so much activity sits somewhere in an ecosystem of internal teams and relationships and it's hard to attribute direct, objective and tangible value to genuinely important and difficult work.

Gene Kim, ITREV, Program Chair13:05:57

@abold: “At LD: we have a habit of feature flags that have been true for a long time, get notified”. Clever!

👍 1
Gene Kim, ITREV, Program Chair13:05:50

Automated code changes of conditionals — what could go wrong, @martinwoodward 😆

Gene Kim, ITREV, Program Chair13:05:17

@martinwoodward I’ve got nothing against automated code changes — I’m just against changes in general. As Monk (famous OCD TV detective) said, “change is bad. you should never do it.” 🙂

Gene Kim, ITREV, Program Chair13:05:46

Treating tests as slot machine; if it doesn’t work, just pull the lever again. 🙂

Gene Kim, ITREV, Program Chair13:05:44

Holy cow: Clever!! From Martin: “flag flakey tests, and take it out of rotation, and open defect (I.e., a “berg”) with the team that owns it.”

Gene Kim, ITREV, Program Chair13:05:10

Ha! A heisenbuild / heisenbug.

❤️ 3
Erik Greathouse13:05:43

Chaos Engineering

Martin Woodward13:05:35

One form of it Erik!!

Erik Greathouse13:05:46

@martinwoodward Thank you... Newbie learning quickly

❤️ 1
Gene Kim, ITREV, Program Chair13:05:51

Andy: “be happy with good enough; don’t gold plate; ship it when it’s good enough, and then improve it”

📝 2
Gene Kim, ITREV, Program Chair13:05:02

Martin: “don’t ask permission; it’s your job to make the system better.”

📝 2
Gene Kim, ITREV, Program Chair13:05:49

Martin: “what’s one thing to improve lives of fellow devs, team, customers; what can you do in one day, and do that; repeat, and you’ll be amazed at how improvements build up”

Gene Kim, ITREV, Program Chair13:05:22

Martin: “the funny thing is, you’ll never remember how bad things were, so you need to take good notes, measure, compare.” Nice!)

👏 1
Slackbot13:05:27

Reminder: The breakout sessions are starting again in 5 minutes. Get in front of your browser and start navigating your way to whichever session you’re attending. https://devopsenterprise.slack.com/files/UATE4LJ94/F01D34MC2KS/image.png

Gene Kim, ITREV, Program Chair13:05:34

Andy: “LaunchDarkly is hiring!” 🙂

Gene Kim, ITREV, Program Chair13:05:42

Thank you, @jcregg @martinwoodward @abold!!!

David Hawes-Johnson (DevOps Enablement - BT)13:05:43

Great discussion thanks @martinwoodward @abold and @jcregg

Philip Day13:05:53

Thanks everyone!

Billy Hudson - ScholarPack - DevOps Engineer13:05:11

Thanks guys, lots of gold dust here!

👍 2
Martin Woodward13:05:52

Well I think that was probably the most fun I’ve had in a panel discussion - thanks for the chat and banter folks. Thanks also to @jcregg for keeping and eye on it all for Andy and I.

🎉 2
❤️ 1
👏 3
Andy Bold13:05:17

Thanks for being here everybody! And thank you to @martinwoodward for his words of wisdom and @jcregg for keeping things moving. thankyou

👏 2
Martin Woodward13:05:59

She was too modest to say so herself, but https://twitter.com/jessicacregg is worth a follow too…

💯 2
Molly Coyne (Sponsorship Director / ITREV)13:05:55

🔥Welcome @asharma for our next session's Q&A Thank you Sonatype!🔥

thankyou 1
😄 1
🔥 1
Jessica Cregg13:05:40

Thank you so much to @abold and @martinwoodward for being excellent to work with and brilliant panellists! Thanks everyone for asking questions and keeping the discussion really interesting and interactive - @fokkov @erik.greathouse277 @toli @phillipgadzinski @eliza.kruszelnicka429 @philipday @slack1599 @richard431 @lloyd.passingham @chris.leeworthy @billy.hudson and of course, @genek thankyou

👏 2
❤️ 5
💯 2
Toli13:05:11

Great discussion, some great nuggets of wisdom @abold & @martinwoodward - and very well moderated too @jcregg 👏

☝️ 1
thankyou 2
➕ 1
Gene Kim, ITREV, Program Chair14:05:51

Noblox.js "typosquat" category of issues are astounding...

Gene Kim, ITREV, Program Chair14:05:06

Invisible backdoors! (Makes one questions our life choices in OSS. :)

Virginia Laurenzano NSA14:05:42

struts 2. keeps coming back

Gene Kim, ITREV, Program Chair14:05:29

Makes one question our life choices in OSS. Again. 😂 There's a terrific session on Log4J in software supply chain context tomorrow by Dr. @stephen — expounded upon from session he did in March.

Ann Marie Fred - Red Hat14:05:21

Why do people usually choose to use these “dependency confusion” packages?

Virginia Laurenzano NSA14:05:26

OSS reminds me of a friend's dad. his assessment of most things: doing this will kill you; not doing this will kill you faster.

😂 1
Virginia Laurenzano NSA14:05:34

Ex: eating that will give you cancer (eventually). not eating it will kill you faster (starvation). In our case, using OSS will make you vulnerable. Not using OSS will make your business not viable.

Gene Kim, ITREV, Program Chair14:05:33

@ann.marie.99 There have been afternoons where I used every OSS component that showed up in a "grid react" Google search — I'm not proud...

😲 1
Gene Kim, ITREV, Program Chair14:05:04

Josh Corman should be happy about that...

⬆️ 2
Ann Marie Fred - Red Hat14:05:12

Publishing SBOMs makes me nervous - aren’t you just advertising your vulnerabilities publicly?

Giulio Vian, Unum14:05:35

the choice between downgrade and upgrade requires human intervention bad for automation

💯 1
Giulio Vian, Unum14:05:08

we need to embed more of this knowledge in tools and move beyond CVEs

Gene Kim, ITREV, Program Chair14:05:49

Paul Fox from Morgan Stanley has an amazing experience report on log4j tomorrow, I think — Neat story of them trying to get assurance from their vendors that they've fixed issues, and incoming requests coming into org asking THEM whether they've handled issues.

👀 1
Ax Sharma14:05:05

Thanks for attending the session friends!! We've been catching more malware into today - I've got some more blogs in pipeline if you'd like to check out: https://blog.sonatype.com/author/akshay-ax-sharma

👏 3
Molly Coyne (Sponsorship Director / ITREV)14:05:20

Welcome Dan Lines from LinearB for our next session's Q&A!

Gene Kim, ITREV, Program Chair14:05:25

Thanks @asharma !

👋 1
😄 1
Martin Jainta14:05:58

Hey Dan 👋 How do you define an "idle" PR in your organization? How is it measured?

Martin Jainta14:05:26

Thanks @mollyc 🙏

Paul Trimble14:05:48

Hey @martin.jainta! To us, Pickup time (from the moment a PR has been issued to when it is actually seen/worked) is all idle time. Review time also has several back and forth moments where the PR sits idle. Check this out:

Martin Jainta14:05:17

Ahh I see, thats great, thank you @paul845 🙏

mcico14:05:31

What about doing pair programming (which is probably more strict than idea #4)?

Hagay Netzer14:05:18

Hi @marco.cicolini I would try to help, with pair programming you are taking a risk of the reviews being done by only one person

Hagay Netzer14:05:59

You can always do that but with larger teams it might not be the easiest and safest way to go

Hagay Netzer14:05:51

We are seeing a lot of organizations but pair programming is not that common

Martin Jainta14:05:57

That being said, do you recommend review sessions with the creator and for example two peers?

mcico14:05:27

I have the impression with PRs we tend to optimize for the individual person (e.g. I raise the PR and then continue on the next work item). While with a more synchronous approach (be it idea #4 or pairing) we optimize for flow because we focus on finishing the work items.

Hagay Netzer14:05:43

Well, it is up to the eng org management to decide, there is no one way to go. As long as you get the review done in a timely manner and with enough depth, it should work

👏 2
mcico14:05:01

I will definitively bring up the ideas to the team, thanks 🙂

👏 1
Hagay Netzer14:05:13

In LinearB we are monitoring and reflecting the review pickup time, review time and depth

🙏 2
Slackbot16:05:25

Reminder: The plenary sessions are starting again in 5 minutes. Start making your way back to your browser and join us in #ask-the-speaker-plenary to interact live with the speakers and other attendees. https://devopsenterprise.slack.com/files/UATE4LJ94/F01D34MC2KS/image.png

Slackbot17:05:21

Reminder: Please submit your feedback for the talks you attended. It’s so valuable for us and the speakers. And after all, feedback is a gift and sharing is caring! Enter your feedback for those talks here: https://members.itrevolution.com/live/schedule https://devopsenterprise.slack.com/files/UATE4LJ94/F03E48CJRF1/image.png