Fork me on GitHub
#discussion-main
<
2023-10-05
>
Use other profile15:10:50

Good morning everyone!

๐Ÿ‘‹ 8
Slackbot15:10:02

Reminder: Get yourself to your seat in Chelsea for the opening remarks. Weโ€™re kicking off the final day of the DevOps Enterprise Summit in 15 minutes at 8:45am PDT! https://devopsenterprise.slack.com/files/UATE4LJ94/F05UG4ZGTLN/timer.png

Nick Eggleston (free radical)15:10:43

Good morning!! Super excited for the last day of DOES... ๐ŸŽ‰

Slackbot15:10:02

Reminder: The final day is starting now โ€“ opening remarks and then plenary talks!

Slackbot15:10:11

Reminder: We want to hear your stories from the Summit. What did you learn? Whom did you meet? What ideas are you taking back with you? What actions are you planning to take? Post in #summit-stories! https://devopsenterprise.slack.com/files/UATE4LJ94/F05UKU0HBTP/stories.png

Slackbot15:10:25

Reminder: Remember all those talks you attended the first two days of the Summit? Please submit your feedback for those! Itโ€™s so valuable for us and the speakers. And after all, feedback is a gift and sharing is caring! Enter your feedback for those talks here: https://doeslasvegas2023.sched.com/ https://devopsenterprise.slack.com/files/UATE4LJ94/F05UG4P024W/feedback.png

Nick Eggleston (free radical)15:10:12

GeneCon!!

๐Ÿ‘ 3
๐Ÿ‘“ 2
Nick Eggleston (free radical)15:10:22

Post your summaries and learnings to #CB0936XFT

Manish Bhasin15:10:15

Perfect compilation and support ๐Ÿ‘ ๐Ÿ‘ ๐Ÿ‘ Thanks Gene

โ˜๏ธ 3
Jordan Stoner15:10:50

Is there a way to apply for the program committee?

๐Ÿ‘€ 4
Nick Eggleston (free radical)15:10:47

How are you feeling about DOES?

Haendel Dorfeuille15:10:03

DevX is definitely something my company has to do!

Ann Perry - IT Revolution15:10:52

๐ŸŽ‰ Kicking us off this morning is: George Kraniotis, Director of Software Engineering and Erin Daugherty, Director of Product from Discover Financial Services.

๐Ÿ‘‹ 2
๐Ÿ‘ 2
Stuart Ainsworth15:10:45

Forget the program committee; I just wanna apply for the music team. ๐Ÿ”ฅ

๐ŸŽต 3
๐Ÿ˜† 1
Gene Kim, ITREV, Program Chair15:10:46

Product areas at Discover Financial Services: Card set up, manage my account, re-engage with my card, transfer balance, strategic partnership, credit actions, portfolio enablement

๐Ÿ”ฅ 2
โ˜๏ธ 1
Andrew Barefield, Liatrio15:10:26

I donโ€™t know how this talk ends, but the Sweet Tango Apple is the new best Apple in the world.

๐ŸŽ 3
โค๏ธ 4
Charlie Betz16:10:13

Another fine product of the University of Minnesota (who also developed the Honeycrisp).

๐Ÿ˜ฒ 1
โ˜๏ธ 1
eric, Liatrio16:10:49

Thatโ€™s what I hearโ€ฆbut until I have more sample size data - Iโ€™ll still be in the honeycrisp camp.

Andrew Barefield, Liatrio16:10:54

Today I learned, @char !

Gene Kim, ITREV, Program Chair15:10:45

What a duo! George: leading 170 technologists; Erin leading 17 product leaders

๐Ÿ”ฅ 4
Gene Kim, ITREV, Program Chair15:10:32

Card Posting and Transactions (post transactions, calculate interest and fees, among dozens of capabilities, spanning cloud and mainframes): 100+ engineers, 5700 batch jobs, 60+ components;

Steve Smith (VP, Equal Experts)16:10:02

Thanks for the shout out @georgekraniotis ๐Ÿ˜˜

Gene Kim, ITREV, Program Chair16:10:09

Last re-architcture of CPT? The year 2000. (23 years ago!!!)

๐Ÿคฏ 2
๐Ÿ”ฅ 2
Paul Gaffney16:10:46

and I wonder if that was just Y2K cleanupโ€ฆ

3
๐Ÿ˜† 3
Alfredo Fuentes16:10:15

Can we have the ITRev Play list ๐ŸŽถ ? ๐Ÿ˜

๐ŸŽต 2
Margueritte Kim (CEO, IT Revolution)16:10:33

I promise to post my Spotify list to this Slack soon!

โค๏ธ 1
Virginia Laurenzano NSA16:10:20

A lot easier to add business logic in a big zoo than clean it up...

โ˜๏ธ 1
Topo Pal16:10:51

Do a ๐Ÿ‘ if you were involved in Y2K conversion.

๐Ÿ‘ 23
๐Ÿ‘Ž 1
Gene Kim, ITREV, Program Chair16:10:38

"We needed to stop treating CPT just like a backend ___". (what word did she use? In my head, I filled in "dumping ground", but I know that's not what she said)

๐Ÿ™ 3
Drew Khoury, Liatrio16:10:04

Event Storming ๐Ÿ’ช

2
โค๏ธ 4
Gene Kim, ITREV, Program Chair16:10:04

"Even our most senior engineer didn't realize how reliant they were to things in CPT"

Tooky16:10:34

90 days of event storming! That gives a real idea of complexity and scale ๐Ÿคฏ

โ˜๏ธ 1
๐Ÿ”ฅ 3
Gene Kim, ITREV, Program Chair16:10:38

What we discovered: CPT was 13 domains and 65 capabilities. (There was no way one team could support all these domains because of cognitive load)

Gene Kim, ITREV, Program Chair16:10:13

90 days of event storming to do the archaeology of 23 years of strata of changes (in the dumping ground)

๐Ÿ˜† 2
eric, Liatrio16:10:08

Empowered Product Teams

Dave Collins - Liatrio (THE Enterprise Modernization Firm)16:10:15

Run ๐Ÿƒ from the monolith!

5
๐Ÿƒ 1
Gene Kim, ITREV, Program Chair16:10:29

First two domains: API Engagement, Delinquencies; the result were two engaged teams, now 8 dedicated and persistent product teams; and acknowledging need for shared platform team

โ˜๏ธ 2
๐Ÿ‘ 1
Ted Copeland16:10:20

When the business can name an engineer who helps solve their problems, THAT is a problem!

๐Ÿ’ฏ 7
eric, Liatrio16:10:51

Letโ€™s call Brent ๐Ÿ™‚

Ted Copeland16:10:13

He's a rockstar

Gene Kim, ITREV, Program Chair16:10:16

Ah. "Treat CPT as a product, not just as a backend technology"

๐Ÿ”ฅ 3
โ˜๏ธ 1
Charlie Betz16:10:27

Team persistence = continuity of knowledge.

โ˜๏ธ 2
Dave Collins - Liatrio (THE Enterprise Modernization Firm)16:10:36

I still have nightmares of people asking me โ€œDo you have a P-code?โ€. Funding product teams for the win! :first_place_medal:

1
Haendel Dorfeuille16:10:49

โ€œDonโ€™t operate like a black box for the customersโ€

๐Ÿ’ฏ 1
Paul Gaffney16:10:02

Kindrel : your mess for less

๐Ÿ˜‚ 2
Ann Bain16:10:33

@erindaugherty - how do you think about โ€œmaking business logic human readable and accessible to allโ€? Iโ€™d love to hear more about this!

โ˜๏ธ 1
โž• 1
Ann Perry - IT Revolution16:10:03

๐ŸŽ‰ Next up: Rosalind Radcliffe, IBM Fellow, CIO DevSecOps CTO at IBM

๐Ÿ‘‹ 1
โค๏ธ 1
๐Ÿ”ฅ 2
Gene Kim, ITREV, Program Chair16:10:44

To state the obvious: I could say those things in my introduction because there's no way that Rosalind could. ๐Ÿ˜‚ But I thought it was super necessary to set the context.

๐Ÿ˜† 4
๐Ÿ˜œ 1
Max Reele16:10:50

THANK YOU for Ctrl Z!

๐Ÿ™ 5
โค๏ธ 1
๐Ÿ’ค 1
Gene Kim, ITREV, Program Chair16:10:15

11 data centers, which now longer own. 1K+ applications managed as pets. Your mess for less, and stays a mess. And now you need bring it all back in!

๐Ÿชƒ 1
๐Ÿ˜† 2
๐Ÿคฏ 2
๐Ÿ˜€ 1
Gene Kim, ITREV, Program Chair16:10:02

"What skills mainframe shortage? I found them!" (She was explicitly not allowed to hire from any IBM clients. But if I understand correctly, some people chose to retire early and join Rosalind at IBM.)

โค๏ธ 4
Jeff Gallimore (CTIO - Excella)16:10:20

It says a lot about the leader when people continue choosing to work with that leader. Go, @rradclif!

๐Ÿ’ฏ 4
Adam Woerz, Toyota, Software Engineer16:10:47

There's a whole other talk here on building great teams that she could tell. I want to hear it too.

โž• 4
๐Ÿ‘ 6
Gene Kim, ITREV, Program Chair16:10:44

Rosalind now known internally as "The Hammer."

๐Ÿ”จ 3
โค๏ธ 1
Gene Kim, ITREV, Program Chair16:10:57

"I have systems that have been written 50-60 years ago. I have systems that were written to support the moon launch that are still running". (!!)

๐Ÿคฏ 2
๐Ÿ”ฅ 3
๐Ÿ˜ฎ 2
๐Ÿš€ 1
๐ŸŒ” 1
Gene Kim, ITREV, Program Chair16:10:22

And we were thinking the 23 year Discover CPT platform was potentially heritage!

๐Ÿ˜‚ 2
๐Ÿคฏ 1
Drew Khoury, Liatrio16:10:01

NOONE TOUCHES PRODUCTION

๐Ÿ’ฏ 1
โ˜๏ธ 1
Gene Kim, ITREV, Program Chair16:10:08

"If they actually need to go into production, it will be read-only"

โ˜๏ธ 1
eric, Liatrio16:10:26

Upping your observably game is also helpful โ˜๏ธ

eric, Liatrio16:10:54

Traces, monitoring, logging, alerting, all the things!

Charlie Betz16:10:44

Rosalind "The Hammer" Radcliffe. Nice.

๐Ÿ”จ 2
Gene Kim, ITREV, Program Chair16:10:02

It's fascinating that all this innovative improvement work is happening only when these thousands of systems are being insourced โ€”ย as opposed to the economics that lead to "your mess for less, which will stay a mess"

2
Nick Shrimankar16:10:14

NO Write access to Prod!!

โ˜๏ธ 2
๐Ÿ’ฏ 1
๐Ÿ‘ 1
Jeff Gallimore (CTIO - Excella)16:10:31

I have never heard someone say 8 9โ€™s with authority and credibility before. It feels weird.

โ˜๏ธ 4
๐Ÿ˜œ 1
๐Ÿ˜ 1
Margueritte Kim (CEO, IT Revolution)16:10:13

Are you questioning The Hammer?!

4
๐Ÿ”จ 1
๐Ÿ˜† 1
Steve McGhee [Google]16:10:46

Iโ€™d love to hear more

Jeff Gallimore (CTIO - Excella)16:10:07

Quite the opposite. I believe herโ€ฆ because she IS โ€œthe Hammerโ€! ๐Ÿ™Œ

Nick Eggleston (free radical)16:10:13

Measured how exactly?

โ˜๏ธ 1
Brian Larson16:10:00

315.58 milliseconds per year of downtime is essentially 0. I agree, measured how?

Rosalind16:10:24

Measured by systems availability and application availability with 0 minutes planned downtime. It helps to start with hardware designed for it but it also takes running a sysplex which allows you to move work dynamically without impact.

Topo Pal16:10:21

Number of production access tells you how mature your DevOps practice is.

โ˜๏ธ 2
๐Ÿคฏ 1
๐Ÿ’ก 1
Adam Woerz, Toyota, Software Engineer16:10:32

She is putting on a masterclass right now on how to humble brag about being the best of the best in leadership.

๐ŸŽ‰ 3
๐Ÿ’ฏ 2
Steve Pereira - Co-Author of Flow Engineering16:10:45

99.999999% (8 nines of availability) monthly downtime would be approximately 0.026 seconds.

๐Ÿคฏ 3
Gene Kim, ITREV, Program Chair16:10:30

Focus on Run vs. Change โ€”ย I have to imagine there can be no ROI on rewriting 1000s of applications that have been running critical business processes for IBM for decades. Interesting technology change.

๐Ÿ™Œ 2
Nick Eggleston (free radical)16:10:28

What did the business case look like?

Jordan Stoner16:10:56

#servantleadership

๐Ÿ’ฏ 5
โ˜๏ธ 5
โค๏ธ 3
Jeffrey Sykes (Platform Engineering, Burns and McDonnell)16:10:33

315.58 milliseconds of downtime per year. Gracious.

Jeff Gallimore (CTIO - Excella)16:10:50

โ€œNew collar talentโ€ love this term.

โค๏ธ 5
Kamran Kazempour16:10:58

This amount of (cultural) change for a company that still supports moon-landing services is inspiring

๐Ÿ’ฏ 2
Martin Deslongchamps - Triton Digital16:10:15

There is so much courage behind this talk @rradclif in leading such a change. It is not only knowing the tech, you got talents to follow you. Bravo! :right-facing_fist::skin-tone-2: :left-facing_fist::skin-tone-2: ๐Ÿ˜Ž

โค๏ธ 6
eric, Liatrio16:10:10

There was a talk at DOES last year about re-skilling and giving folks an opportunity to be welcomed into our open community.

Margueritte Kim (CEO, IT Revolution)16:10:53

Yes! A Lightning Talk by Alex McCleod. She is absolutely amazing.

eric, Liatrio16:10:33

Thank you @mvk842 ๐Ÿ™ - thatโ€™s the one! One of my favorite talks last year!

Margueritte Kim (CEO, IT Revolution)16:10:15

I will pass along the message to her. She will be so touched that her talk was remembered. Her work with ReUP is so important.

โค๏ธ 1
Jim Mercer16:10:28

Also very impressive her career and credentials as a woman and how rare that was when she started her career!!

โค๏ธ 1
Nick Eggleston (free radical)16:10:25

Do you still consider yourself an IC, @rradclif?

Gene Kim, ITREV, Program Chair16:10:03

I'll answer for you โ€”ย the answer is YES!!!

Rosalind16:10:09

Absolutely, I have no people management responsibilities but do have leadership responsibilities

Ann Perry - IT Revolution16:10:13

๐ŸŽ‰ Next up: Christof Leng, SRE Engagements Engineering Lead at Google

๐Ÿ˜Ž 1
๐Ÿ‘‹ 3
Nick Shrimankar16:10:18

Mohawk, Bond music and awesome tech talks! You've got to love tech oriented conferences!!

๐Ÿ’ฏ 3
Gene Kim, ITREV, Program Chair16:10:47

My favorite @cleng quote: "you really don't understand a system until it's on fire... in production... with live customer traffic!"

๐Ÿ’ฏ 2
๐Ÿ”ฅ 6
โค๏ธ 3
Gene Kim, ITREV, Program Chair16:10:47

My favorite @cleng quote: "you really don't understand a system until it's on fire... in production... with live customer traffic!"

๐Ÿ’ฏ 2
๐Ÿ”ฅ 6
โค๏ธ 3
Nick Eggleston (free radical)16:10:21

Nothing like a failure in production to surface unacknowledged dependencies...

Virginia Laurenzano NSA16:10:54

And some generals are demanding an explanation....and time to correct...

Ankit Patel16:10:01

Fastest way to #learnfromincidents

Topo Pal16:10:28

โ€œHope is not a strategyโ€

โ˜๏ธ 1
โž• 2
Steve McGhee [Google]16:10:23

โ€œSpes concilium non estโ€

๐Ÿ‘ 1
Daniel Underwood (BOK Financial)16:10:48

Hope is necessary but not sufficient.

Steve Pereira - Co-Author of Flow Engineering16:10:30

Unreliability CAN be taken for granted.

1
โ˜๏ธ 1
Dinkar16:10:10

Drifting from economy of abundance to economy of scarcity... applies well to the reliability

๐Ÿ‘ 1
Matt Gross16:10:25

Machines are not living beingsโ€ฆ.yet.

Virginia Laurenzano NSA16:10:01

No cows were harmed in the making of this talk

๐Ÿ˜† 3
Gene Kim, ITREV, Program Chair16:10:40

People need to be able to point out the weaknesses and vulnerabilities in our systems.

Nick Eggleston (free radical)16:10:44

How is this incentivized and sustained?

Ross Everett16:10:03

Less that it's lncentivized more that its not disincentivized - the incentive is then by admitting the issue exists that others will help solve it and you will be able to learn from the process

Steve Pereira - Co-Author of Flow Engineering16:10:00

โ€œWhere there is fear, you do not get honest figuresโ€ - Deming

๐Ÿ‘ 5
๐Ÿ‘ 1
Hemant Mahamwal16:10:17

Thatโ€™s one tough cat

๐Ÿ’ฏ 1
Ross Everett16:10:35

"wrong lever!" "Why do we even have that lever?!"

Drew Khoury, Liatrio16:10:40

Metrics have risks.

Nick Eggleston (free radical)16:10:55

Be very careful in selecting what you measure

๐Ÿ‘ 1
Steve Pereira - Co-Author of Flow Engineering16:10:01

How dare someone from Google highlight that something like DORA metrics arenโ€™t the only thing to measure ๐Ÿ˜ณ

nathenharvey16:10:10

Not even team dora would claim that!

dora 4
๐Ÿ”ธ 1
Steve Pereira - Co-Author of Flow Engineering16:10:25

I imagine you folks may spend more time sharing the disclaimer than the metrics these days ๐Ÿ˜…

โ˜๏ธ 1
Steve McGhee [Google]16:10:49

The metrics are just step 1 of INF

Jeff Gallimore (CTIO - Excella)16:10:29

โ€œThe best way to understand a system is to watch it go up in flames with real user traffic.โ€ ๐Ÿ”ฅ ๐Ÿ˜ณ :rolling_on_the_floor_laughing:

๐Ÿ”ฅ 1
Matt Ring (he/him) - Sr. Product/Engineering Coach, John Deere16:10:20

"When we look at history, heroes tend to have a short lifespan."

๐Ÿ‘† 2
๐Ÿ™€ 1
Paul Gaffney16:10:58

In a world that rewards fire fighting, no one wants to go into fire preventionโ€ฆ

โ˜๏ธ 3
๐Ÿ’ก 5
๐Ÿ’ฏ 2
Ivan Krnic (Director of Engineering at CROZ)16:10:06

Heroes have a short lifetime expectancy ๐Ÿ˜‚ ๐Ÿ˜‚

Maria Mentzer (VP See to Solve)16:10:07

If you reward firefighting you get an army of arsonists

๐Ÿ”ฅ 4
โ˜๏ธ 3
๐Ÿ’ฏ 3
1
Gene Kim, ITREV, Program Chair16:10:20

Ops worst nightmare. You're in the middle of the night, working an outage, alone, not knowing what to do.. and you have no one to call.

๐Ÿ’ฏ 1
๐Ÿ˜ฑ 4
Luke Rettig - Target, Sr Director-Global Inventory Mangement16:10:05

your therapist and pizza/beer delivery peeps

2
Ted Copeland16:10:51

Applaud the heroes, then put them out of a job

Steve McGhee [Google]16:10:53

Donโ€™t run buckets of water into the fire faster, invent a fire truck.

โ˜๏ธ 1
Steve Pereira - Co-Author of Flow Engineering16:10:15

โ€œAutomate yourself out of your current set of tasks every 18 monthsโ€

๐Ÿ‘ 2
๐Ÿคฏ 1
Gene Kim, ITREV, Program Chair16:10:52

"You need to aggressively automate. Not for efficiency, but for consistency. Without doing this, you get more pets."

๐Ÿข 1
โค๏ธ 9
๐Ÿฑ 1
๐Ÿฆ– 1
๐Ÿน 1
๐Ÿถ 1
๐Ÿฆ 1
Max Reele16:10:07

It would be nice to expand our community understanding of heroes in the system. I'm hearing it spoken about as always a bad thing but I don't think that's universally true. Sometimes a great outcome of a strong system is that its design inherently offers the space for heroes to emerge.

Stuart Ainsworth16:10:33

I like @genek โ€˜s concept of โ€œsceniusโ€ and โ€œinteresting friendsโ€ instead. โ€œHeroesโ€ usually fly solo, which leads to dependence on an individual to solve problems rather than a community that changes the scene.

๐ŸŽฏ 1
Nick Eggleston (free radical)16:10:16

Heroes are celebrated because they protect and save us from a disaster... but we need to learn from the hero's journey to make the necessary and difficult changes so that the hero can retire

๐Ÿ’ฏ 2
Stuart Ainsworth16:10:41

So if we talk about heroes, it should be more about the Justice League than Superman.

KristiB16:10:44

Agree. Heroes become a problem if thereโ€™s consistency of breaks and saves.

๐Ÿ’ฏ 1
Christof Leng (Google)17:10:26

Being a hero should only happen under very special circumstances and not be a career path. Like production freezes, you sometimes need heroes, but you should never mistake them for the solution to your problem.

๐Ÿ‘ 2
Max Reele17:10:51

Agreed. And I don't mean hero as in the weekend worker... I mean the team member who emerges to solve incredibly vexing problems. Like the enlisted member who fixed the water pump on Adm Richardson's nuclear submarine. The culture on the sub allowed him the space to fix that pump on his own. It encouraged his behavior. we all surge when we have to but must take liberty when we can

โž• 1
Stuart Ainsworth18:10:57

@mreele perhaps the distinction is in the definition of a hero and designing systems that try to minimize the need for a hero. In the situation you described, there was a need for someone to step up, and absolutely agree that the culture provided an outlet for them to do that. But what happens next is of interest to SREโ€™s; how do we build systems that minimize the need for people to do extraordinary work to accomplish ordinary goals? Itโ€™s a balance, and unfortunately, tech has a lousy track record of exploiting extraordinary behaviors instead of systematic thinking about ways to meet that need.

Stuart Ainsworth18:10:38

But yes, we should build a culture that encourages people to solve hard problems and be bold in doing so, but then goes the next step to figure out ways that those bold steps are required less often for system dependencies.

Max Reele19:10:33

Yep, I completely agree. I personally am having dissonance with the concept of heroes in the system as part of the system design and encouraging that exceptional performance yet needing an architectural resiliency that doesn't require heros at all.

โ˜๏ธ 1
Stuart Ainsworth20:10:05

Iโ€™d love to hear more discussion about this distinction as a community. I think thereโ€™s enough ambiguity around the term hero that it warrants it. And this brief exchange showed me that Iโ€™m looking at it from one lens and thereโ€™s definitely more.

Scott Sweeney16:10:40

โ€œPeople think automation is about efficiency. It's really about consistency.โ€

๐Ÿ’ฏ 7
Steve McGhee [Google]16:10:20

Gradual Change > YOLO

Gene Kim, ITREV, Program Chair16:10:43

Was thinking the same thing as Christof described winging it, pushing out config change without version cont rol or review. ๐Ÿ˜‚

Ankit Patel16:10:44

Production freezes donโ€™t solve the underlying problem. They just pause them, temporarily.

โ˜๏ธ 1
๐Ÿ’ฏ 2
Christof Leng (Google)17:10:03

Yes! They're a mitigation, not a fix. Sometimes you need a mitigation, but you should never confuse it with a fix.

Steve Pereira - Co-Author of Flow Engineering16:10:46

:thinking_face: If you add a $ to โ€œDonโ€™t deploy on Fridaysโ€, does Charity still appear in the chat?

2
๐Ÿ˜‚ 3
Jeff Gallimore (CTIO - Excella)16:10:33

We all test in production. Some more safely than others.

๐Ÿ‘ 2
Christof Leng (Google)17:10:27

I should have said: When your code hits production should not be whenit gets tested for the first time.

Syed Ali16:10:50

โ€˜Donโ€™t be a Hero. SRE heroes leave an influence and that creates a culture of over workingโ€™

โ˜๏ธ 2
๐Ÿ‘ 1
Steve McGhee [Google]16:10:08

The bigger your forest, the more lightning strikes. Donโ€™t try to reduce that count, improve your response.

๐Ÿ’ก 3
โ˜๏ธ 3
๐Ÿ˜† 4
1
๐Ÿ‘ 1
Brian Larson16:10:28

Minimize impact. Recovery-based-Architecture

Gene Kim, ITREV, Program Chair16:10:32

NO HAUNTED GRAVEYARDS!!! ๐Ÿ˜‚

๐Ÿ‘ป 2
Ross Everett16:10:56

Everyone has a test environment, some people are lucky enough to have a separate production environment...

๐Ÿ’ก 3
Steve Pereira - Co-Author of Flow Engineering16:10:13

Anyone can build complex systems (even by accident) - try building simple systems

Steve McGhee [Google]16:10:27

Scooby Doo was an SRE

Gene Kim, ITREV, Program Chair16:10:36

Hilarious. Christof always pushes the big red buttons, and deletes code that says "DO NOT TOUCH" to see what happens. ๐Ÿ˜‚

๐Ÿ˜† 7
๐Ÿ”ด 5
๐Ÿ‘ 3
Joy Lopez16:10:35

Glad to see Iโ€™m not the only one ๐Ÿ˜‚

Christof Leng (Google)17:10:38

Signs that my 3yo has strong SRE potential: (1) likes to push every red button (2) likes to break things to see what happens (3) gets overly excited around chaos (4) likes to nap during office hours

Adam Woerz, Toyota, Software Engineer16:10:09

"Don't touch" technical debt is a booby trap for change.

๐Ÿ’ฃ 1
Dinkar16:10:09

Anyone can build a complex system, build a simple one ๐Ÿ‘†

โค๏ธ 2
๐Ÿ’ฏ 2
๐Ÿ‘ 1
Tim Haagenson (American Airlines)16:10:18

Lines of code deleted is my favorite stat in a pull request

๐Ÿ’ฏ 4
Marcelo Ancelmo16:10:41

"We are Engineers!"

Brian Larson16:10:53

Build boring systems

Satish Mogasati16:10:58

โ€œDonโ€™t work hard. Work smart.โ€ Quote of the day!

Ann Perry - IT Revolution16:10:38

๐ŸŽ‰ Next up: Josh Corman, Founder, I am The Cavalry (dot org)

๐Ÿ‘‹ 1
Nick Eggleston (free radical)16:10:35

โ€ข Speak Truth to Power

๐Ÿ‘ 1
Nick Eggleston (free radical)17:10:05

No one is coming to save us

Gene Kim, ITREV, Program Chair17:10:59

The person in the upper-left is the late Dan Kaminsky, famous for his work for "saving the Internet" by coordinating a critical DNS fix. https://en.wikipedia.org/wiki/Dan_Kaminsky HD Moore, inventor of Metaspoit: https://en.wikipedia.org/wiki/H._D._Moore

๐Ÿคฏ 2
๐Ÿ”– 1
Joshua Corman17:10:57

Also. Gene Kim โ€ฆ Alex Huttonโ€ฆ and David Etue.

Caleb Henshaw17:10:18

Killing for good!

Nick Eggleston (free radical)17:10:27

Until people die they won't spend a penny on security

Drew Khoury, Liatrio17:10:28

Cheese :cheese_wedge: > People ๐Ÿง‘โ€๐Ÿคโ€๐Ÿง‘

Jason Cowdy17:10:43

From WI, can confirm: Cheese is life :cheese_wedge:

5
2
๐Ÿ’ฏ 1
Nikhil Das Nomula(Platforms - JD Finishline)17:10:58

My wife is from WI and I double confirm lol

Steve Pereira - Co-Author of Flow Engineering17:10:58

Your capacity is equal to the capacity of your constraint

โ˜๏ธ 3
Gene Kim, ITREV, Program Chair17:10:24

Force multiplier -> Force divider

โค๏ธ 2
๐Ÿ”ฅ 4
John Liehr17:10:46

Medtech hides behind the FDA and compliance, fighting change and progress.

Nick Eggleston (free radical)17:10:14

One might argue that is the real purpose of the agency

eric, Liatrio17:10:59

The work and ๐Ÿง  power of this collective community is being applied to critical life or death situations! โค๏ธ

โ˜๏ธ 4
Dinkar17:10:21

Latency sensitivity analysis.. ๐Ÿ‘†

Steve Pereira - Co-Author of Flow Engineering17:10:26

Latency sensitivity is a great lens for prioritization and sequencing โฑ๏ธ

๐Ÿ‘ 6
Ann Perry - IT Revolution17:10:38

๐ŸŽ‰ Next up: JD Black, Director of Digital Transformation at Northrop Grumman

๐Ÿ‘‹ 3
Bryon Kroger (CEO, Rise8)17:10:24

Helluva call to action. Iโ€™ll reiterate my recommendation of (Re)coding America. The Calvary isnโ€™t coming. Become the Calvary.

๐Ÿด 5
๐Ÿ”– 1
Joshua Corman17:10:23

Horses not crucifixions ;)

Nick Eggleston (free radical)17:10:05

don't forget that it's often the innocent who get crucified

eric, Liatrio17:10:28

Love the passion and call to action I feel after Josh Cormanโ€™s talk! The Calvary isnโ€™t coming!!! Bravo sir!

โ˜๏ธ 2
Joshua Corman17:10:16

Thx. Cavalry. Cavalry ;) such an unfortunate near-name โ€ฆ

Virginia Laurenzano NSA17:10:38

Public policy and regulations aren't sexy. But it's often written in blood.

๐Ÿฉธ 3
๐Ÿ’ฏ 4
โ˜๏ธ 1
Nick Eggleston (free radical)17:10:25

So true. Bad policy leads to many harms, including death, but those responsible rarely are held to account. Incentives...

Gene Kim, ITREV, Program Chair17:10:42

"Boots on regolith". YES! Never heard that!

๐Ÿš€ 2
Gene Kim, ITREV, Program Chair17:10:41

Systems engineering: 50 pounds of software ---> 250 pounds of documentation. Systems engineering enabled massive successes, but they don't know what to make of DevOps practices.

๐Ÿ”ฅ 4
โค๏ธ 1
Drew Khoury, Liatrio17:10:53

"what architecture model?"

Gene Kim, ITREV, Program Chair17:10:43

This is incredible effort brought down Authority to Operation from 9 months to 2 hours. ๐ŸŽ‰

โ˜๏ธ 3
๐Ÿคฏ 1
๐Ÿช„ 1
Hasan Yasar17:10:01

Once more, It is all about breaking silos between all stakeholders including system engineering team, not just Dev&Ops!

โ˜๏ธ 3
Gene Kim, ITREV, Program Chair17:10:06

One success: they detected a late discovery of an interface compatibility; they were able to quickly fix. (I will need to watch this video again.). These are amazing examples of true integration of systems integration into the daily work of the rest of the value stream. Amazing.

๐ŸŽ‰ 3
Gene Kim, ITREV, Program Chair17:10:23

Some of the things that JD is talking about, he wrote on weekends to prove these concepts out.

Paul Gaffney17:10:13

End PowerPoint Engineering in our time!!!

๐Ÿ’ฏ 3
Andrew Davis - AutoRABIT - DevSecOps for Salesforce17:10:15

For anyone who missed the context, JD's team is focused on shooting down incoming nuclear missiles

๐Ÿคฏ 1
Joshua Corman17:10:24

If people want a slower overview of the pandemic related mapping and lessons: https://youtu.be/XrSVXbWGZHw

๐Ÿ”– 3
๐Ÿ™ 1
Joshua Corman17:10:22

If youโ€™d like the decade later (emotional) birthday keynote to the hacker conference where we were born: title: โ€œAnd together we crossed the Riverโ€ 1 hr starts here: https://youtu.be/Eh6b1H_-U20?t=2008

๐Ÿ”– 1
Matt McLarty17:10:05

Hoping everyone can join our talk, โ€œAPIs, Optionality & The Science of Happy Accidentsโ€ up next in Chelsea! It may not sound like it, but itโ€™s a pirate storyโ€ฆ Arrrrr! ๐Ÿดโ€โ˜ ๏ธ

๐Ÿ‘ 2
Slackbot17:10:02

Reminder: The breakout sessions are starting in 5 minutes. Start navigating your way to whichever session youโ€™re attending. https://devopsenterprise.slack.com/files/UATE4LJ94/F05UG4ZGTLN/timer.png

Nick Eggleston (free radical)18:10:27

Pirates need wealthy merchant ships to plunder, and some legal cover to avoid the costs... :rolling_on_the_floor_laughing:

๐Ÿดโ€โ˜ ๏ธ 1
1
๐Ÿ˜„ 1
Nick Eggleston (free radical)18:10:33

@internettitan @internettitan @mattmclartybc would Miners be a better metaphor than Pirates?

Matt McLarty19:10:07

There are so many metaphors! Farmers might even be betterโ€ฆ sow the seeds, see which ones grow

Stephen Fishman19:10:54

We talked about miners/drillers/farmers with the idea of natural resourcesโ€ฆ pirates won for two reasons 1) we liked the idea of the โ€œland of 1000 shovelsโ€ 2) pirates are more fun ๐Ÿดโ€โ˜ ๏ธ

Nick Eggleston (free radical)18:10:08

(The right) constraints encourage creativity

Tooky18:10:45

Interested in optionality, you might like Chris Matts' and Olav Maasen's https://www.infoq.com/articles/real-options-enhance-agility/principle.

๐Ÿ‘€ 1
๐Ÿ”– 1
Maria Mentzer (VP See to Solve)18:10:28

So value dynamics constructs the new pirate map

Jeff Gallimore (CTIO - Excella)18:10:05

โ€œPaving the feedback loopsโ€ ๐Ÿ’ก

๐Ÿ’ฏ 2
Maria Mentzer (VP See to Solve)18:10:41

I'm new to DevOps community can someone give me a quick explanation of pets to cattle? Thx!

Maria Mentzer (VP See to Solve)18:10:41

I'm new to DevOps community can someone give me a quick explanation of pets to cattle? Thx!

Maria Mentzer (VP See to Solve)18:10:44

Thanks! I found a similar post ๐Ÿ˜€๐Ÿ‘

Maria Mentzer (VP See to Solve)18:10:06

Actually, the exact same one ๐Ÿ˜€

๐Ÿ™Œ 1
Paul Gaffney20:10:43

Missing from that post are what I think to be some of the key elements of the metaphor: Pets have names and their owners care about them as individuals They are expensive Cattle are generally interchangeable, not named, and the owners of the herd donโ€™t care very much about individual members of the herd It should be easy to cull problem cattle from the herd It should be easy to expand the herd In other words, you have a totally different approach to managing cattle than to managing pets

โค๏ธ 1
๐ŸŽฏ 2
Matt Ring (he/him) - Sr. Product/Engineering Coach, John Deere18:10:41

That's a quality trophy there @nathen.harvey and @amandalewis!

๐Ÿ† 3
โค๏ธ 1
๐Ÿ’ฏ 1
Maria Mentzer (VP See to Solve)18:10:38

Culture is essential for people to thrive, which drives a thriving company

Jeff Gallimore (CTIO - Excella)18:10:31

Iโ€™m sure @nathen.harvey said that point about the culture first ๐Ÿ˜‰ . But thereโ€™s this person named John Shookโ€ฆ :thinking_face: https://sloanreview.mit.edu/article/how-to-change-a-culture-lessons-from-nummi/

๐Ÿ˜ 1
โค๏ธ 2
๐Ÿ”– 1
Maria Mentzer (VP See to Solve)18:10:09

YES! Such a great proof point of importance of culture!

Matt Ring (he/him) - Sr. Product/Engineering Coach, John Deere19:10:11

Elite teams are back in 2023! Woot! ... wait, @nathen.harvey arguing against eliteness as a goal? Oh c'mon Nathen-with-an-E! ๐Ÿ˜‹

๐Ÿ˜‚ 3
Stephan Guay19:10:35

Can u put the QR code again?

Slackbot20:10:30

Reminder: The breakout sessions are starting again in 5 minutes. Start navigating your way to whichever session youโ€™re attending. https://devopsenterprise.slack.com/files/UATE4LJ94/F05UG4ZGTLN/timer.png

Ted Copeland21:10:39

Any reason the slides for this session are not on the upper screens?

Slackbot22:10:21

Reminder: The final plenary sessions are starting again in 5 minutes. Start making your way back to Chelsea. https://devopsenterprise.slack.com/files/UATE4LJ94/F05UG4ZGTLN/timer.png

Ann Perry - IT Revolution23:10:55

๐ŸŽ‰ Welcome, Damon Edwards, Senior Director, Product at PagerDuty

๐Ÿ”ฅ 3
Drew Khoury, Liatrio23:10:10

Now we're in charge of the computers ๐Ÿ–ฅ๏ธ

๐Ÿ˜† 2
Gene Kim, ITREV, Program Chair23:10:40

And anyone who knows English can be in charge of computers... and data... ๐Ÿ˜‚

๐Ÿ˜† 2
๐Ÿ’ฅ 1
๐Ÿ‘ 1
Gene Kim, ITREV, Program Chair23:10:15

Damon's big surprise: he found more enthusiasm for AI/LLMs in the internal operations people communities vs. external product revenue generating side of business. Why? The opportunities to reduce margins (people, toil, coordination...). Until now, difficult to automate.

๐Ÿ‘€ 1
Hemant Mahamwal23:10:46

What happens when AI isnโ€™t just a tool but a partner?

โž• 2
1
Topo Pal23:10:20

Someday AI will do my job and I can attend conferences

โœจ 3
๐Ÿ˜‚ 12
Gene Kim, ITREV, Program Chair23:10:49

Interestingly, IIRC, Patrick Debois' work was driven and funded by Marketing (at least in short term)

๐ŸŽ‰ 2
Gene Kim, ITREV, Program Chair23:10:36

If you hear "train a model", stop the meeting. Too hard! ๐Ÿ˜†

๐Ÿ˜† 2
Steve Pereira - Co-Author of Flow Engineering23:10:58

Operationalizing vs building seems more relevant than buy vs build

Kamran Kazempour23:10:17

Want change: โ€œdonโ€™t bring the experts in, learnโ€

โค๏ธ 1
Stephan Guay23:10:41

Would need an AI to be quick enough to capture this great material... so quick ๐Ÿคฏ

Stephan Guay23:10:04

Hope the presentations are available post conference?

Stephen Walters23:10:49

โ€œHoney, thereโ€™s a Data Scientist in the bath. Can you get rid of it?โ€

Topo Pal23:10:14

English and emojis are next generation languages.

๐Ÿ˜† 1
Gene Kim, ITREV, Program Chair23:10:21

A big shock: I was blown away when I heard that companies like Notion were using 3-4K token prompts โ€”ย which was 50% of the token limit of the time.

Ann Perry - IT Revolution23:10:39

๐ŸŽ‰ Welcome, Ivan Krniฤ‡, Director of Engineering at CROZ

๐Ÿ‘‹ 5
Gene Kim, ITREV, Program Chair23:10:47

The real differentiator is not technology, but the ability innovate around it. (Agh, missed the quote.)

โ˜๏ธ 3
1
Steve Pereira - Co-Author of Flow Engineering23:10:59

โ€œCapacity to navigateโ€ is an excellent focus for org performance!!!

โค๏ธ 2
๐Ÿ”ฅ 1
Dave Burnison23:10:35

@genek after @joshcormanโ€™s talk this morning I feel like I should give you the shirt off my back.

๐Ÿ”ฅ 2
โค๏ธ 2
Dave Burnison23:10:35

@genek after @joshcormanโ€™s talk this morning I feel like I should give you the shirt off my back.

๐Ÿ”ฅ 2
โค๏ธ 2
Steve Pereira - Co-Author of Flow Engineering23:10:06

After running 1hr workshops for two days, I highly recommend enabling constraints for making exercises productive :)

๐Ÿ‘ 1
Margueritte Kim (CEO, IT Revolution)23:10:25

An AI support agent with a human voice?! ๐Ÿคฏ

๐Ÿ˜ฎ 3
1
๐Ÿคฏ 1
Gene Kim, ITREV, Program Chair23:10:45

Wow. Depts participating! Internal audit! Risk! Compliance! Board member in charge of infosec!

๐Ÿ”ฅ 2
โ— 1
eric, Liatrio23:10:17

HR as well! ๐Ÿคฏ

Eric Key23:10:16

What about a human agent with AI voice? :thinking_face:

๐Ÿ˜‚ 3
Gene Kim, ITREV, Program Chair23:10:28

Nice. โ€ข strategic work vs. "more work" โ€ข flow vs. headcount โ€ข missionaries vs. mercenaries โ€ข community vs. zero-sum

๐Ÿ™Œ 1
โค๏ธ 1
๐Ÿš€ 1
Steve Pereira - Co-Author of Flow Engineering23:10:23

Great terms in this talk: โ€œconnecting org islandsโ€

โค๏ธ 1
โ˜๏ธ 1
๐Ÿ’ฏ 1
Matt Ring (he/him) - Sr. Product/Engineering Coach, John Deere23:10:01

Another Marty Cagan name drop in the DevOps community. Nice.

๐Ÿ’ฏ 5
Ann Perry - IT Revolution23:10:27

๐ŸŽ‰ Welcome, Moied Wahid, EVP, CTO Consumer Information Services, Housing, Verification Solutions, and Employer Services, Experian

๐Ÿ‘‹ 2
๐Ÿ‘ 1
Gene Kim, ITREV, Program Chair23:10:23

1.2MM customer profiles. 100s of models. (!!)

๐Ÿคฏ 1
Gene Kim, ITREV, Program Chair23:10:20

Batch jobs run against RocksDB (!!)

๐Ÿคฏ 1
Gene Kim, ITREV, Program Chair23:10:01

"DevOps is table stakes now. ML is still super complex."

โž• 1
Gene Kim, ITREV, Program Chair23:10:58

100s of PB data. Access controls super critical. (All relevant to PII, I'm guessing.)

โ˜๏ธ 1
Gene Kim, ITREV, Program Chair23:10:16

JupyterHub. One click into CI.

Topo Pal23:10:36

Ops have a new enemy: data scientists.

๐Ÿ˜† 6
Drew Khoury, Liatrio23:10:34

I want to see the Terraform modules ๐Ÿ‘€

โ˜๏ธ 1
Gene Kim, ITREV, Program Chair23:10:39

It's amazing to see the structure of these highly evolved MLOps stuff. Hundreds of models running in production, pipeline for getting them into production, A/B testing in performance, challenger models, etc. Input are data scientists in JupyterHub. Wild.

๐Ÿ™ 2
๐Ÿ‘† 2
๐Ÿคฏ 1
Tim Haagenson (American Airlines)23:10:23

Casually fixing a bug in the open-source spark code! ๐Ÿ’ช

๐Ÿ’ฏ 2
Gene Kim, ITREV, Program Chair23:10:45

Yes, that was quite the flex. Awesome. ๐Ÿ˜‚

๐Ÿ’ฏ 2
Manish Bhasin23:10:52

Irrational Optimism drives Transformation

๐Ÿ‘ 1
โ˜๏ธ 3
Sumit Agarwal (Cloud Engineering - JP Morgan Chase)23:10:16

Product teams with cloud budgets - awesome FinOps practice.

Ann Perry - IT Revolution23:10:58

๐ŸŽ‰ Welcome, Sascha Schรคrich, DevOps Evangelist at Deutsche Telekom IT

๐Ÿ‘ 2
๐Ÿ”ฅ 1
raghavema-fanniemae23:10:11

Working with machines is easy, working with people is tricky! ๐Ÿ˜‚

Drew Caldwell23:10:54

Landlines is not a word I thought I would hear in 2023

๐Ÿ“ž 5
โ˜Ž๏ธ 2