Fork me on GitHub
#genecon
<
2022-05-12
>
Ivan Krnic (Director of Engineering at CROZ)07:05:07

This is just an appreciation post for all organizers and speakers in US and Asia time zones... these 3 days must have been crazy for you...thanks 🍺 ...or should I say 😅

❤️ 11
6
🎉 1
BMK-SECTION6-TransformationArchitect08:05:53

Thank you Ivan; That shows how passionate people are , how great this community is 🙂 and how important attending DOES 🙂 Share, Learn, Grow together (Get together to go faster, IMHO Get together to grow together)

🎉 1
❤️ 2
Margueritte Kim (CEO, IT Revolution, she/her)09:05:57

Thank you for being here @ikrnic! For sure, we have all consumed a lot of caffeine 🤪 but happy to be here connecting with everyone. That being said, we absolutely cannot wait to be back in person in October! Have a great Day 3.

Slackbot09:05:09

Reminder: Remember all those talks you attended the first two days of the Summit? Please submit your feedback for those! It’s so valuable for us and the speakers. And after all, feedback is a gift and sharing is caring! Enter your feedback for those talks here: https://members.itrevolution.com/live/schedule https://devopsenterprise.slack.com/files/UATE4LJ94/F03E48CJRF1/image.png

Slackbot09:05:18

Reminder: We want to hear your stories from the Summit. What did you learn? Whom did you meet? What ideas are you taking back with you? What actions are you planning to take? Post in #summit-stories! https://devopsenterprise.slack.com/files/UATE4LJ94/F01CD9VE6JE/stories.png

Gene Kim, ITREV, Program Chair09:05:58

Looking forward to our fantastic Day 3 plenary morning talks! First up will be @christina_yakomin and @rdaitzman from Vanguard! 🎉 🎉

❤️ 2
🔥 1
Leah Brown - IT Revolution09:05:47

Head here to grab the journal and some free ebooks! https://members.itrevolution.com/free-ebooks

🌟 2
Ann Perry - IT Revolution09:05:50

Opening up Day 3 is @christina_yakomin, SRE and @rdaitzman, Senior Manager - Intermediary Technology Platform, of Vanguard, here to present: Iterative Enterprise SRE Transformation

Gene Kim, ITREV, Program Chair09:05:59

Vanguard Financial: created the index fund in 1975: in 2021, they had $8 trillion Assets Under Management (!!)

Gene Kim, ITREV, Program Chair09:05:55

PS: I loved the NPR Planet Money episode on index funds and Vanguard Financial founder John Bogle here: https://www.npr.org/sections/money/2019/01/23/688018907/episode-688-brilliant-vs-boring

🎉 1
Gene Kim, ITREV, Program Chair09:05:39

Vanguard: no physical retail presence — all contact is thru phone or web.

Gene Kim, ITREV, Program Chair09:05:28

@rdaitzman now is certified on wine serving and tasting! Wow! 🎉

🍷 3
Gene Kim, ITREV, Program Chair09:05:49

@christina_yakomin @rdaitzman From that NPR Planet Money interview, it sounded like technology was key from the very beginning to drive down management overhead costs, ideally down the zero — is that assumption correct?

Christina Yakomin09:05:35

absolutely! Vanguard had a mainframe before it was cool 😂

🔥 1
😎 1
Robbie Daitzman - Vanguard (Speaker - he/him)09:05:02

Yes! To find ways to scale the fund offerings without linearly scaling the “people” in the org was absolutely key!

Gene Kim, ITREV, Program Chair09:05:49

(I think I heard somewhere that some of those initial mainframe applications are still the system of record for customer accounts, which use VSAM tables?)

🙂 1
Christina Yakomin09:05:22

The majority of Vanguard's portfolio of applications has been modernized or has a modernization effort in progress. Last I heard, we're on track for 90% modernized/cloud-native by the end of 2024. As for that last 10%? 🙂 Let's just say we haven't completely freed ourselves of that mainframe yet

🔥 1
Gene Kim, ITREV, Program Chair09:05:16

If all goes well, we’re going to hear from ING CTO Ron van Kemenade on how they turned off their last mainframe three weeks ago! (And retired 250+ applications which ran on them!)

🤯 2
😲 2
🙌 3
Jeff Gallimore (CTIO - Excella)09:05:46

i can imagine what things looked like (or didn’t, as the case may be) with that lack of visibility. it’s like flying blind.

🙂 1
Scott Prugh (DOES Prog Committee)09:05:37

Adding visibility to heritage systems is a great technique to modernize... Marianne Bellotti in Kill It with Fire discusses this at length.

Gene Kim, ITREV, Program Chair09:05:12

The initial route to cloud was to “lift and shift” the PaaS to cloud, which would bring all the applications that run on it with it!

Gene Kim, ITREV, Program Chair09:05:01

Great for the developers — not nearly as great for the people who had to run the platform! 😆

😂 1
Jeff Gallimore (CTIO - Excella)09:05:55

@christina_yakomin what were the problems that the “lift and shift” approach caused?

Christina Yakomin09:05:24

Product teams weren't familiar with cloud operations, and the team supporting the underlying microservice platform wasn't able to scale their support offering as quickly as the cloud infrastructure was scaling. The team became overwhelmed quickly (source: I was on the team, that on-call was ROUGH), and the teams leveraging the platform weren't receiving the level of service and availability that they wanted. This prompted the transition to something that looks more like DevOps.

👀 1
Ann Marie Fred - Red Hat09:05:35

Failing to account for the work required to keep Kube clusters healthy - I’ve seen that mistake made multiple times!

☝️ 1
Gene Kim, ITREV, Program Chair09:05:10

(route to the cloud; some embraced serverless Lambdas, some ran in Kubernetes)

Christina Yakomin09:05:31

I gave a talk on FMEA at Vanguard at SRECon22! https://youtu.be/NuZ0y8W3Cjk

📝 2
👍 1
Andrew Salt - ScholarPack09:05:07

@christina_yakomin you mention AWS ECS (Fargate). Interesting you decided to go that route rather than K8S. We also run on Fargate and it works really well with low overhead. Interested to hear why you chose Fargate?

1
Christina Yakomin09:05:28

Fargate was much more similar to our private cloud containerization platform than the complex orchestration of K8S. Teams barely understood basic operations, and we were trying to transition them to DevOps. Kube is great, but it's a little intimidating at first even for experienced ops folks. For the majority of our workloads, we agreed with your assessment. Fargate works really well with low overhead. Now that we're a few years into our cloud journey, an internal platform for running apps on EKS has emerged and we're running our first few production applications for Vanguard as of this year. This is a great option for the teams that are ready to handle K8S orchestration and want the benefits of that control plane, an in-cluster mesh, etc.

👍 1
Andrew Salt - ScholarPack09:05:48

Can you call out what the limitations were for Fargate that made you look at K8S?

Christina Yakomin09:05:18

not so much limitations of Fargate as it was the ease of extensibility of K8S. So much is built to plug-and-play with K8S, and some teams were eager to get their hands on those (mostly open-source) tools.

Christina Yakomin09:05:37

You can achieve the same with Fargate in most cases, just need to write it yourself

Andrew Salt - ScholarPack09:05:30

Thats good. Thanks Christina. We have been using it since released in the London region and love the fact it is fully managed. Our biggest challenge is defining the correct scaling policy per service.

Andrew Salt - ScholarPack09:05:00

I hear a lot of talk about K8S and wonder why would anyone want to manage all of that. Your answer helps 👍

Gene Kim, ITREV, Program Chair09:05:59

As part of Chaos Engineering effort, @christina_yakomin crashed build systems , to see the mayhem that would ensue?!

Ben Grinnell - North Highland and DOES PC09:05:08

So many cloud migrations I see are just that. Lift and shift driven by a data centre contract ending - the benefits of the cloud transformation are much greater than that

❤️ 4
Robbie Daitzman - Vanguard (Speaker - he/him)09:05:48

For us, the “lift and shift” was a great way to take the first step into cloud and get ourselves out of the business of running a data center. It's been great to see the transformation since then to take advantage of more of the benefits!

Matt Cobby (Director of Engineering, Deloitte)09:05:59

Fire Drills - I've been discussing the idea that we do office building evacuations on a regular basis and test alarms every week, so why don't we test our operational outage procedures every week?

👏 2
Christina Yakomin09:05:24

Last week I was in the office nearby some friends running an elevation, when the member of their team who was running point got a call to pick up his daughter from day care - she was sick. Another team member said "this is just like our fire drill!" referencing a time when we'd made a key team member off-limits for a simulated incident. It was really gratifying.

❤️ 2
Gene Kim, ITREV, Program Chair09:05:06

(these fire drills are a great way to train engineers who are just starting to be trained to go on-call. 😆 )

Gene Kim, ITREV, Program Chair09:05:02

OMG. The ultimate heisenbug problem — how do you recreate the build system failures only caused by high build loads! (Not in the code — in the build systems that build/test the code, that cause the build system to fail!!!)

😮 1
Luke Rettig - Target09:05:21

load testing (kinda) the build system… cool

🔥 3
💥 1
Leena Pradhan09:05:27

Break testing CICD pipeline! Great!

Sandeep Dabas09:05:54

@christina_yakomin It would be great to see designs for the changing architecture from monolithic to microservice and moving to cloud/containers

Christina Yakomin09:05:50

This re:invent keynote appearance from our former CTO is a few years old now, but tells this monolith-to-microservice story pretty well! Arch. diagrams show up about halfway through: https://youtu.be/8kzOj9cStGo

👏 3
🙏 1
👍 1
Ann Marie Fred - Red Hat09:05:38

Cloning dashboards - interesting story.

😆 1
Gene Kim, ITREV, Program Chair09:05:46

“teams no longer had to submit tickets to get metrics.”

❤️ 2
Leena Pradhan09:05:55

How did you balance between really required logs and not so critical logs?

3
Christina Yakomin09:05:34

This is difficult, because we never want to tell teams they can't log something that might be useful in an incident. Ultimately, we've identified what "should be a log" vs. what "should maybe be in a metric/trace" and focused on getting the right telemetry data into the right tool. Now, we're looking at keeping the logs for apps with the most operational complexity in a higher-powered tool, like Splunk, while putting most logs in a more simple log aggregator, like CloudWatch Logs

👍 1
Gene Kim, ITREV, Program Chair09:05:42

(When Google Cloud deprecated v2 of their monitoring/graphing service, I went from too many dashboards to none. 😆 😭 )

Gene Kim, ITREV, Program Chair09:05:58

(I may have missed a couple of emails warning about deprecation…)

Gene Kim, ITREV, Program Chair09:05:31

“Logging costs increased exponentially.” “We could no longer see important error messages.” 😆

Ann Marie Fred - Red Hat09:05:22

I thought CloudWatch was a log aggregator? Do you have a different term for it?

Ann Marie Fred - Red Hat09:05:40

Funny story. Literally this Monday, one of our engineers added the audit logs for our staging cluster to CloudWatch. Remembering my past experience with these things, I asked him to check our AWS bill. $100/day for those audit logs from staging. OOPS.

😆 1
Gene Kim, ITREV, Program Chair09:05:38

Awesome. (When I was first playing with Google BigQuery with the GitHub data set, I ran my first $100 query, by accidentally scanning every row. Cloud is amazing, but can be super expensive. 🙂

🤑 1
Robbie Daitzman - Vanguard (Speaker - he/him)09:05:43

We look at CloudWatch more from a metrics (and events) perspective rather than the log aggregation side Also, cost is such a great leading indicator for things that might be misconfigured in a cloud environment! My teams always wonder why I want to make sure our tags are up to date 😂

💯 1
Gene Kim, ITREV, Program Chair09:05:01

(The idea that I could have easily racked up $10K in query costs was… bracing.)

😭 1
Christina Yakomin09:05:03

CloudWatch logs is a log aggregator, CloudWatch metrics is a metrics aggregator. We use both, but Splunk has long been our primary log aggregator. CloudWatch Metrics is our primary metrics aggregator today

👍 1
Jeff Gallimore (CTIO - Excella)09:05:49

up = good, down = bad: good for executives, bad for engineers. :rolling_on_the_floor_laughing:

😂 3
Leena Pradhan09:05:10

Product owners to buy-in into SRE concept..

💯 2
Christina Yakomin09:05:09

our product owners now go through a full day Product Operations training, with a 2-hour block dedicated to SRE concepts. The feedback has been fantastic. Product owners are loving to be included in these conversations!

💯 1
Luke Rettig - Target09:05:14

Does your build pipeline slow down? Do you have so much logging you struggle to find what you are looking for? Do the amount of alerts you get overwhelm you? - well - the good news is, you are doing something right. the bad - you may need to iterate your SRE. Here’s a talk from @christina_yakomin that can help you

Luke Rettig - Target09:05:23

you are watching it - link to come at the end of the talk 😀

😂 1
Billy Hudson - ScholarPack - DevOps Engineer09:05:23

@christina_yakomin Monitoring is traditionally a role that falls on support/Ops, how did you get Developers excited to put together, build and use their own dashboards/metric data? was it putting them oncall? 😆

Christina Yakomin09:05:17

Giving them accountability (with some training wheels - there's still an escalation path) put them on the front lines, and gave them a stake in the availability of their apps. Better behavior around observability followed quickly. Now, some areas of the organization are begging our central teams to bring in more (expensive) observability tooling that they're excited about. It's a great sign!

🙌 1
Jennifer Riggins09:05:25

How big is the whole SRE team?

1
Christina Yakomin09:05:41

our SRE Coaching team is about 4 people, but there are SREs throughout the organization now! For example, our Retail sub-division of IT has its own team of SRE experts, with about 5 people + a manager. Then there's a few distributed SRE resources throughout the subdivision aligned directly to product teams. This is one of our fastest growing roles.

Scott Prugh (DOES Prog Committee)09:05:29

@christina_yakomin Yes! I love OpenTelemetry! Having great distributed tracing really improves understanding and 2 of the High Performance Capabilities!

1
👍 1
👀 1
Leena Pradhan09:05:02

How do the SRE leads interact or collaborate with DevOps teams?

Christina Yakomin09:05:23

Today, the SRE Leads are either embedded within the teams, or they're being engaged by the DevOps teams' on-calls through their own SRE on-call rotation (or 24/7 staffed SRE support team) as an escalation path. It differs throughout different pockets of the organization, and I'd say it's still evolving right now as teams try and test different approaches. We're figuring out what works best.

🎯 2
☝️ 1
💡 1
Gene Kim, ITREV, Program Chair09:05:02

I love this story from @rdaitzman — newer Vanguard Advisory business unit, which help external financial advisors who help with retirement planning, etc. As investment industry all race to lower costs, this is an important initiative for Vanguard — “it’s our first SaaS product.” I LOVE IT!

Gene Kim, ITREV, Program Chair09:05:23

“first time we’ve ever built an application like this.” So cool, @rdaitzman

Habib Rahman09:05:30

What differentiates the SREs from the traditional BAU teams?

Christina Yakomin09:05:14

They're not always separate teams! Sometimes a product team has both Application Engineers and Site Reliability Engineers. Application Engineers focus on feature delivery, while Site Reliability Engineers focus on availability, resilience, and other non-functional requirements. And both are responsible for ensuring everything they do is aligned with security controls, of course. In this scenario, they typically share on-call.

👍 1
Ann Marie Fred - Red Hat09:05:50

Talking to business partners about potential failures, wow.

🤝 3
Robbie Daitzman - Vanguard (Speaker - he/him)09:05:45

It was a big step for us! Not has it helped shift our partners from the “100% available mindset” but also catalyzed conversations around graceful degradation

1
Jennifer Riggins09:05:22

how do you define business partners in this situation?

Gene Kim, ITREV, Program Chair09:05:55

“Failure modes and effects analysis: we engage both engineers and business partners; deliberately understand what business response should be: comms and notification, processes that would be activated.” <--- so amazing.

Gene Kim, ITREV, Program Chair09:05:08

Totally, @ann.marie.99 — so great!!!

🤯 3
Leena Pradhan09:05:33

Great learning from this session! Loving it.

❤️ 3
Moira Cheng09:05:10

@christina_yakomin could you kindly explain what the SRE champions do vs. the SRE Leads? Do SRE leads have accountability for SRE techniques and practices for their application and the SRE champions have a more holistic view across different project teams/product lines?

Robbie Daitzman - Vanguard (Speaker - he/him)09:05:08

I have SRE Champions on my team who focus on SRE practices and tooling to be able to integrate into our apps, but they are also developers on my team - SRE isn't a full time job for them but they help to instill that focus within the team!

🙌 1
☝️ 1
Christina Yakomin09:05:05

In short, yes. We have 3 job titles for SRE. SRE Practitioners are typically embedded on a product team, focused on a single application. SRE Leads handle SRE tasks and coordination across a family of related products. they're still doing hands-on SRE work, but they're typically the ones that you go-to for large-scale load tests, chaos tests, incident reviews, etc. SRE Champions are higher level. They're a central team per sub-division of IT (we have several) responsible for sub-division specific tooling decisions, patterns, communication across subdivisions and within their own sub-division, and more. They collect and distribute metrics regarding SRE adoption, and provide consultative expertise as we're in the interim state of trying to scale up SREs across the organization.

☝️ 2
👍 3
Moira Cheng09:05:33

Appreciate the detailed answer @christina_yakomin ❤️

Alex Broderick-Forster, IT Revolution, Event Staff09:05:56

@jenniferkriggins think you’ll find these comments interesting ^^

❤️ 1
Jennifer Riggins09:05:40

indeed thanks Alex!

Gene Kim, ITREV, Program Chair09:05:03

“uh, yeah, we’d have to send an apology letter to everyone, apologizing for incorrect account balances and values — and it may be a week before correct account balances can be shown. can we delay the deployment until we’ve tested this?” (I was telling @rdaitzman that Phoenix Project would have been written totally differently had I heard this talk before writing it!!) I just love this presentation from Robbie and @christina_yakomin!

😆 3
Moira Cheng09:05:30

Also @christina_yakomin what's your view on having DevOps Coaches and SRE Coaches? Would you amalgamate?

👍 2
Christina Yakomin09:05:25

Today at Vanguard, we have Agile Coaches and we have SRE Coaches, but we don't really have DevOps coaches. Maybe we should have 😂 but for us, yeah I'd say the SRE Coaching team does handle a lot of coaching and evangelism in the DevOps space beyond just the fundamentals of SRE

Moira Cheng09:05:22

Brilliant - understood. Thank you 🙂

Leena Pradhan09:05:25

From buzzword to mindset - real transformation.

☝️ 2
👍 2
Gene Kim, ITREV, Program Chair09:05:50

“every hour taken from engineer to do things like SRE training is an hour they’re not writing a feature — how do we balance?” “boring week-long training?” (I doubt it’s actually boring!!)

Leena Pradhan09:05:30

This is always a challenge - getting people to spend time on training when they could be doing more on delivering.

Fokko V.09:05:28

This is about short term investments vs long term investments. Generally the choice needs to be supported by higher management 😐

👍 1
Jeff Gallimore (CTIO - Excella)09:05:43

budgeting pops up again as something that needs to change to support continued transformation. 💰

☝️ 3
1
Gene Kim, ITREV, Program Chair09:05:24

@christina_yakomin Is there an easy answer for the question: “given these success stories, to what extent have you gotten all the leadership support you’ve been seeking?”

👍 1
Christina Yakomin09:05:45

I'd say we're about 60% there. Some leaders are more cautious, and also still pressed for funding as always, and having a really hard time hiring like most are right now. They're in most cases optimistic about the approach, but waiting to see a longer history of demonstrated positive results before investing valuable time and effort (and $$$) to hire SREs in one of the most challenging talent markets we've ever seen.

🔥 1
Gene Kim, ITREV, Program Chair09:05:04

Amazing work — congratulations again on your amazing achievements!!!

Ann Marie Fred - Red Hat09:05:35

Learning how to set up your first metrics and alerts is HARD. 😅

Gene Kim, ITREV, Program Chair09:05:52

“in pockets where implemented, blameless PIR leads to amazing breakthroughs”

💯 3
❤️ 3
Jennifer Riggins09:05:26

Are Vanguard engineers co-located or a distributed team?

👀 1
Robbie Daitzman - Vanguard (Speaker - he/him)09:05:47

A little bit of both! We do have teams based in our various offices across the globe, but a lot of teams are also distributed across those locations!

Scott Prugh (DOES Prog Committee)09:05:43

Nicely done @christina_yakomin and @rdaitzman!!

Gene Kim, ITREV, Program Chair09:05:45

THANK YOU SO MUCH, @christina_yakomin and @rdaitzman!!

👏 5
1
Ann Perry - IT Revolution09:05:04

Now introducing @andrew.letherby, Service Owner of the Customer Insight Platform, HMRC and @caitlin.smith, Delivery Lead, Equal Experts, presenting: Dynamic Decision Making at HMRC - Moving to Data as a Product During a Pandemic. Here to answer any questions with Caitlin today is @mercedes.gozalbomorag, CIP Service Designer.

Christian Knuth09:05:08

Enjoyed your presentation very much! Helps me a lot!

🎉 2
Moira Cheng09:05:34

Amazingly insightful and completely relevant to what we're going through in Vodafone as well. Thank you @christina_yakomin d @rdaitzman

❤️ 4
Alex Broderick-Forster, IT Revolution, Event Staff09:05:44

Iterative Enterprise SRE Transformation (Vanguard) Christina Yakomin, SRE, Vanguard Robbie Daitzman, Senior Manager - Intermediary Technology Platform 🎉 Now Available for Sharing: https://videos.itrevolution.com/watch/708991000/

Leena Pradhan09:05:47

Amazing presentation, learning, insights. Thanks so much @christina_yakomin , @rdaitzman

❤️ 2
Sandeep Dabas09:05:59

@christina_yakomin Nice to hear covering most of the e2e journey for transformation. Good luck with next steps 👏

Gene Kim, ITREV, Program Chair09:05:05

Hello, @andrew.letherby and @caitlin.smith014!!! Continuing the amazing story told last year from HMRC!

👋 3
Gene Kim, ITREV, Program Chair09:05:25

…this is the story hinted at last year… so great that we’ll learn more about how UK averted economic ruin thru the heroic work done at HMRC!

❤️ 2
Gene Kim, ITREV, Program Chair09:05:01

“HMRC: you may know them from income tax, VAT.. but much more… import/export, customs declaration, manage govt banking service that support govt orgs (payments and repayments across govt)” (!!)

Gene Kim, ITREV, Program Chair09:05:46

@andrew.letherby mentioned to me that of the 20 government services deemed “critical,” 17 are run by HRMC. Became very relevant as pandemic started shutting society down, as these services were depended upon by some of the most vulnerable in population. Thank you for your amazing work, @andrew.letherby and team!

😮 1
Jeff Gallimore (CTIO - Excella)09:05:32

as the flow of work coming in goes up, and capacity stays the same, i wonder what would happen… :thinking_face:

😥 2
Scott Prugh (DOES Prog Committee)09:05:10

Just add more people quickly to get the work done. That usually goes well.:satisfied:

1
Gene Kim, ITREV, Program Chair09:05:53

(What didn’t make it into @andrew.letherby introduction: he spent over 20 years at HMRC, including investigations of banana smugglers. 🙂

🎉 1
🍌 1
Gene Kim, ITREV, Program Chair09:05:40

“pandemic led to mandate that said, if you can work from home, you should; entire sectors of economies unable to trade.”

Gene Kim, ITREV, Program Chair09:05:45

“HMRC: get financial help as fast as possible to those in need.” (Hundreds of billions GBP put into economy — but it was an obvious target for organized crime (!!!))

Gene Kim, ITREV, Program Chair09:05:46

“we thought about what HMRC analysts needed; we’ll have an unprecedented # of cases to analyze, to detect whether it was a fraudulent claim” (all citizens were entitled to monies; however, criminals creating fictitious organizations claiming monies were not.)

Jeff Gallimore (CTIO - Excella)09:05:32

“to prepare for uncertainty, we designed for flexibility”. 💯

☝️ 3
Gene Kim, ITREV, Program Chair09:05:07

@caitlin.smith014 was tasked with more than doubling the size of this team supporting these efforts on VERY short notice…. (less than 4 weeks for the initial efforts, I think?)

😱 1
Gene Kim, ITREV, Program Chair09:05:19

the astonishing opportunity: “could we automate the detection/prevention to filter fraudulent/erroneous transactions, or even prevent it”

Gene Kim, ITREV, Program Chair09:05:04

“we focused on well understood problems: specific attributes already used by business analysts; addresses, bank accounts, where we had good sources of information, could be verified; good sources for intelligence within and outside of HRMC, how specific accounts/addresses were/are being used.” “by sticking to these familiar concepts: we check a thing, we decide good/bad, and decide what next step is; these familiar concepts enabled business to know how to use these capabilities.”

Gene Kim, ITREV, Program Chair09:05:02

“could we assess claims while people were filling out form?” <----- WOW

🤯 4
Gene Kim, ITREV, Program Chair09:05:48

“good claims would be sent through — potentially problematic ones would be reviewed. external data could be brought to bear, to increase fidelity.”

Caitlin Smith09:05:47

The first iteration of the insights service for the Statutory Sick Pay scheme took four weeks to build. The existing team integrated our existing data pipeline. We only grew the team when new capabilities were added.

👀 1
💪 1
❤️ 3
Gene Kim, ITREV, Program Chair09:05:29

“to scale efforts, we needed to onboard new team members; we reached out to people who already worked on digital platform at HMRC; so many wanted to come back, b/c of their great experiences at HRMC.” ^^ amazing — what a testament to HMRC being a great place to work?? (am I getting that right, @caitlin.smith014?)

Caitlin Smith10:05:47

The culture within HMRC Digital is perhaps extremely different from what people might expect. It’s a great place to work full of brilliant and dedicated people trying to do the right thing for the public.

❤️ 3
Caitlin Smith10:05:16

The whole organisation came together to deliver for citizens during the pandemic.

❤️ 3
Gene Kim, ITREV, Program Chair09:05:24

“we discouraged overtime; only had to do it once or twice;”

Jeff Gallimore (CTIO - Excella)09:05:45

“we brought in a burnout specialist” ❤️ ❤️ ❤️ ❤️ ❤️ ❤️ ❤️

❤️ 8
Gene Kim, ITREV, Program Chair09:05:56

“we brought out burnout specialist; some were at home alone, where colleagues were a lifeline; others had families, struggling with homeschooling. we knew we had to look after team.” ❤️

❤️ 3
Gene Kim, ITREV, Program Chair10:05:34

Proof of their ability to scale with pandemic

🤯 1
Gene Kim, ITREV, Program Chair10:05:03

“3,960 claims per minutes — far higher than could have been analyzed manually”

🎉 5
Gene Kim, ITREV, Program Chair10:05:39

“100% of claims were being assessed upfront — another first” <-- ensuring that public monies going to those in need (vs fraudulent claims!)

Gene Kim, ITREV, Program Chair10:05:31

“150% more users; 2.3MM searches internally” <-- showing how widely these capabilities were accessed, as well as supporting internal workload needed to process claims.

Gene Kim, ITREV, Program Chair10:05:28

https://devopsenterprise.slack.com/archives/C015DQFEGMT/p1652349707995149?thread_ts=1652349509.705569&amp;cid=C015DQFEGMT “The culture within HMRC Digital is perhaps extremely different from what people might expect. It’s a great place to work full of brilliant and dedicated people trying to do the right thing for the public. “The whole organisation came together to deliver for citizens during the pandemic”

Gene Kim, ITREV, Program Chair10:05:01

“data was not only available, but it was safe and secure.” 🎉

andy henderson10:05:30

@caitlin.smith014 - how did you scale up that user-onboarding, did you change the platform to make it simpler to adopt, or improve the training materials/engagement?

Caitlin Smith10:05:40

We created a reusable onboarding pack. We also worked side by side with the people who already used the tool to onboard people into their teams. To make this work we made sure there were clear roles and responsibilities.

❤️ 1
Gene Kim, ITREV, Program Chair10:05:46

“data is our product”

🎯 4
Luke Rettig - Target10:05:05

i’m seeing a trend here Gene

❤️ 2
Jeff Gallimore (CTIO - Excella)10:05:21

@simone.steel talked about that yesterday, too

Luke Rettig - Target10:05:41

and @chanade.hemming on Tuesday

🎯 2
💡 1
Chanade Hemming13:05:46

data products everywhere 😍

Gene Kim, ITREV, Program Chair10:05:47

“looking back: without those teams, we wouldn’t have been able to build [these critical capabilities] at HMRC.” What an incredible contribution, @caitlin.smith014!

Gene Kim, ITREV, Program Chair10:05:36

“Our role: deliver a critical service, but at the same time, because of its attractiveness to criminals: we could prevent/detect it happening, have information to pursue people who took money they weren’t entitled to.”

❤️ 3
Gene Kim, ITREV, Program Chair10:05:17

“the whole of HMRC had a single focus for this period of time; delivering of critical services; we had a unique line of communications from delivery teams to policy makers in central government, setting levels of risk appetite very clearly, so we could balance delivery of services vs management of risk. “this clear line of communications made decision making clear and direct; we are attempting to preserve this post-pandemic; policy goals directly and alongside delivery teams.” Amazing. Unity of mission/purpose, allowed massive decentralized execution.

Jeff Gallimore (CTIO - Excella)10:05:35

we heard about that from @jmrichardson1 and @emily356 in their leadership workshop yesterday. https://videos.itrevolution.com/watch/687309828/

🔥 1
Gene Kim, ITREV, Program Chair10:05:22

Help they’re looking for: how does one fund innovation across a large service?

Gene Kim, ITREV, Program Chair10:05:52

Thank you so much, @andrew.letherby and @caitlin.smith014 — was amazing to hear about this side of the story of the heroic work at HRMC!

❤️ 7
Alex Broderick-Forster, IT Revolution, Event Staff10:05:19

Dynamic Decision Making at HMRC - Moving to Data as a Product During a Pandemic Andrew Letherby, Service Owner of the Customer Insight Platform, HMRC Caitlin Smith, Delivery Lead, Equal Experts 🎉 Now Available for Sharing: https://videos.itrevolution.com/watch/708991017/

👏 2
Ann Perry - IT Revolution10:05:22

✈️ Let's welcome back @pieter.jordaan, here to give an update on TUI Group's Transformation Journey ✈️

Gene Kim, ITREV, Program Chair10:05:08

(I’m so fascinated that UK HMRC has obviously created a reputation of being an amazing place to work. ❤️ 🎉 I’m less sure this is the case for similar agencies in other countries…)

2
Gene Kim, ITREV, Program Chair10:05:44

Thank you for giving us an update on the heroic work you talked about last year, @pieter.jordaan!!! First data point: he obviously hasn’t been fired. 🙂

🙌 3
❤️ 1
👏 1
Pieter Jordaan (CTO -TUI Group)15:05:09

Yes, happy to report! Like the Croods say " Still ALIVE!"

Gene Kim, ITREV, Program Chair10:05:11

(Which was on my mind when I reached out to him — was wondering whether my email to him would bounce. 🙂

👍 1
Gene Kim, ITREV, Program Chair10:05:02

How utterly visionary: use period when there was 95%+ drop in transactions to do massive modernization/rewrite of most core applications.

💯 1
Jeff Gallimore (CTIO - Excella)10:05:30

a real-world example of this ☝️

Luke Rettig - Target10:05:52

a time when most divest their spend… these folks doubled down!

🎉 1
💥 1
Vlad Ukis11:05:44

@jeff.gallimore thanks a ton for this reference!

thankyou 1
Gene Kim, ITREV, Program Chair10:05:43

Second data point: focus was the more core processes at TUI Group: airplane inventory, routing, pricing, sales, seating, fulfillment, ticketing. “We completely rewrote these applications.” (!!) I still can’t get over this.

👏 1
Gene Kim, ITREV, Program Chair10:05:01

“We rebuilt these applications in record time.”

Giulio Vian, Unum10:05:23

the opposite of other business who saw increase in work

☝️ 1
🎯 1
Gene Kim, ITREV, Program Chair10:05:32

“We are now on the second season running on these new platforms.” (I.e., they’re working, and supporting all relevant business operations.) 🤯

Jeff Gallimore (CTIO - Excella)10:05:10

so… it worked. 🤷 🙌

🔥 1
Gene Kim, ITREV, Program Chair10:05:30

I honestly was wondering whether it would work last year — and it did!!!

Pieter Jordaan (CTO -TUI Group)15:05:27

@jeff.gallimore, working and getting better. Remember, the design is wrong, how do we make it less wrong?

🙌 1
Gene Kim, ITREV, Program Chair10:05:02

“we digitalized so much of the travel experience: flight cancellation, baggage updates, what bus to get on; these are what make/break travel experience.” (This is something that thousands of people used to do; hold signs to show people what bus to get on!)

Gene Kim, ITREV, Program Chair10:05:08

“we weren’t flying; we weren’t sending people on airplanes; it’s easy to say ‘let’s rebuild it.’ “but now it was time to say ‘let’s fly!!’” <--- more scary to say! 😆

😉 1
Moira Cheng10:05:38

Is it only me... but this talk is making me want to book a flight (of course via TUI) to try the experience and go on holiday somewhere:rolling_on_the_floor_laughing:

🏝️ 1
❤️ 1
👏 1
Pieter Jordaan (CTO -TUI Group)15:05:08

Thank you @moira.cheng, Aviation is having a rough time to restart. Lot of supply chain issues in airports etc but we are very excited to have used this opportunity to reposition ourselves.

Jeff Gallimore (CTIO - Excella)10:05:54

“rebuild the aviation platform while flying” 👈 a different take on “the rebuild the airplane in flight” :rolling_on_the_floor_laughing:

😂 1
🎉 1
Christian Knuth10:05:57

A company having bad times at the markets but nonetheless spending money in renovating their IT! One needs to have this mindset (instead on 'we must reduce costs now') and the sufficient funds to do that! Hats off!

👍 1
1
Scott Prugh (DOES Prog Committee)10:05:01

This is a great story of how "survival instincts" change normal operating behavior to deliver outcomes that can never be done with "BAU behavior". We also seem to see this post war-room or code-yellow working scenarios where teams get amazing things done and afterwards go back to "BAU" and fail to achieve these results. Question: How do we deliver these outcomes in "BAU times"??

👀 2
👍 1
❤️ 1
Gene Kim, ITREV, Program Chair10:05:03

^^^ (wait for the testimonial!!! it’s wild.)

Luke Rettig - Target10:05:32

“Business Migration follow IT Consolidation” - interesting to see Organizational structure evolution as companies continue to align business and tech strategy into one

Gene Kim, ITREV, Program Chair10:05:04

“we didn’t have the luxury to start small; we had to go big.” (focus was moving largest market to cloud, to save the most $$$ fastest.)

Gene Kim, ITREV, Program Chair10:05:17

“If you took a holiday in last year, you were flying on our new platform!” (!!!)

Gene Kim, ITREV, Program Chair10:05:43

It’s funny: I really did ask him, “were you fired? of are you a hero?” His answer is amazing and profoud.

Virginia Laurenzano NSA/MARFORCYBER10:05:20

Free Solo is a terrifying movie and I love that quote!

❤️ 1
💯 1
Gene Kim, ITREV, Program Chair10:05:20

Alex Honnold: “Free Solo movie” “If you fall, you die.”

❤️ 2
Gene Kim, ITREV, Program Chair10:05:18

“Mimsdai Purja: climbed 22K foot peaks (?) in 6 months; previous record; 14 years!!!) ^^^ “no one wanted to be associated with him, no one wanted to give him $$, because they didn’t want to be associated with failure.” “to put in context, most climbs would take 3 months to get ready.”

Gene Kim, ITREV, Program Chair10:05:37

“then climbed another mountain 48 hours later (!!)”

Philip Day10:05:47

Some of @pieter.jordaan’s comments early in this talk about UX reminded me of this recent travel IT/system UX horror show that appeared on my linkedin feed - BA need to get on this conference! (It happened to Dan North personally, who happens to be a major figure in the software world, and is therefore able to articulate it particularly well) https://dannorth.net/2022/04/20/a-bad-system/

❤️ 2
Randy Shoup10:05:12

Horrible story, extremely well told.

Gene Kim, ITREV, Program Chair10:05:59

“they didn’t set a new milestone; they broke barriers, and set a whole new mindset for future efforts”

👏 1
Scott Prugh (DOES Prog Committee)10:05:04

This commentary on risk is so key. Many folks in business view the status quo as the least risky path and change as the riskiest path. This stance emboldens sunk cost fallacy.

💯 4
🎯 2
👏 1
Gene Kim, ITREV, Program Chair10:05:00

“2.5 years ago, this was an idea on powerpoint slide; now it’s reality.”

❤️ 1
👍 1
Glenn Wilson, Author of DevSecOps10:05:10

I’m reminded of Shigeo Shingo who changed the way of thinking about die cast setup, reducing them from over a day to single minute tasks (Single Minute Exchange of Die - SMED). It was a game changer in manufacturing

👍 1
Gene Kim, ITREV, Program Chair10:05:37

(I know a tech executive at a travel booking company who looks at this story with genuine envy.)

❤️ 1
Gene Kim, ITREV, Program Chair10:05:16

“Everything is risky, but doing nothing is the riskiest approach”

💯 1
🙌 3
❤️ 2
Jeff Gallimore (CTIO - Excella)10:05:34

there is so much awesome wisdom packed on this one slide.

Gene Kim, ITREV, Program Chair10:05:15

“embolden successes” 🔥

Luke Rettig - Target10:05:34

“how can we make it less wrong?” … such a critical mindset

🎯 4
💯 1
Gene Kim, ITREV, Program Chair10:05:12

“we try to make the one-way decisions early; avoid them when we can. why? because we know our design is wrong.”

Glenn Wilson, Author of DevSecOps10:05:24

“how can we make it less wrong” is a much better mindset than “adopting best practices”

Gene Kim, ITREV, Program Chair10:05:32

“what is the magic? was it cloud? devops? the real magic is in small, motivated and focused teams”

❤️ 1
☝️ 2
1
Scott Prugh (DOES Prog Committee)10:05:10

Yes!! Doing nothing is often the riskiest approach!! Also, doing something that isn't working(or working well) is also a risky approach. Even late in the game, be willing to change your approach if there are better options to success!

❤️ 2
andy henderson10:05:16

Love that 'less wrong' point - we often use similar "what's least worse choice we can make - let's try that"

👍 1
❤️ 2
Giulio Vian, Unum10:05:33

this is mostly a business decision to invest in improving when tide is low to be ready when the tide is high and sail

Gene Kim, ITREV, Program Chair10:05:22

“there is so much passion in products now; collaboration in remote-first; all had to be figured out post-pandemic; all very difficult dynamics.” “was made clear when Ukarainian engineers, who we tried to help as much as possible; take care of your personal situation; we planned for them to take time off; but despite all this, even though they are in cities being bombed, they keep showing up to work; there’s no pressure to work, but it’s driven by passion and comraderie”

❤️ 2
🫂 1
Pieter Jordaan (CTO -TUI Group)15:05:36

This is the real magic. Passion of people. ❤️

Giulio Vian, Unum10:05:31

the great [story] is that business realized that every company is a sw company (cit.) and invested in IT

Anshul10:05:44

Your Design is Wrong. How to make it Less Wrong? This statement is such a mindset changing @pieter.jordaan

❤️ 1
Pieter Jordaan (CTO -TUI Group)15:05:33

@anshul.kumarbansal,This is a huge mind-shift but really get to the core believe of agile mindset.

Gene Kim, ITREV, Program Chair10:05:58

“when people find love for their product, and a place to hook onto exciting vision, that’s the treasure. Money isn’t the scarce commodity; small motivated teams are.”

Gene Kim, ITREV, Program Chair10:05:45

@moira.cheng here’s the testimonial… a TUI Group customer, who also happens to be an engineer at Google!

❤️ 2
Gene Kim, ITREV, Program Chair10:05:06

“I would show this to the team, who worked so hard during pandemic… the fruit of our labors has put us into the position post-COVID that is completely different pre-COVID.”

Giulio Vian, Unum10:05:16

another element is the joy of engineers allowed to redo, improve and change

Gene Kim, ITREV, Program Chair10:05:59

“when he started, everyone thought he was crazy; he did in 14 months what everyone else did in 14 years; he had no money, had to re-mortgage his home to fund his climbs.”

❤️ 3
Anshul10:05:04

There is already an assumption that the design is wrong which means, it's quite possible that some issue may crop up in future in the design, but that should not stop us from proceeding

Jeff Gallimore (CTIO - Excella)10:05:31

Successful companies often do that 😉 🙌 :rolling_on_the_floor_laughing:

Gene Kim, ITREV, Program Chair10:05:56

(I am so grateful for @pieter.jordaan for putting that presentation together — so much was based on my question, “did you get fired, or are you a hero?” His response was so startling, thoughtful, and even profound.) THANK YOU, @pieter.jordaan!!

👏 3
❤️ 1
Pieter Jordaan (CTO -TUI Group)15:05:47

Thank you @genek, I know a lot of companies are going through similar journeys. Happy to answer any questions here today.

Randy Shoup10:05:10

Really inspiring, thanks @pieter.jordaan!

❤️ 4
Pieter Jordaan (CTO -TUI Group)15:05:33

Thank you @rhinders. Feel free to drop me a DM. Would love to hear what part inspired you.

Ann Perry - IT Revolution10:05:18

Now introducing DevOps Enterprise Summit Programming Committee member, @ben.grinnell, Managing Director and Global Lead of the Technology and Digital at North Highland, presenting Sourcing in the Age of Digital Business

Alex Broderick-Forster, IT Revolution, Event Staff10:05:28

TUI Group: Transformation Journey Pieter Jordaan, Group Chief Technology Officer - Technology Transformation, TUI Group 🎉 Now Available for Sharing: https://videos.itrevolution.com/watch/708991029/

Pieter Jordaan (CTO -TUI Group)15:05:35

Thank you for the opportunity. Please feel free to contact me for any discussion and feedback is much appreciated. Thank you for the compliments and feedback in the chat.

Gene Kim, ITREV, Program Chair10:05:58

Up next: @ben.grinnell — he’s been to all 15 DevOps Enterprise Summits, has been on the Programming Committee since 2015, but has never spoken here!

Ben Grinnell - North Highland and DOES PC10:05:05

Really inspiring - thanks for scheduling me after that one Gene

4
😆 2
Gene Kim, ITREV, Program Chair10:05:25

An uncomfortable moment: @ben.grinnell pointed out a bias we’ve had against outsourcers — deliberate, as we typically favored enterprises who brought in engineering talent. But as a result, we’ve been overlooking something very important — the result is this presentation, and the start of an important and new effort.

☝️ 2
Luke Rettig - Target10:05:55

I’m 1000% biased… i’m at the edge of my seat

🔥 1
Gene Kim, ITREV, Program Chair10:05:51

“enterprising using outsourcers, and outsourcers are under-represented here.”

Gene Kim, ITREV, Program Chair10:05:34

“In my government service, there was nothing that supported trust, collaboration; they were set against each other; impossible to implement great practices discussed here in that context.”

Gene Kim, ITREV, Program Chair10:05:19

“every enterprise I speak to are trying to grow their capabilities. True?” <--- put emoji here.

👍 7
Ben Grinnell - North Highland and DOES PC10:05:33

Every enterprise I speak to is trying to grow their internal IT capability. If you are working for a large enterprise, let us know if that's the case in your firm? 👍 👎

👍 4
Matt Cobby (Director of Engineering, Deloitte)10:05:12

Absolutely. I spent 3 years doing just that. I'm now the other side and teaching others how to do it.

Matt Cobby (Director of Engineering, Deloitte)10:05:40

It's part of a personal mission to help more people realise their career dreams and to change the technology market and how it approaches talent to help make it more inclusive.

Phil Gadzinski Bupa11:05:54

its actually my role here at Bupa - Global Centre of Enablement (not excellence!) focused on tech capability uplift

🙌 1
Virginia Laurenzano NSA/MARFORCYBER10:05:54

'like it or not, they're in the IT business'

🎯 1
💯 1
Gene Kim, ITREV, Program Chair10:05:01

“being fastest is now more important than being first”. 🔥

Gene Kim, ITREV, Program Chair10:05:58

Profound question: “do I have to insource to accelerate my DevOps goal” There’s a knee-jerk reaction, given how common “70% outsourced -> 30% outsource.” Answer is more complicated!

Gene Kim, ITREV, Program Chair10:05:34

There was a great session yesterday from TCS, about bringing DevOps to their customer portfolio — let me find link.

👍 2
Alex Broderick-Forster, IT Revolution, Event Staff10:05:25

@dhruba.chaudhuri @leena.pradhan @musthafa.s @tm.manikandan.b @tabrez.muhammad

Gene Kim, ITREV, Program Chair10:05:45

Thank you for these amazing presentations!! @ben.grinnell and I will be following up!

Leena Pradhan11:05:24

Thanks @genek

🙌 1
Gene Kim, ITREV, Program Chair11:05:20

PS: great presentation @leena.pradhan and team!!!

👍 1
Jeff Gallimore (CTIO - Excella)10:05:37

💯 to the lack of alignment across functions

💯 1
Gene Kim, ITREV, Program Chair10:05:59

@ben.grinnell does an amazing job framing the challenge for outsourcer and their clients.

Ben Grinnell - North Highland and DOES PC10:05:12

The right hand side of this slide should read: 1.Help growing their internal capability and reducing the dependency 2.Increased automation to reduce delivery/maintenance effort and time 3.Collaboration with competitors in multi-supplier engagements to help everyone bring their ‘A’ game

Randy Shoup10:05:32

Trying to help outsourced IT be more effective is feeling like the next iteration in the DOES journey: First not unicorns, but horses. Now not horses as we think they should be, but horses as they are.

Ben Grinnell - North Highland and DOES PC10:05:58

Most enterprises with aspirations to grow their internal IT capability are falling short of their goals, why? – Slack Poll - please vote with the emoji reaction below for the 1-2 most impacting your enterprise or reply with others: 1. It's an aspiration without a plan or concerted effort:eyes: 2. A lack of leadership alignment about what that means or the priorities:open_hands: 3. A lack of constructive collaboration and shared ownership with HR:sweat: 4. There is a lot more to growing a sustainable internal capability that the current scope:open_mouth: 5. Struggling to put a compelling proposition to the market:mega:

👀 4
📣 6
😓 3
👐 5
😮 4
Gene Kim, ITREV, Program Chair10:05:48

When @ben.grinnell said, “I hear clients say to their outsourcers, ‘I don’t want your people learning on the job.’ That is the opposite of what they should be saying.” One of the most profound things I’ve ever heard!!

👆 6
🙌 1
1
Gene Kim, ITREV, Program Chair10:05:09

“Your mess for less” era is dead.

Nigel Lester10:05:46

The 2018 State of DevOps report warns against functional outsourcing: Analysis shows that low-performing teams are 3.9 times more likely to use functional outsourcing (overall) than elite performance teams…

✔️ 1
Matt Cobby (Director of Engineering, Deloitte)10:05:23

Outsourcing is a reality though and that is why this talk is so important. How do we help these firms (often state/federal/national government) be better and, in turn, create a better society?

Nigel Lester10:05:49

Care needs to taken on how it is done, and not to create silos or internal boundaries.

Matt Cobby (Director of Engineering, Deloitte)11:05:59

Absolutely. Contracts are not kind and they don't have empathy. That is down to relationships and the human connection.

Gene Kim, ITREV, Program Chair10:05:02

Outsourcer goals: “grow margin, secure long term revenue” Client wants “I want someone who has done this 10 times before”

Bernard Voos (FedEx)10:05:21

@ben.grinnell - what challenges do you see with budget/headcount from executives when insourcing?

Gene Kim, ITREV, Program Chair10:05:23

Procurement wants multiple firms competing for work — difficult to create One Team.

👍 1
Scott Prugh (DOES Prog Committee)10:05:56

Argh!! Procurement managing IT outsourcing is a HUGE anti-pattern. We aren't buying servers and trying to get the lowest price!

☝️ 4
1
Philip Day11:05:06

Should firms be trying to buy hardware/services at the lowest price? :thinking_face: Is there a more general procurement anti-pattern to be discussed? I've seen it done badly, heavily silo'ed, where formal processes and cost-saving objectives ruled the day, lowering quality and racking up opportunity cost and overhead down the line. What should procurement functions be optimising for (quality, support, lead times, cost, etc) and what does agile procurement look like?

❤️ 2
Jim Moverley12:05:15

soo much, it often impedes velocity and capability...

Steve Pereira - Visible Value Stream Consulting13:05:48

Stream-aligned procurement? Procurement following an enabling team model and guided by ToC?

Sandeep Dabas10:05:04

Agreed to some extent - "less for mess", should be stopped @ben.grinnell

👏 1
Gene Kim, ITREV, Program Chair10:05:06

MSP has to develop workforce; “I don’t want your people learning on the job” <-- “that phrase has to be put to rest; we need everyone growing their careers.” 🤯

Quinn Daley10:05:43

how does “don’t ban learning for suppliers” gel with IR35 here in the UK? I assume there are tax implications of including suppliers in employee activities, annoyingly

Gene Kim, ITREV, Program Chair10:05:24

Balance between funding products/projects vs. community is key — power follows money. (This came up in the Dr. Westrum work in “Sidewinder” book — this balance is actually big tension in functional organizations! @ben.grinnell has finger on a very important problem.)

Chris Leeworthy (he/him)10:05:47

The idea that you can treat people like cogs that can just be ‘dropped in’ fully functional, shows a complete misunderstanding of what we all do. Even if you know a technology forwards, backwards and inside out, you will still need to learn about the existing systems and ways of doing things in that organisation before you can be effective.

☝️ 1
Gene Kim, ITREV, Program Chair10:05:54

(and is at the root of what is called the Second Toyota Paradox — that Toyota didn’t have cross-functional teams, instead they had very powerful functional silos. Much like how Google SRE is organized.)

Matt Cobby (Director of Engineering, Deloitte)10:05:08

Can we deprecate the phrase "Centre of Excellence" (CoE) and use "Centre of Enablement" instead? It sets the right tone to switch from controlling to helping those who are responsible for a line of business to achieve their goals and business value?

👍 2
Radoslaw Wiankowski10:05:20

The way I like looking at it is to have a Centre of Enablement within the Circle of Expertise/Excellence. A centre of excellence quickly becomes a bottleneck.

Phil Gadzinski Bupa11:05:41

i think the Other way around @rwiankowski. You can have centres of excellence in broad centres of enablement… focused on specific standards and practices and learning.

👍 1
Matt Cobby (Director of Engineering, Deloitte)11:05:14

Distribute power (knowledge & guardrails) to the masses @rwiankowski and they will out-invent you every day. That continual improvement is the compounding interest of getting better. Use InnerSource techniques to harvest the best pockets of innovation back in and redistribute back out, further increasing the ROI on the uplift investment.

👏 1
Matt Cobby (Director of Engineering, Deloitte)11:05:36

Somebody called it the flywheel effect.

Radoslaw Wiankowski11:05:18

eventually yes they will. But what I see, especially in larger and more complex environments is that the centre of enablement really needs to provide a lot of expertise and experience into the masses as you call them. I’m both hands down for freedom within guardrails, but many teams don’t really know what to do with that freedom and need guidance/help. Once you’ve injected enough momentum, the flywheel will keep spinning on its own and the community skyrockets with inventing cool stuff. That initial period when you need support the teams is key though, and the biggest challenge I see is how to scale the enablement.

Ben Grinnell - North Highland and DOES PC10:05:11

@bernard.voos usually enterprise headcount constraints getting in the way of the capability build, and too much focus on the individual cost rather than value

👍 1
Gene Kim, ITREV, Program Chair10:05:22

Haha. “Many will point of SODR functional outsourcing finding.” (yes.)

Christoph Hagedorn10:05:30

This session is not in the video library, is it? I can't find it.

Gene Kim, ITREV, Program Chair10:05:37

“Do you buy T&M time & materials or outcomes?”

Gene Kim, ITREV, Program Chair10:05:56

(@hagedorn video will be posted as soon it finishes airing.)

Gene Kim, ITREV, Program Chair10:05:39

“if you want people to commit their careers are, you need to be very clear about your goals, what you’re trying to build” <--- relevant to any leader

Gene Kim, ITREV, Program Chair10:05:10

“don’t ask partners to do something that’s not already in their repretoire, engagement model, etc.” 🔥

Moira Cheng10:05:23

Superb analysis and summary @ben.grinnell

👏 1
Gene Kim, ITREV, Program Chair10:05:46

“suppliers -> partners”. (interesting, right? 🔥 )

Jeff Gallimore (CTIO - Excella)10:05:11

words matter. so do the behaviors that back them up and reinforce them.

Giulio Vian, Unum10:05:52

preaching to choire

🎵 1
Gene Kim, ITREV, Program Chair10:05:50

“Where has service providers enabled growth of capabilities, and how have they been rewarded?” (we see this more on app dev, less on app maintenance.) We’ll have a track dedicated to this, to focus attention on this. Join the Birds of Feather on this topic!!! THANK YOU, @ben.grinnell!!! Amazing work!!!

👏 1
🔥 1
Gene Kim, ITREV, Program Chair10:05:26

May be of interest to you! Please join! @dhruba.chaudhuri @leena.pradhan @musthafa.s @tm.manikandan.b @tabrez.muhammad

👍 1
Leena Pradhan11:05:29

Yes, I am there. Thanks very much for recognising both our stories.

Alex Broderick-Forster, IT Revolution, Event Staff10:05:53

Sourcing in the Age of Digital Business Ben Grinnell, Managing Director and Global Lead of the Technology and Digital Service Line, North Highland 🎉 Now Available for Sharing: https://videos.itrevolution.com/watch/708991041/

👏 1
Matt Cobby (Director of Engineering, Deloitte)10:05:08

Thank you @ben.grinnell, a great talk with lots of insights.

👏 3
🔥 2
Quinn Daley10:05:34

Thanks for this insight @ben.grinnell

Scott Prugh (DOES Prog Committee)10:05:08

Great talk @ben.grinnell!! I'd love to help with this problem!!

Gene Kim, ITREV, Program Chair10:05:56

Incredible work, Ben — thank you!

❤️ 1
Radoslaw Wiankowski10:05:08

great talk! thanks!

Gene Kim, ITREV, Program Chair10:05:16

Hello, @bill.bensing (or is it @bill.bensing) — agh, not sure about how Slack does name conflicts. 🙂

Ann Perry - IT Revolution10:05:18

Let's now welcome, @wbensing, Software Factory - Managing Architect at Red Hat and co-author of the upcoming Investments Unlimited, here to present, We're Sorry, Love DevOps

Bill Bensing10:05:22

Hello all!!!

👋 4
1
Topo Pal10:05:44

"CI/CD was yesterday" - yes! @bill.bensing

Gene Kim, ITREV, Program Chair11:05:21

“Imagine you’re the CEO, and you’re at home, making pizza with family.. and you get a call from board member.”

Topo Pal11:05:39

MRIA - Matter Requiring Immediate Attention - aka Its on Fire

💣 1
Gene Kim, ITREV, Program Chair11:05:55

MRIA: “matter requiring IMMEDIATE attention” how do you get an MRIA? by not effectively responding to a bunch of MRAs: “matter requiring attention”. (Audit findings. 🙂 😆

Bill Bensing11:05:53

@tapabrata.pal #rockstar!

🙏 2
Gene Kim, ITREV, Program Chair11:05:56

“This is one of my best experiences” — I hear this so much when one gets to work on projects with kindred spirits on important problems. So true!!! Which is why I love this community so much!

❤️ 3
Gene Kim, ITREV, Program Chair11:05:59

“organizations” === “efforts that require efforts larger than can be done by one person alone”. (which is why we need to say it so often! 🙂

Gene Kim, ITREV, Program Chair11:05:47

“this is more than someone saying ‘we think you should do this’ — it’s what successful organizations have done.” (and not gotten fired. 🙂

Topo Pal11:05:14

Codification of Governance

🎯 1
🔥 2
Gene Kim, ITREV, Program Chair11:05:26

“audit toil” — so much discussed yesterday by @lucasc5 and @tod.bickley. (one of the most startling presentations I’ve ever seen.)

👍 2
Gene Kim, ITREV, Program Chair11:05:43

(toil that was okay, because “they’re only auditors”. 🙂

Jeff Gallimore (CTIO - Excella)11:05:02

it shifts the approach from using controls based on heavy documentation with manual reviews to automated controls that uses evidence as the “digital exhaust” of daily work

2
Vaidik Kapoor (Speaker) - Technology Consultant11:05:12

very cool idea to use rego/OPA for codifying compliance @wbensing! that's how you do governance at scale!

❤️ 3
Bill Bensing11:05:09

@kapoor.vaidik, we have this example in the opensource upstream

Vaidik Kapoor (Speaker) - Technology Consultant11:05:17

In my last company, this idea came to us really late so we didn't get to trying. so glad to see that someone has tried this.

Vaidik Kapoor (Speaker) - Technology Consultant11:05:59

we did something with python and jupyter notebooks for codification, continuous monitoring of controls and auditing (internal / external)

Vaidik Kapoor (Speaker) - Technology Consultant11:05:12

have to say that after automation, making changes for good became much easier

Topo Pal11:05:20

Good DevOps Practices = Better Governance

1
🎯 1
💯 1
Gene Kim, ITREV, Program Chair11:05:28

(@bill.bensing side note: Rego/OPA is written in Haskell, yes?)

Scott Prugh (DOES Prog Committee)11:05:59

Oh yeah! Re-purposing CAB!

🙌 2
Jeff Gallimore (CTIO - Excella)11:05:09

CAB ➡️ Modern Governance Platform Team 🤯

😁 1
🤯 2
Matt Cobby (Director of Engineering, Deloitte)11:05:00

Remove Blockers:white_check_mark:, Reduce fear of change , Career Growth , create advocates

Bill Bensing11:05:23

@genek, OPA does follow the functional paradigm! I have to admit, I don't know if it's written in Haskell...I'm rapidly looking that up!!

Vaidik Kapoor (Speaker) - Technology Consultant11:05:11

OPA is written in golang. I think rego is as well.

Gene Kim, ITREV, Program Chair11:05:13

https://github.com/fugue/fregot I remember hearing something like that Rego reference implementation was in Haskell, or something like that. Not urgent, but caught my attention. 😆 cc @topo.pal @stephen

Jeff Gallimore (CTIO - Excella)11:05:45

why not? traditional CABs don’t work anyway. 🤷 (don’t believe that? read Accelerate.)

Topo Pal11:05:56

CAB Members may work, the CAB does not 🙂

💯 1
Bill Bensing11:05:00

@jeff.gallimore #truth

Gene Kim, ITREV, Program Chair11:05:21

Congrats to entire team behind “Investments Unlimited” — it’s fabulous!

Leah Brown - IT Revolution11:05:22

Join some of Bill's coauthors today in a live podcast interview around modern governance!! Today at 1:50 pm BST: https://members.itrevolution.com/live/links

🎉 1
Ivan Krnic (Director of Engineering at CROZ)11:05:20

So looking forward to this! 🙂

❤️ 1
Ivan Krnic (Director of Engineering at CROZ)11:05:41

This book is amazing! We will talk more about insights from Investments Unlimited with co-authors during AMA later in the day. Please remember to register, link is here in the session description. See you all! :) https://doeseurope2022.sched.com/event/11x66/0800-devops-live-podcast-session-with-co-authors-of-investments-unlimited?iframe=no

❤️ 1
🎉 1
☝️ 1
💯 1
Leena Pradhan11:05:41

Continuous compliance in an automated manner. Great talk!

Quinn Daley11:05:44

thank you @bill.bensing - lots to think about… the road to changing to automated governance sounds painful but worth it

Alex Broderick-Forster, IT Revolution, Event Staff11:05:52

We’re Sorry, Love DevOps Bill Bensing, Software Factory - Managing Architect, Red Hat and Co-author of Investments Unlimited (https://members.itrevolution.com/free-ebooks) 🎉 Now Available for Sharing: https://videos.itrevolution.com/watch/708991065/

Bill Bensing11:05:12

@quinn.daley, welcome!!! It's well worth the pain

Gene Kim, ITREV, Program Chair11:05:17

Thank you @bill.bensing!!!

Bill Bensing11:05:19

But yes, painful it can be!

Luke Rettig - Target11:05:20

excited for the new book @wbensing !

Bill Bensing11:05:48

Thank You All!

❤️ 1
Molly Coyne (Sponsorship Director / ITREV)11:05:36

:fire:*Want to hear more about _Investments Unlimited*?_ Our final live _Ask-Me-Anything_ of the conference to do just this is ahead at 1:50 BST! You won't want to miss this LIVE AMA! Ivan Krnić, from CROZ, will talk to Topo Pal, Jason Cox, Bill Bensing, and John Rzeszotarksi, co-authors of _Investments Unlimited._ <https://us06web.zoom.us/webinar/register/WN_3son46R4TAiLLcFYaKG9cA> ALSO: Be sure to download your *free copy of the book* on the conference platform under the tab "FREE EBOOKS." <https://members.itrevolution.com/free-ebooks>

👍 3
🔖 2
🎉 2
Gene Kim, ITREV, Program Chair11:05:36

(PS: @scott.prugh @rshoup You just observed how these videos are more coupled than you’d expect — it’s surprisingly difficult to move talks around, because of dependencies that are impossible to change; all videos recorded independently, asynchronously, sequence not known in advance.)

Randy Shoup16:05:08

Coupling is a constraint in all things :-)

Slackbot11:05:09

Reminder: The action has moved to the breakouts! Join the following channels to interact with speakers live while their talks air: #ask-the-speaker-track-1 #ask-the-speaker-track-2 #ask-the-speaker-track-3 #ask-the-speaker-track-4

Slackbot11:05:12

Reminder: The action has moved to the breakouts! Join the following channels to interact with speakers live while their talks air: #ask-the-speaker-track-1 #ask-the-speaker-track-2 #ask-the-speaker-track-3 #ask-the-speaker-track-4

Slackbot16:05:26

Reminder: The final plenary sessions are starting again in 5 minutes. Start making your way back to your browser. https://devopsenterprise.slack.com/files/UATE4LJ94/F01D34MC2KS/image.png

Gene Kim, ITREV, Program Chair16:05:22

ANNOUNCEMENT: we are still waiting to get the approvals needed to air my interview with ING Bank CTO Ron van Kemenade — So we will be airing the amazing presentations from Disney (@jason.cox @brian.l.scott, and then Dr. @steve773! If we can get the necessary approvals, we’ll air Ron’s talk at the end of the day — otherwise, we will put that amazing interview in the Video Library, and we’ll send out an email.

💥 1
🎉 3
👋 2
🤞 4
Ann Perry - IT Revolution16:05:33

disney1 And NOW, we are excited to introduce the team from The Walt Disney Company – Alexi Varanko, VP, Transformation Engineering; @jason.cox, Director, Global SRE and @brian.l.scott, Sr Staff Technical Evangelist, Global SRE. They will be presenting: Disney Global SRE — Creating Digital Magic disney1

Jason Cox - Disney16:05:27

Hi everyone! 🙌

👏 2
disney1 2
👋 2
Gene Kim, ITREV, Program Chair16:05:06

Hi, @jason.cox and @brian.l.scott!!!! Jason has been with us from the very beginning!!! 🎉

Brian Scott16:05:35

👋:skin-tone-4: Hello!

Jeff Gallimore (CTIO - Excella)16:05:36

getting thanked by senior leaders all the time? just a typical day for @jason.cox 🤷

❤️ 1
Gene Kim, ITREV, Program Chair16:05:53

Was that Galaxy’s Edge ride?

Gene Kim, ITREV, Program Chair16:05:06

“technology is critical as Disney enters Direct to Consumer world — we’re in 2nd or 3rd inning” (in baseball, there are 9 innings 🙂

Luke Rettig - Target16:05:13

and now we all want to work at Disney

❤️ 2
2
disney1 1
Gene Kim, ITREV, Program Chair16:05:25

I had an outrageous time reading this massive book of bringing Marvel Cinematic Universe to the big screen — https://www.amazon.com/gp/product/1419732447/

Jason Cox - Disney16:05:31

Love that video because it is full of the people we get to work with. Diane is our CIO - my boss ^ 2. She is awesome. Hope to have her join us one of these days. 🙂 Nick is our CTO at Animation, great partner. Same with Aaron (ESPN), Mike and Joe (Streaming).

🎉 1
Margueritte Kim (CEO, IT Revolution, she/her)16:05:24

Grogu! Doesn’t get more magical than him.

❤️ 3
Jason Cox - Disney16:05:44

"Optimize for Innovation" is really a lot about story creation and story telling.

❤️ 2
Gene Kim, ITREV, Program Chair16:05:54

Yes! (Where did you find an actor that small who could play him? (her?))

Gene Kim, ITREV, Program Chair16:05:06

Great book! “Ride of a LIfetime!”

💯 1
Brian Scott16:05:40

Must of read that book a few times now

Jason Cox - Disney16:05:18

Side note on SRE - so much great content from many of this community on SRE at DOES - check it out: Site Reliability Engineering (SRE) Playlist - https://itrevolution.com/jason-cox-sre-playlist/

❤️ 4
💯 2
Gene Kim, ITREV, Program Chair16:05:52

“we went from 10s of servers, to 100s.. to 1000s… to 10,000s”

Jason Cox - Disney16:05:09

There are over 100 servers just for one attraction (Toy Story Mania) ... good example of digital explosion of the business.

😱 1
Topo pal16:05:15

Tickets are must - lol

Luke Rettig - Target16:05:16

“the internet group” classic - those peeps over there doing interweb things

Gene Kim, ITREV, Program Chair16:05:29

“there’s so much you need to know” — and that was back when life was much, much simpler… all you needed to know was Linux, Apache, MySQL, Perl, maybe nginx.

Brian Scott16:05:42

Now its all the container and serverless tech

Gene Kim, ITREV, Program Chair16:05:48

“everything required a ticket — even going to bathroom, you’d need a ticket” 😆

2
Topo pal16:05:56

Sleepy server is no good

Fokko V.16:05:27

Grumpy probably neither 😂

Jeff Gallimore (CTIO - Excella)16:05:28

“grumpy threw errors, sleepy had lots of latency, bashful disappeared from the network for days at a time.” :rolling_on_the_floor_laughing:

Gene Kim, ITREV, Program Chair16:05:35

At Sun in 1986, I thought it’d be cute to name my Sun 4 workstation “panic” — dumb idea. Every log message gave me a heart attack.

Alexander Fritz16:05:33

We used animals as names for our test platforms and organs for the services in it at a project in the past. This lead to funny alarm messages at times: "Bison has a problem with heart - please care!" "something is wrong with the brain of Elephant - please care!" 🙂

😂 1
Virginia Laurenzano NSA/MARFORCYBER16:05:36

Jason always has the best slides

5
Gene Kim, ITREV, Program Chair16:05:53

I think @jason.cox idea of “Embedded” changed Ops for so many organizations — I remember how shocking it was when he first presented on this in 2016, and how rapidly it was adopted across industry!

Brian Scott16:05:32

I really love the embedded model, really helps us understand our business and how to bring Value to our guests

Gene Kim, ITREV, Program Chair16:05:51

“Platforms! And that’s we made a massive bet on… Mesos!” (quoting a friend)

Jason Cox - Disney16:05:06

Yes! ^^ Full Stack Teams not Engineers - Jason Cox, Christian Posta, Cornelia Davis, Dominica DeGrandis, Jim Stoneham, Thomas A. Limoncelli. (2019). Full Stack Teams, Not Engineers. *P*ortland, Oregon: IT Revolution. https://itrevolution.com/book/full-stack-teams-not-engineers/

Brian Scott16:05:50

"Treating Operations like Software" - Powerful

❤️ 1
Jeff Gallimore (CTIO - Excella)16:05:43

“as SREs, we are technology sherpas”

❤️ 4
Jeff Gallimore (CTIO - Excella)16:05:26

wonderful at helping people get to the top of big mountains

Gene Kim, ITREV, Program Chair16:05:23

I think one of the reasons people thank @jason.cox and team so much: having platform engineers available to help, is so helpful. Because, I found learning docker pretty difficult, and kubernetes was super super difficult. Better to just ask your buddy to help!

Jason Cox - Disney16:05:19

Using the first-person way to describe your job really helped us shape our identity and how we optimize our team/brand/efforts for success within Disney.

❤️ 1
🤯 1
Leena Pradhan16:05:25

Great way to describe the job in first person. Imbibes more ownership.

💯 2
Brian Scott16:05:40

"Define who you are through that first-person"

Gene Kim, ITREV, Program Chair16:05:46

@jason.cox engagement model:

Gene Kim, ITREV, Program Chair16:05:16

Super interesting: “Our role from Global SRE — help those SREs connect with other SREs”

👍 1
Gene Kim, ITREV, Program Chair16:05:28

(share the pockets of awesome; cross-pollination of cool)

Brian Scott16:05:03

Going to quote myself - "Great pollinators of cool"

😂 1
❤️ 2
🐝 1
Shikha Srivastava16:05:08

I agree with full stack team. Full stack engineer is not a thing.

💯 1
Vlad Ukis04:05:21

How full stack can an engineer become? What are the limits?

Gene Kim, ITREV, Program Chair16:05:55

Flexible Support Assignments: months to years

👆 2
Boris Martinovic16:05:14

Knowing the business side as an engineer helps so much.

Gene Kim, ITREV, Program Chair16:05:44

highest level: Dedicated Embedded Team: embedded within the business unit, and now you’re able to bring to SRE Global (mothership) needs from edges of organization (where the money is at!)

👍 2
Jeff Gallimore (CTIO - Excella)16:05:23

“we ship happiness” 🙌

Shikha Srivastava16:05:28

yes, each service knows the ins and out. Pretending for a global SRE to manage platform+ srvice both does not work

Brian Scott16:05:34

"Better, Faster, Safer & Happier"

Radoslaw Wiankowski16:05:47

Better, Faster, Safer, Happier sounds very much like BVSSH ❤️

👍 1
1
Jason Cox - Disney16:05:06

Remember when I first talked to @jonathansmart1 about this, he would say... Sooner Safer Happier - Jonathan Smart - https://itrevolution.com/sooner-safer-happier/ 100%

👍 1
Gene Kim, ITREV, Program Chair16:05:09

Better, Faster, Safer, Happier. (Loving how these words are being used so widely — go @jonathansmart1!)

🎉 1
👍 1
Gene Kim, ITREV, Program Chair16:05:33

Hi, @brian.l.scott!!! Congrats on your new role!!!

Brian Scott16:05:51

Thanks @genek!

Gene Kim, ITREV, Program Chair16:05:46

“We bring security folks to help everyone learn how to make their things more secure!” 🎉

Ann Marie Fred - Red Hat16:05:41

For some reason I didn’t expect Continuous Delivery to Smugglers’ Run.

🤯 1
Gene Kim, ITREV, Program Chair16:05:54

How utterly awesome — so cool, @ann.marie.99!!!

Jason Cox - Disney16:05:51

Imagine... sitting in ride vehicle, see an issue, pull out your laptop to pull down the code, fix it, push it to the repo and see the CI/CD deliver and deploy the new code on the next cycle. Fast feedback = better products = more happiness. ; )

💥 1
😎 3
🤯 1
Brian Scott16:05:09

It's pretty incredible!

Gene Kim, ITREV, Program Chair16:05:20

I love these stories of elevating productivity of some of these epic / legendary groups, @brian.l.scott!!!

❤️ 1
Gene Kim, ITREV, Program Chair16:05:43

“Excel spreadsheets” “opening tickets” 🎉

😥 1
Brian Scott16:05:51

I love being able to tell stories of our SRE Teams!

❤️ 3
Gene Kim, ITREV, Program Chair16:05:09

“I’m embedded in Industrial Light and Magic.” (mic drop) @brian.l.scott

Jim Moverley16:05:05

what a presentation.. where do i sign up 😄

👆 1
Jim Moverley16:05:40

ahhh i was expecting a meeting at the "cantina" 😄

1
🥃 1
Ann Perry - IT Revolution16:05:39

And we're EXCITED to present the next session, An Interview with Ron van Kemenade, CTO of ING Bank

🎉 2
👏 1
Gene Kim, ITREV, Program Chair16:05:45

THANK YOU @jason.cox @brian.l.scott!!!!

👏 4
2
1
Brian Scott16:05:53

Thank you all!

Shikha Srivastava16:05:09

great presentation. SRE stories are most inspiring

❤️ 3
👏 1
Adrian Hunt16:05:40

Will the presentation be available to download and re-watch please?

Alex Broderick-Forster, IT Revolution, Event Staff16:05:26

It will be in the video library soon! I’ll post the link.

👍 2
🎉 1
Jason Cox - Disney16:05:48

Yes via the members library. Right @annp and @alex?

Werner Pantke16:05:26

Including the slides? (couldn't find them on github)

Jason Cox - Disney16:05:30

@werner.pantke send me your contact info.

👍 1
Gene Kim, ITREV, Program Chair16:05:00

“only reason it wasn’t worse was that you couldn’t rate an app with less than one star” 😆 I loved that joke. Went into Unicorn Project.

😂 3
Jeff Gallimore (CTIO - Excella)16:05:46

a mainframe is difficult to power down? confirm four times, please!

😂 4
Gene Kim, ITREV, Program Chair16:05:35

Fans do not turn off for several hours, even after you power it off — because there’s very few scenarios where you wouldn’t turn it back on!

Akis Sklavounakis16:05:15

Nobody designs a mainframe for a graceful exit...

Gene Kim, ITREV, Program Chair16:05:00

the problem wasn’t the mainframe — “the problem was the 250 applications were running on it; payment and many of our product engines, like mortgages, consumer loans, business loans, a lot of business processes related to those products, our customer and agreements, uh, administration was running on. Part of the financial accounting was running on it. So consider it literally the core of the bank.”

1
Gene Kim, ITREV, Program Chair16:05:35

“Mainframes are one of the most long living and fully supported and very much alive platforms in the world still I’m supporting a lot of enterprises, governments.”

❤️ 2
Gene Kim, ITREV, Program Chair16:05:23

“It’s harder to find great programmers in ALGOL, PL/1, COBOL anymore. The ones we have are leaving the bank” — retiring.

Gene Kim, ITREV, Program Chair16:05:52

This apparently was a race against time — the knowledge pool was shrinking the whole time over the 7 years of this project!!

Nick Eggleston16:05:04

How easy is it to find staff for these mainframes now? And projection for the future...

Jeff Gallimore (CTIO - Excella)16:05:53

harder and harder every day…

Jeff Gallimore (CTIO - Excella)16:05:22

that part of the workforce isn’t being replenished

Gene Kim, ITREV, Program Chair16:05:25

“success of this required trusting your engineers; and endurance to persevere over such a long conviction. and absolute conviction that there’s value in doing this”

Gene Kim, ITREV, Program Chair16:05:19

“No one wants to sit on pile of legacy that no one understands anymore; it’s not easy, but a sheer necessity” (!!)

Jeff Gallimore (CTIO - Excella)16:05:56

i don’t think he means “legacy” in the positive connotation :rolling_on_the_floor_laughing:

Margueritte Kim (CEO, IT Revolution, she/her)17:05:09

I’m pretty sure we have a pile of this legacy in our garage.

Virginia Laurenzano NSA/MARFORCYBER17:05:29

totally where my mind went, too. stealing this quote

Jeff Gallimore (CTIO - Excella)16:05:25

“no one wants to sit on a pile of legacy that no one understands anymore” :rolling_on_the_floor_laughing:

3
Gene Kim, ITREV, Program Chair17:05:44

“when they heard what we wanted to do, located, they literally said, ‘Ron, you’re crazy.’”

Gene Kim, ITREV, Program Chair17:05:26

“Along the way, we encountered many… complications. People retiring faster than expected, funding / priority issues.” (I tried not to laugh, but the story is so good.)

Gene Kim, ITREV, Program Chair17:05:49

Imagine a journey when the people required to succeed kept disappearing (retiring, which they deserved!!!)

😰 1
Gene Kim, ITREV, Program Chair17:05:25

the courage it takes to “let’s go; if we have problems, we’ll fix it on the other side — we must have enough bravery to keep going.”

Jeff Gallimore (CTIO - Excella)17:05:57

another common element of these journeys: courage.

👏 3
1
Gene Kim, ITREV, Program Chair17:05:29

yes! very much evokes the image of a person climbing a mountain, with party shrinking as climb goes on. @jeff.gallimore

Gene Kim, ITREV, Program Chair17:05:54

“I truly understand your concerns; but I take responsibility, and now we’ve got to go.”

💯 2
Gene Kim, ITREV, Program Chair17:05:15

“if you don’t take the last step, you can always turn back” (oooh…). “we must leave this world behind.”

🙌 2
Randy Shoup17:05:49

Burn the ships

🎯 1
Randy Shoup17:05:43

"Now I can actually retire"

Gene Kim, ITREV, Program Chair17:05:45

“he was standing next to the guy who was there in 1980 when they wheeled in the mainframe; and he was there, powering it off. he was not sad; he was proud; he was the last man to turn it off; his account to power it off”

❤️ 5
🔌 2
Gene Kim, ITREV, Program Chair17:05:51

“now I can actually retire”

Jeff Gallimore (CTIO - Excella)17:05:15

i’m not crying! you’re crying!

🔥 1
Siddharth, NatWest Group, DevOps CoE (he/him)17:05:36

how many have worked on COBOL here 🙌. Raise your hand. I did 😄

Manikandan Balasubramanian17:05:30

I worked om COBOL 😀

🙏 1
Gene Kim, ITREV, Program Chair17:05:01

went from 2 day international payment settlement to 15 minutes; it wasn’t the code — it was the way all those applications were chained together!!!!

🚀 4
Jeff Gallimore (CTIO - Excella)17:05:38

architecture! dependencies!

Gene Kim, ITREV, Program Chair17:05:30

“Wow, what happened. You processed my dollar payment in 15 minutes!” (So great!!!)

Gene Kim, ITREV, Program Chair17:05:23

“No one wants to have PL/I on their CV” (anymore)

Gene Kim, ITREV, Program Chair17:05:30

“we scheduled it to 5 years; it took 7 years” — but they will benefit from this project every year forever.

❤️ 1
Philip Day17:05:55

5 -> 7 is good!!

Gene Kim, ITREV, Program Chair17:05:28

Totally!!! the image of team continually shrinking is for me utterly epic…

Neil Waller17:05:33

@jason.cox will the slides be made available for your presentation? Some useful points I would like to share with my team

Alex Broderick-Forster, IT Revolution, Event Staff17:05:19

Disney Global SRE — Creating Digital Magic Alexi Varanko, VP, Transformation Engineering, The Walt Disney Company Jason Cox, Director, Global SRE, The Walt Disney Company Brian Scott, Sr Staff Technical Evangelist, Global SRE, The Walt Disney Company 🎉 Now Available for Sharing: https://videos.itrevolution.com/watch/709121287/ Slides: https://github.com/devopsenterprise/2022-virtual-europe/blob/main/Jason%20Cox%20-%20Brian%20Scott%20-%20Alexi%20Varanko%20-%20DOES%202022.pdf

Steve Spear17:05:24

@jason.cox @brian.l.scott You all rocked.

❤️ 1
Randy Shoup17:05:45

These replatforming / rearchitecture efforts require so much courage and conviction because you only get the benefits at the end. It's all or nothing. And while you are in the transition period between architectures things are worse -- you have 2 systems to keep in sync, in addition to all of the horrors of the old system.

🔥 2
❤️ 2
🎯 2
Gene Kim, ITREV, Program Chair17:05:24

“must have conviction that there will be benefits to customers and firm; the risk was that every year, critical knowledge is leaving the firm. every CIO must be worried about this — a real risk”

Gene Kim, ITREV, Program Chair17:05:58

“every CIO who takes responsibility must think about [the risk of] knowledge legacy, because it will sneak up and haunt you.”

Gene Kim, ITREV, Program Chair17:05:12

@rshoup @rshoup This bit is so interesting… wait for it…

Scott Prugh (DOES Prog Committee)17:05:32

There are great parallels here with the TUI story from @pieter.jordaan around risk. Many times doing nothing is WAY riskier than forging the new path!

❤️ 1
Scott Prugh (DOES Prog Committee)17:05:07

There is almost nothing riskier then your talent evaporating away...

Steve Spear17:05:07

Agile spagetti eating!

Jeff Gallimore (CTIO - Excella)17:05:10

what if your plate of spaghetti looks like this?

Gene Kim, ITREV, Program Chair17:05:15

I.e., if architecture is sufficiently homogeneous and no partitions, it actually doesn’t matter where you start. It’s all the same, so it doesn’t matter where you start,” COME ON, that’s amazing, right?!?

🙌 1
Steve Spear17:05:51

If the system is heterogenous but modular, where you start matters little too.

💡 1
Gene Kim, ITREV, Program Chair17:05:18

Can you confirm with an emoji that you appreciate the amazingness of that story? I want to share this with Ron! THANK YOU!!! ❤️

👏 13
❤️ 5
🔥 3
👍 1
🙌 4
🍕 1
Ann Perry - IT Revolution17:05:48

:unicorn_face: And now, bringing us to a close, our very own @genek and @steve773, who will teach us, What is Architecture, and Why It Matters bowtie

👏 2
Slackbot17:05:02

Reminder: Please submit your feedback for the talks you attended. It’s so valuable for us and the speakers. And after all, feedback is a gift and sharing is caring! Enter your feedback for those talks here: https://members.itrevolution.com/live/schedule https://devopsenterprise.slack.com/files/UATE4LJ94/F03E48CJRF1/image.png

Gene Kim, ITREV, Program Chair17:05:32

ARCHITECTURE!! Very pertinent given that last talk!!! Hi Dr. @steve773!!

❤️ 1
Margueritte Kim (CEO, IT Revolution, she/her)17:05:52

Yes, @genek, your book will come out THIS YEAR.

🙌 1
🔥 1
Jim Moverley17:05:15

this year Gene this year!!!!! 😄

Steve Spear17:05:16

Bring on the questions and comments folks 🥊

👏 1
Gene Kim, ITREV, Program Chair17:05:24

(Incidentally, when I took that class in 2014, I discovered in Boston that I forgot to pack pants. 😱 )

1
😄 1
Margueritte Kim (CEO, IT Revolution, she/her)17:05:01

If you find me at DOES Las Vegas in October, I will tell you this story. It is a classic.

👍 1
Randy Shoup17:05:12

Other than the ones you were wearing on the plane? Uh ... you were wearing pants on the plane, right? ...

Gene Kim, ITREV, Program Chair17:05:33

Yes, I can confirm I did have pants on the plane. 😆

Timbo17:05:49

You know that pants means a different thing in the UK, right? :rolling_on_the_floor_laughing:

Gene Kim, ITREV, Program Chair17:05:13

Help! I don’t! Please enlighten!

Timbo17:05:59

pants are usually underwear for us

Leah Brown - IT Revolution17:05:50

2023 everyone. Book comes 2023. Just in time for the 10th Anniversary of IT Revolution !!!

😂 2
👏 1
1
❤️ 2
Gene Kim, ITREV, Program Chair17:05:54

“First we design our buildings, and then they design us” — Winston Churchill

🤯 3
Steve Spear17:05:38

We behave on them, and then they behave on us…how we architect our engineered systems determines how they architect us.

Jim Moverley17:05:05

Function over form!

Gene Kim, ITREV, Program Chair17:05:40

Fascinating. With what @steve773 said: the problem that Ron van Kemenade was trying to solve was the calcified pathways of work, caused by the 250+ mainframe applications that they could no longer change. Change the applications, and suddenly you can go from 2 day settlement times to 15m. Not the applications, it’s the way they’re arranged.

Steve Spear17:05:46

@jim@james.moverley Structure determines dynamics, form determines function.

❤️ 1
Gene Kim, ITREV, Program Chair17:05:14

Very relevant to the @mik talk yesterday!

❤️ 1
Mik Kersten (Project to Product, Tasktop)17:05:29

Hello! Sorry, have a personal thing going on so have not been able to keep up :/

Gene Kim, ITREV, Program Chair17:05:57

Hi, Mik! So glad you’re here!

Mik Kersten (Project to Product, Tasktop)17:05:34

Pains me to have been away this long! never again 🙂 Catching up on threads…

Steve Spear17:05:16

Chapter 4 about Alcoa achieving 200X delta on Safety. Chapter 5 about huge deltas on R&D productivity and quality.

❤️ 2
🔖 1
Steve Spear17:05:40

Everyone starts at a very low low level of competency and capability.

Jeff Gallimore (CTIO - Excella)17:05:53

@steve773 quality journey from worst to first started with the toyota pet toyopet, right?

Steve Spear17:05:55

Some thoughts on Toyota and why it found it so important and differentiating to institutionalizing it’s management system…https://conta.cc/38lVhFI

Gene Kim, ITREV, Program Chair17:05:13

So interesting to consider the opposite of this: “people trapped in a system that constrains their full problem solving potential” (the common battle told in almost all DevOps Enterprise stories!)

Steve Spear17:05:51

@jeff.gallimore yes. 1958. Toyopet. Terrible car and terrible production system and terrible labor managment relations. Worst in the world. Within 20 years, best in the world.

🙌 1
Gene Kim, ITREV, Program Chair17:05:35

Comparing the great orgs from the horrible orgs — you can isolate the dependent variable to one thing… the management system, which yields the structure and architecture of which people are working within.

Gene Kim, ITREV, Program Chair17:05:13

“Fragile vs. Agile” 🙂

Gene Kim, ITREV, Program Chair17:05:50

PS: it’s been so super fun to work on a project with Trent Hone, whose work @steve773 has been citing for 5+ years. 🎉

Steve Spear17:05:45

Integrated systems: Inflexible in Dev and Fragile in Ops Modular systems: Flexible in Dev and Resilient in Ops

💡 1
Gene Kim, ITREV, Program Chair17:05:46

“we were reliant upon 250 applications that we could no longer change” (Right?!?)

Jeff Gallimore (CTIO - Excella)17:05:38

“modular and nested” that sounded like an important point

🔥 1
Gene Kim, ITREV, Program Chair17:05:14

This is a view of the Toyota Production System that I finally understood through this lens — 3500 Andon cord pulls/day is only possible if you have a modular / nested structure. Otherwise, you’d shut the whole plant down every time. Like what happens in the contemporary GM plant!

🤯 1
Gene Kim, ITREV, Program Chair17:05:58

This theme has shown up so many times today: @ben.grinnell said: “every major program had 7+ sourcing partner”; Ron vK mentioned something similar at ING 2010, where nothing could get done.

Randy Shoup17:05:58

This is a tremendous framing. So powerful. Strong parallels to Dave Farley's new book Modern Software Development 🙂

🔖 3
Vlad Ukis05:05:14

Dave pushes Engineering these days

piotr.papros17:05:03

this is so good, i will rewatch it few more times 🙂 dr Spear , i have just become a fun of your work !

🙌 1
Steve Spear17:05:13

@jeff.gallimore Modular + nested --> mean we can express our creativity locally and in parallel and get to flexible Dev and resilient Ops.

🎯 1
Steve Spear17:05:28

@piotr Thank you!

Steve Spear17:05:52

@genek little that’s complementary about a GM plant…certainly not the GM management…ugghh.

Gene Kim, ITREV, Program Chair17:05:39

I heard from a friend: “we are all coupled together, because of the technology platform we run on — in order to get anything urgent done, you see these signals like pebbles on a pond, as nodes try to signal to each other. Only CEO intervention can get the system oriented onto one urgent things, usually a customer issue.” Coupling.

Steve Spear17:05:01

@genek “some systems constrain or even extinguish the creativity of people in the system…” That’s what despots do!

Steve Spear17:05:45

@genek nested modularity…necessary for having a well functioning liberal democracy?

🔥 1
1
Jim Moverley17:05:06

As an architect : it does become clear how the design affects flow - this can also be said true of the low-level software architecture of components and how developers can interact. What fascinates me is the "reverse Conway manoeuvre" and the impact of the architecture on your enterprise organisation!

👍 1
💯 1
Radoslaw Wiankowski17:05:12

This should be fundamental teaching for everyone in the industry. Across all roles. No exceptions.

Giulio Vian, Unum17:05:46

API were a Microsoft mantra for all their products

Steve Spear17:05:18

@rwiankowski Thanks! We’re ready to take our show on the road. PRODUCT PLUG! We’re bringing my exec ed course at MIT Sloan on line by this Summer, in which we build on the idea of simplified, standardized, stabilized, and synchronized.

❤️ 3
👀 1
Radoslaw Wiankowski17:05:25

That’s amazing. Do you have a link/ can you share any resources? Learning at MIT Sloan is a huge dream of mine.

Virginia Laurenzano NSA/MARFORCYBER17:05:18

this afternoon has a spaghetti theme

😂 2
🍝 1
Steve Spear17:05:49

To @james.moverley’s point…all depends on the direction of design. Start with your silos and see what you design, or start with the design objective of nested modular, etc. and see what silos are needed. This is a huge idea in the idea of EMERGENT DESIGN (architecture). Initially, the profile of problems that have to be solved in the baseplate engineered system determine who has to talk about what with whom Eventually, when the engineering system’s structure stabilizes, so does the organizations. The problem being, both calcify…rarely does a change in the problem profile result in a change in the organization’s architecture.

👏 1
Jim Moverley17:05:46

thanks for this awesome reply @steve773!!

Steve Spear18:05:43

@james.moverley a case study of working across the silos

Jim Moverley14:05:31

Thankyou so much for this Dr Spear! amazing input as always!

Nick Eggleston20:05:29

More interesting things to add to the reading list. Thank you @steve773!!

Jeff Gallimore (CTIO - Excella)17:05:25

order of magnitude improvement in four years

Gene Kim, ITREV, Program Chair17:05:06

PS: here’s the original picture:

Gene Kim, ITREV, Program Chair17:05:21

blood curdling, no?

👹 1
🙀 3
Gene Kim, ITREV, Program Chair17:05:12

I can’t find the picture of The Shining, that looks just like this!!!

Steve Spear17:05:13

…early next year…

😂 1
Steve Spear17:05:37

late 2022. I told you, when can we lock him a room with one keyboard, one monitor and no internet other than to our google drive?

Jeff Gallimore (CTIO - Excella)17:05:31

1. unity of command/effort 2. enable decentralized execution

Topo pal17:05:35

Dopes at the top -> dopes everywhere !

Gene Kim, ITREV, Program Chair17:05:53

“If you have a dope at the top, you will have, or soon will have, dopes all the way down.” Jack Rabinow Rule #23 of Leadership

Gene Kim, ITREV, Program Chair17:05:11

The Sociotechnical Maestro • High energy • High standards • Great in the large • Great in the small (so they can ask good questions) • Loves walking the floor

❤️ 3
🙌 2
1
🎯 2
Steve Spear17:05:15

All attributes necessary to see and solve for problems in the ‘social circuitry’

Jim Moverley17:05:32

Amazon have great hiring technique that involves a "bar raiser"

Gene Kim, ITREV, Program Chair17:05:15

I still get goosebumps when we talk about the management system and structure/architecture as being the dominant variable of performance. And to what extent the dynamics give us fast, forceful feedback, which is what allows for learning.

🎉 1
👏 1
Randy Shoup17:05:35

Tremendous, thanks!!

Steve Spear17:05:50

BOOK BOOK BOOK BOOK BOOK. Let’s get it done.

❤️ 4
📖 2
🎉 1
Luke Rettig - Target17:05:52

book in October… holding you guys to that

Jim Moverley17:05:00

awesome presentation @genek @steve773

👏 1
Steve Spear17:05:08

@james.moverley Thanks!

Topo pal17:05:20

Amazing talk!

Gene Kim, ITREV, Program Chair17:05:21

WOOOOOOO!!!! We made it!!!

🎉 4
Boris Martinovic17:05:23

Great presentation @genek @steve773

Leena Pradhan17:05:45

Thanks for the great talk. 👍:skin-tone-2:

Steve Spear17:05:46

@tapabrata.pal Stay in touch. Please check out the links that are above.

❤️ 1
Manikandan Balasubramanian17:05:49

Great presentation @genek, @steve773

Jim Moverley17:05:54

Another fantastic DOES!!!!!! \o/

Steve Spear17:05:00

Thanks @leena.pradhan

Gene Kim, ITREV, Program Chair17:05:00

Was this best DevOps Enterprise Summit you’ve attended? Please put emoji! ❤️

❤️ 15
👏 2
😍 2
gratitude-thank-you 1
🎉 3
piotr.papros17:05:58

my 1st, still i'm extreemly impressed, and i am returning for sure !

Steve Spear17:05:47

@boris.martinovic Thank you!

Alex Broderick-Forster, IT Revolution, Event Staff17:05:50

What is Architecture, and Why It Matters Gene Kim, Program Chair, Author & Researcher, IT Revolution Dr. Steven Spear, Author, HVE LLC 🎉 Now Available for Sharing: https://videos.itrevolution.com/watch/709121311/

Randy Shoup17:05:13

No slides available? Boo!

1
Gene Kim, ITREV, Program Chair20:05:42

I'll get slides together. It may take awhile but I can send mine to you in an email!

Fokko V.17:05:52

Thanks Programming Committee for this great summit! Hope to see everyone next year in person... 🙂

👍 4
Topo pal17:05:16

DNS error !!! :rolling_on_the_floor_laughing:

Steve Spear17:05:29

@genek @annp @erin I believe it’s Miller Time for you all… When it’s time to relax, One thing stands clear, Beer after beer, If you’ve got the time, We’ve got the beer, MIller beer…

🍻 4
🍾 2
Nick Eggleston17:05:43

Gather Bar!

🍻 2
Gene Kim, ITREV, Program Chair17:05:50

Thank you to my boss, @mvk842!!!!

👏 6
🔥 6
❤️ 3
Jim Moverley17:05:09

Yes : do join us at the gather bar!! : https://itrevolution.com/gather

🍻 3
🙌 3
❤️ 2
Leena Pradhan17:05:33

To the entire program team, @genek , @annp and the wonderful speakers - we had a great summit, great learning, great collaboration! Thanks a lot! Awesome!! 👏:skin-tone-2:👍:skin-tone-2:

🎉 2
👍 2
❤️ 6
🔥 2
Gene Kim, ITREV, Program Chair17:05:39

OH YES!!! See you at DevOps Enterprise Summit Las Vegas! It’s going to be super special!!!! ❤️

👏 12
❤️ 10
🎉 6
🤯 2
1
1
Gene Kim, ITREV, Program Chair17:05:55

It’s awesome that all talks are available here! https://videos.itrevolution.com/

gratitude-thank-you 2
3
thankyou 1
Mitesh (DOES Event Staff / Engineer at Gaiwan)17:05:29

Thank you to all the speakers, and everyone else who made it possible! This was an amazing event!!! 💚

❤️ 2
🔥 3
💙 1
Jim Moverley17:05:36

The videos are invaluable!!

🎉 5
Gene Kim, ITREV, Program Chair17:05:42

I use the library so often — the fact that transcripts are there and easily searchable is so great!

❤️ 2
Jim Moverley17:05:02

Has been amazing to show people evidence!

Randy Shoup17:05:56

How do we search the transcripts? Are you thinking Cmd-f on the page?

Gene Kim, ITREV, Program Chair17:05:25

Sorry! It’s running on my laptop — I’m working on getting it available. @mitesh Let’s talk next week! 🙂

❤️ 1
Gene Kim, ITREV, Program Chair17:05:38

It’s in XTDB and Lucene!!! 🙂

Gene Kim, ITREV, Program Chair17:05:23

Randy, it’s effing awesome. Searching entire 900+ library of talks for specific words!!! I want to make word cloud!

❤️ 1
Jason Cox - Disney17:05:36

Thanks @jeff.gallimore!!! 👏

❤️ 2
Steve Spear17:05:45

@mvk842 Live in Vegas in October…Gene’ll be signed drafts of the manuscript?

1
🎉 1
Margueritte Kim (CEO, IT Revolution, she/her)17:05:20

So subtle, Dr. Spear! I have not forgotten about your text request. It is my next order of business!

Gene Kim, ITREV, Program Chair17:05:51

Thanks @jeff.gallimore!!!

thankyou 3
Steve Spear17:05:02

@genek Nice job, Gene!

👏 1
Gene Kim, ITREV, Program Chair17:05:34

Sh!t. Now we gotta write a book, Steve. “laugh

Steve Spear17:05:37

Like…duuuuhhhhh.

👏 1
Topo pal17:05:11

👏 👏 👏

Ian Richards17:05:21

How long did you say access to the video library is available?

Ian Richards17:05:03

The after it's $350 for an individual licence

Manikandan Balasubramanian17:05:30

Thanks @genek and the entire Program team for such a wonderful conference ❤️

👏 3
❤️ 5
1
Jim Moverley17:05:59

👏👏 well done @itrev team for another fantastic summit!!!!

❤️ 3
👏 3
Gleb Andreev17:05:10

Thanks to the whole ITREV team for a great event!

1
👏 2
Musthafa Soukathali17:05:17

Usually, conferences are overrated. But this is exceptionally g'rate'!!

😂 1
Balázs Cziráky17:05:53

Thank you so much for this experience. This was my first time and will be with you next time for sure... 🙂 👏❤️

❤️ 1
Anitha Iniyavan17:05:50

This is my first ever virtual conference that I attended in my career, what a great experience and I can’t imagine the amount of effort gone in to conduct it virtually. Thank every one of you who put this together. I am already looking forward to implementing the take aways from the last 3 days. Congratulations to @genek and team for conducting yet another successful event. 🙏👏

❤️ 9
🎉 1
Jim Moverley18:05:40

Bar is open in gather! 😄

Philip Day18:05:58

🍻 ongoing