Fork me on GitHub

Reminder: The final day is starting now – opening remarks and then plenary talks! Join the conversation in #ask-the-speaker-plenary.

Marc Price - Nationwide Building Society (Speaker)16:10:37

Hi All, Looking forward to chatting with you all as we share our journey so far coming up next! We hope you can make it!

👍 2
Zsolt Berend, Co-Author - Sooner Safer Happier16:10:07

+1 hi all, looking forward to chatting

❤️ 2
Ann Perry - IT Revolution16:10:18

⚡ Let's get ready to welcome @marc.price and @zsolt.berend, here to present Nationwide Building Society: Measure for Learning ⚡

👍 2
Ganga Narayanan16:10:49

Inspiring story! Congratulations on being a finalist at the Digital Transformation Project award!

🙌 2
🙏 1
Glenn Wilson, Author of DevSecOps16:10:18

This anti pattern to pattern slide is awesome!

🙏 1
🙌 1
Ganga Narayanan16:10:26

Weaponised (and game-ized) metrics.. This is all-too familiar. Same story around the world!

❤️ 1
👍 3
Andy Nelson16:10:23

@marc.price @zsolt.berend when do you start the clock for lead time? At idea creation? Or at time work is allocated to start?

👍 1
Zsolt Berend, Co-Author - Sooner Safer Happier16:10:26

Hi Andy, the clock starts when work started (the first time when a ticket moves from 'backlog' state to any in progress state until the chg implemented in prod

👍 1
Zsolt Berend, Co-Author - Sooner Safer Happier16:10:15

#days from work started to released to prod

Zsolt Berend, Co-Author - Sooner Safer Happier16:10:39

and release lead time is the max of all ticket lead times within the release

Andy Nelson16:10:34

do you measure any of the up front portfolio planning or product discovery work? i.e. how the larger level asks get broken down into the given set of stories?

Meghan Glass - PrdMgr Best Buy16:10:35

This slide is showing "issue created date" not "status" - could you talk more on that?

Zsolt Berend, Co-Author - Sooner Safer Happier16:10:26

@isuanelson we measure alignment - golden thread

Zsolt Berend, Co-Author - Sooner Safer Happier16:10:43

%stories that linked all the way up to strategic objectives

Marc Price - Nationwide Building Society (Speaker)16:10:46 we look to see that when an issue is created, does it instantly link into the golden thread alignment to strategy or not

Meghan Glass - PrdMgr Best Buy16:10:12

so golden thread = ideation to backlog and lead time = progress start to release?

Marc Price - Nationwide Building Society (Speaker)16:10:58

golden thread is alignment from stories all the way up to strategic outcomes

Meghan Glass - PrdMgr Best Buy16:10:20

so current measures are: does the work align and then the lead time for the work from start to release, but no "lead time" calculations for idea to backlog/work start at this stage

Marc Price - Nationwide Building Society (Speaker)16:10:09, we do have a lead time measure starting from the time an issue is created, but it depends of the persona for usage. For example a support team working on incidents might want to start their lead time from the time of creation, so they can choose that metric also

Glenn Wilson, Author of DevSecOps16:10:26

What data are you using to measure happiness?

Marc Price - Nationwide Building Society (Speaker)16:10:16

Hi Glenn, We created a bespoke survey that we send out to the teams and followed up a few months later to track how happiness has changed over time.

👍 1
Ganga Narayanan16:10:40

"None of these should be measured at the expense of the other." Truth!

👍 1
❤️ 2
John Ediger16:10:55

What is the "golden thread"?

👍 1
Zsolt Berend, Co-Author - Sooner Safer Happier16:10:33

it is alignment from stories all the way up to strategic outcomes

John Ediger16:10:27

I see. How is that measured?

Zsolt Berend, Co-Author - Sooner Safer Happier16:10:49

so whether what the teams are working on are aligned to quarterly, yearly and multi-year strategic outcomes

Zsolt Berend, Co-Author - Sooner Safer Happier16:10:14

measure: %stories that linked up to strategic outcomes

John Ediger16:10:22

who decides that?

Marc Price - Nationwide Building Society (Speaker)16:10:40

we partnered with our value realisation office

Zsolt Berend, Co-Author - Sooner Safer Happier16:10:57

so practically in Jira: story - epic - OKRs - yearly/multi-yr outomes

Adam Bell16:10:23

parent/child relationship of a sort?

Craig Cook - IBM16:10:52

value realisation office?

Jon Smart [Sooner Safer Happier]16:10:02

VRO = reinvented PMO. From traditional to supporting the organisation on value (articulated via OKRs)

👍 1
Jon Smart [Sooner Safer Happier]16:10:02

pg. 202 in Sooner Safer Happier book

👍 2
thankyou 1
Tashfeen Mahmood16:10:41

do you use tools like Jira, ServcieNow, etc. to measure these flow metrics?

Zsolt Berend, Co-Author - Sooner Safer Happier16:10:36

and we link logically to measure end to end lead time

Andy Nelson16:10:02

and SNOW chg requests are created/completed as part of deployment pipelines for the most part with some exception manual ones?

Zsolt Berend, Co-Author - Sooner Safer Happier16:10:22

our flow metrics is not reliant on how the changes being created - we link it through fix versions

👍 1
Matt Ring (he/him) - Sr. Product/Engineering Coach, John Deere16:10:30

What was the dashboard / visualization tool you leverage? I think it was just meentioned but I missed it.

Marc Price - Nationwide Building Society (Speaker)16:10:59

lets here from our customers....

👍 1
Andy Nelson16:10:09

this accelerate metrics view - is this at the app level? product team? value stream? or multiple different "flight levels"?

Marc Price - Nationwide Building Society (Speaker)16:10:44

Hi Andy, We allow the users to filter to their context. So all levels of the society

👍 1
Marc Price - Nationwide Building Society (Speaker)16:10:26

Mel was one of our early adopters of our product within the intelligent controls team!

Craig Cook - IBM16:10:35

Is any of this tooling open source?

Marc Price - Nationwide Building Society (Speaker)16:10:58

We are using a tool called ThoughtSpot for reporting and self service

thankyou 2
Marc Price - Nationwide Building Society (Speaker)16:10:49

its great to see how Craig has implemented our product within his own context in the payments team

👏 1
Ganga Narayanan16:10:07

This is great. The lead time in the dashboard, how was it calculated? What tool?

Ganga Narayanan16:10:07

Thank you!

👍 1
Zsolt Berend, Co-Author - Sooner Safer Happier16:10:10

lead time: #days from work started to release in prod (snow)

Ganga Narayanan16:10:47

Got it, thank you!

Matt Ring (he/him) - Sr. Product/Engineering Coach, John Deere16:10:49

@berendzs Did you need to standardize on JIRA board columns for this to work or did teams retain some flexibility in identifying their state columns (to visualize the flow of work)?

Zsolt Berend, Co-Author - Sooner Safer Happier16:10:43

@mring on purpose we wanted to give full autonomy for teams to choose their workflow as it is unique so no one size fits all

❤️ 1
Zsolt Berend, Co-Author - Sooner Safer Happier16:10:45

instead we created a map from all states to todo in progress done - this is an essential step to make the calc. agnostic the workflow states as well as to teams ways of working

Tashfeen Mahmood16:10:23

Do you find that your metrics are only as good as teh data that people provide in Jira/SNOW?

Marc Price - Nationwide Building Society (Speaker)16:10:11

we have seen that, yes. It requires good discipline in updating stories and change records etc

Tashfeen Mahmood16:10:55

how do you improve that discipline? how granular are the workflow states?

👀 1
Marc Price - Nationwide Building Society (Speaker)16:10:39

we are agnostic to data sources as we do mapping within our data model to map to todo, in progress and done so are not reliant on all teams using the same workflow

Zsolt Berend, Co-Author - Sooner Safer Happier16:10:17

teams have the ability to reflect their unique flow of work

Tashfeen Mahmood16:10:31

how many teams realize this and go back to look at their flow states?

Tashfeen Mahmood16:10:40

versus just giving up?

Tashfeen Mahmood16:10:17

I find that most do not invest that time... that is just tribal knowledge... people do not care about putting that in metrics

Marc Price - Nationwide Building Society (Speaker)16:10:22

we provide a link back to jira within the reporting etc

Zsolt Berend, Co-Author - Sooner Safer Happier16:10:40

what we see is that using VSM data from our dashboards teams go on the journey to make their queues and work states visible

Zsolt Berend, Co-Author - Sooner Safer Happier16:10:15

they go from simple todo in progress done boards to boards that reflect their vsm

Glenn Wilson, Author of DevSecOps16:10:28

I like this golden thread approach to measuring productivity. It’s simple yet very effective

❤️ 2
Marc Price - Nationwide Building Society (Speaker)16:10:35

Thanks for joining of session today, its been great chatting with you all. Please reach out to keep the conversation going!

👏 3
🎉 3
Andy Nelson16:10:34

👏 @marc.price @zsolt.berend - Great presentation - glad to see BVSSH in action and the value it provides the org

❤️ 4
Ganga Narayanan16:10:39

Thank you! And congratulations on the amazing work you've done!

❤️ 2
Ann Perry - IT Revolution16:10:00

☀️ More from Discover, let's now welcome @sakthikasiramalingam and @bryanpayton here to present, The Shift to a DevOps Model While Building Our Cloud Platform - You Build it, You Run it! ☀️

👍 1
👋 2
Tashfeen Mahmood16:10:04

Thank you @marc.price and @berendzs

❤️ 2
Bryan Payton16:10:22

Thanks @annp !! Here to answer any questions.

👍 1

@bryanpayton @sakthikasiramalingam How do you manage daily production tickets?


especially for systems like Payroll or HR

John Ediger17:10:28

Is this single backlog & rotation being done in the app teams or is this just for this platform team?

Graham McGregor17:10:08

how big are those teams that were combined?

Bryan Payton17:10:29

@babita.sood We have an operational rotation 24 hrs a day segmented into 8 hour periods. As they incidents come in, we prioritize them and open an internal response call until resolved. During this we maintain an master station log of activities performed and times. If there is a rotation shift we turn over the master station log between engineers. Escalations for SME's happen through our escalation process, but with the consolidation of Ops/Eng we found we have fewer escalations.


Thank you, how is Backlog managed in this case

Bryan Payton17:10:19

We have weekly refinement and weekly planning/grooming meetings to keep inline with priorities and pivots. We also have weekly incident/issue calls for reflection and creation of backlog work for fix-actions and remediation. The individuals on the operational rotation take 'operational' work from our backlog when issues/incidents are slower and capacity permits. This keeps us progressing both engineering and operational deliverables while providing production response.


Thank you!!

Bryan Payton17:10:53

@halfmoondad Currently this is down on the platform and product side (e.g Databases, Cache, Messaging, etc). We are not directly involved in the AppDev side, but we have shared our success and lessons with that community. They are adopting.

👍 2
Bryan Payton17:10:36

@graham_mcgregor The current team size is roughly 40 people.

Bryan Payton17:10:26

@graham_mcgregor we have found the 'more the merrier' with this approach as the rotations are less frequent and the perspectives are rather 'fresh'.

Sakthi Kasiramalingam17:10:14

@graham_mcgregor True, we had to bring more of our engineers into the rotation schedule to enable this

Brian W. Spolarich - Cal Poly17:10:42

We're using the concept of paired Eng/Ops teams today in my org that share a common portfolio. It works if the leads are aligned. Did you collapse these into single teams with a single leader?

Sakthi Kasiramalingam17:10:24

@brianspo Yes these teams were all already under one organization

Bryan Payton17:10:25

@brianspo Yea, same objectives. Myself and a Senior Product Owner oversee all products/deliverables within this space. However, each product has its own PO and PA. Operations has a Ops Manager that is consistent and we simply rotate the individuals in and out. All of our leaders are aligned and were involved in this decisions.

Brian W. Spolarich - Cal Poly17:10:06

I think what is resonating for me here is the product thinking, which means you need to start further upstream. How you're organized to build and deliver is a downstream effect of the upstream org approach.

Sakthi Kasiramalingam17:10:37

Agree it’s about the product orientation instead of a project focused approach

💡 1
Brian W. Spolarich - Cal Poly17:10:42

@bryanpayton - Are slides for this talk avaialble? I didn't see them in the library.

Sakthi Kasiramalingam17:10:56

@annp are the slides that we submitted typically shared out ?

Ann Perry - IT Revolution17:10:47

Yes, they will be available along with your video in the Video Library. If you're okay with that, I'll make sure they're included.

👍 1
Brian W. Spolarich - Cal Poly17:10:28

BTW, I'm really appreciative of the presence the Discover folks have had at the conference. I've found your talks really relatable from the standpoint of where I'm at in my (much smaller, academic IT) org.

👍 1
Bryan Payton17:10:14

Appreciate the conversation everyone - thanks for attending our session!! We will get the slides published.

❤️ 2
Ann Perry - IT Revolution18:10:00

🔵 As we return from the break, we now welcome @cmchugh and @jdirks from TrueBlue, here to present A Force for Good: How TrueBlue is "Accelerating" Connecting People with Work 🔵

Scott Prugh (CTO - CSG) DOES Prog Committee18:10:28

Go @cmchugh and @jdirks!! Good luck!

Carter McHugh (He/Him)18:10:42

I'm thrilled to be here today with @jdirks!

Jeff Dirks (TrueBlue CIO/CTO)18:10:56

Hi Everyone! I'm excited and honored as well to be a part of this great event.

Scott Prugh (CTO - CSG) DOES Prog Committee18:10:51

Love the Flywheel of Decay visualization!

Scott Prugh (CTO - CSG) DOES Prog Committee18:10:21

The downward spiral is one we often see in transformations

Carter McHugh (He/Him)18:10:27

We really felt "landlocked"...almost held hostage by forces many felt powerless to change.

Jeff Dirks (TrueBlue CIO/CTO)18:10:46

@scott.prugh Yes, it's a play on TrueBlue's Flywheel of Innovation we discuss with our Board and Investors.

Scott Prugh (CTO - CSG) DOES Prog Committee18:10:51

We often call this the "Pit of Despair"

Jeff Dirks (TrueBlue CIO/CTO)18:10:16

Love "pit of despair" and have seen that movie a few times in 30 years of tech.

Carter McHugh (He/Him)18:10:08

Tools are important, but we needed more to change our culture.

Ganga Narayanan18:10:43

"Many opinions, but far fewer facts". Fact!

Scott Prugh (CTO - CSG) DOES Prog Committee18:10:29

The performance metrics in Accelerate have been so critical in many transformations! @nicolefv @genek @cmchugh @jdirks

👍 1
Carter McHugh (He/Him)18:10:21

We wanted to avoid two major impacts here: teams opting out with reasons (e.g. we're legacy, our customers won't allow it, etc) and also avoid gaming the system with proxy/vanity metrics.

Carter McHugh (He/Him)18:10:16

Accelerate talks about how important it is that teams set the commitments and pick their tools. We needed to embrace that.

Scott Prugh (CTO - CSG) DOES Prog Committee18:10:49

Not setting hard targets for the measures is a great approach. Let the teams measure to learn to then set their own goals.

Ganga Narayanan18:10:03

The comment about some teams being green-field having different goals from some that are legacy is very true. We don't want a comparison between the teams in goals and targets!

Carter McHugh (He/Him)18:10:13

The teams had lots more opinions after we started this!

Carter McHugh (He/Him)18:10:43

But they started innovating...

Carter McHugh (He/Him)18:10:11

One team had really good CI/CD already in place, but were bottlenecked by cumbersome CM policies and procedures.

Carter McHugh (He/Him)18:10:38

They set to work with the CM team to define a new, automated "Standard Change" policy that enabled them to deploy daily.

Scott Prugh (CTO - CSG) DOES Prog Committee18:10:49

I love the compass analogy! The key is to teach teams to measure to learn to improve and coach them to do that!

Jeff Dirks (TrueBlue CIO/CTO)18:10:58

Yes - we talk a lot about compasses over maps. Get on the field of play, get more feedback loops and use those for orienteering towards the end goal. Maps tend to promote waterfall approaches.

🎯 1
Carter McHugh (He/Him)18:10:15

Some legacy teams found a way to do some even very simple improvements...many legacy procedures were simply done again and again just because no one even questioned them anymore.

Carter McHugh (He/Him)18:10:33

" only takes our release engineer 2 mins to do that"

Carter McHugh (He/Him)18:10:06

But added up, they were spending hours in manual tasks and this created fear, error, and declining motivation.

Scott Prugh (CTO - CSG) DOES Prog Committee18:10:30

Wow. Some great 10x results in changes deployed!

Jeff Dirks (TrueBlue CIO/CTO)18:10:12

Yes - we focused thematically on making it easier to get work done -- and automating everything.

Carter McHugh (He/Him)18:10:53

The big learning for me, is how shifting into smaller release, suddenly showed up in the other metrics. KTLO came down because there was less to troubleshoot each day, or plan for rollbacks, or other non-value add activities.

Carter McHugh (He/Him)18:10:28

We know teams like innovating, and this "broke the logjam". Forces that folks couldn't describe well let alone affect for good, were given clarity in "Accelerate"

Scott Prugh (CTO - CSG) DOES Prog Committee18:10:29

Yes. Start the journey where ever you are! Don't get discourage by: 1) the size of the challenge or 2) where others are

❤️ 1
Scott Prugh (CTO - CSG) DOES Prog Committee18:10:13

Remember: "The best time to plant a tree was 20 years ago. The second best time is today."

❤️ 1
Javier Magaña - Walmart18:10:44

We are pushing from the bottom up, but there seems to be a lack of support from the top. This causes our message to get missed sometimes. Everyone is too busy to improve

👍 2
Carter McHugh (He/Him)18:10:56

In my first DOES event in 2016, I think how I felt @jason.cox were so far ahead. How can we even begin. But I learned everyone is on their own journey and start where you are.

Ganga Narayanan18:10:16

TIL: Goodhart's Law. (Also reminds me of the Cobra effect :P)

Carter McHugh (He/Him)18:10:31

@ganga.narayanan The Cobra effect is a frequently cited example of Goodhart's law.

👍 1
Javier Magaña - Walmart18:10:38

Thanks for the session and the tips!

Jeff Dirks (TrueBlue CIO/CTO)18:10:22

Thanks all! We are happy to share our journey and learn from others along the way!

Carter McHugh (He/Him)18:10:29

It's important to be aware...measure many things, avoid universal hard targets. Encourage transparency, learning....a Generative culture.

💯 1
Carter McHugh (He/Him)18:10:53

Thanks all! Great talking with you today.

❤️ 4
Jill Mead18:10:37

@erickramlinger and I are so excited to be here!

Ann Perry - IT Revolution18:10:48

🎧 Excited to welcome @jillmead2018 and @erickramlinger who will share, From Grass Roots to Brush Fire: Creating Massive Organizational Momentum Using Communities of Practice During a Global Pandemic 🎧

Scott Prugh (CTO - CSG) DOES Prog Committee18:10:20

Wait!! @jillmead2018 No more Dumpster Fires??

❤️ 2
Jill Mead18:10:32

@scott.prugh hahah

Jill Mead18:10:43

Not today. 🙂

Scott Prugh (CTO - CSG) DOES Prog Committee18:10:01

Darn. I'm a big fan of those! 🙂

Ganga Narayanan18:10:04

Was going to say waiting for the brush fire to start, but wasn't sure if that would be appropriate 😄

❤️ 1
Eric Kramlinger18:10:14

Hoping to put some rain on the dumpster fire today!

Jill Mead18:10:24

@ganga.narayanan haha!

Brian Smith18:10:39


😜 1
Scott Prugh (CTO - CSG) DOES Prog Committee18:10:21

Wait. Is this the conference call we got off of this morning?

🎉 1
❤️ 1
😂 1
Javier Magaña - Walmart18:10:34

I was about to say that they appeared to be in the same room

❤️ 1
Jennifer Collings18:10:08

Love the humor!!

❤️ 2
Shashi Tippabhatla18:10:15

silos to individual vacuums...LOL

❤️ 1
Scott Prugh (CTO - CSG) DOES Prog Committee18:10:57

Hah. If I had a $1 for every time I heard: "That won't work here" I wouldn't still be doing this DevOps thing.

❤️ 4
😆 1
Ganga Narayanan18:10:53

If I had $1 for every time I heard that, I'd be able to buy a gold chain like the one Eric has!

Jill Mead18:10:41

@scott.prugh 100%

Scott Prugh (CTO - CSG) DOES Prog Committee18:10:47

Yes. Model the behavior you want to see in your people!!

❤️ 1
Andrew Davis - Copado - DevOps and Testing for Low Code18:10:02

@erickramlinger that is some first-class bling you have going.

❤️ 1
Andy Nelson18:10:20

Its no salesforce tattoo though...

Carter McHugh (He/Him)18:10:27

I get energy here too!

👏 2
❤️ 1
Nicole Forsythe18:10:30

LOVE THIS energy

❤️ 1
👏 2
Eric Kramlinger18:10:38

@abd3721 - THANK YOU! Of course, my dog had to literally take a bite out of the corner. 🙂

❤️ 1
Scott Prugh (CTO - CSG) DOES Prog Committee18:10:26

I got involved in this community because of the: 1) learning, 2) energy, 3) hope

❤️ 3
Tashfeen Mahmood18:10:27

The high from DOES lasts for at least a month!

❤️ 2
Nicole Forsythe18:10:52

@erickramlinger are you at US Bank or Traive?

Eric Kramlinger18:10:15

@nicole.forsythe – just made a move to Traive 3 weeks ago

👏 1
Andrew Davis - Copado - DevOps and Testing for Low Code18:10:18

@jillmead2018 great to see you presenting again this year. Always very engaging

❤️ 1
Jill Mead20:10:52

thanks @abd3721

Nicole Forsythe18:10:58

I stopped saying “eat our own dog food” in part because I don’t like dog food…we’ve been using “sip our own champagne”

❤️ 7
🐶 1
Carter McHugh (He/Him)19:10:46

Ha, we say the same now too!

👏 1
Ray Padron19:10:24

I say "drink our own champagne" a lot, but that's probably because I rarely sip. 😅

😜 1
Jill Mead20:10:26

@nicole.forsythe Yummy champaigne.

Eric Kramlinger18:10:40

Brilliant @nicole.forsythe!

Nicole Forsythe19:10:23

Probably came from someone smart, definitely not original to me

Javier Magaña - Walmart18:10:47

Is it better to start weekly to gain momentum?

👏 1
Jill Mead19:10:54

I think so.

👍 1
Jill Mead20:10:03

Honestly - I think many lose steam because it is so infrequent. We were consistent to start. We allowed the community to build when it needed to the most.

Jill Mead18:10:55

@abd3721 haha! We have a blast with these videos!

Ray Padron19:10:01

Did this CoP have any people exclusively dedicated towards its cultivation or was it just part time for everyone?

Jill Mead19:10:55

Hey Ray - We have a point person, but it is very parttime

Eric Kramlinger19:10:02

Jill and I were dedicated to it, and other folks jumped in as volunteers as it grew.

👍 1
Nicole Forsythe19:10:04

Coming from Target and knowing the crew that went to US Bank, how much do you attribute your CoP to what Target had done?

Jill Mead19:10:29

Hmm. good questions

Jill Mead19:10:05

@nicole.forsythe I think it helped to pull from past target experiences and all the creativity that went into so many things. But I never experienced a super engaged, effective product cop at Target during my tenure there. We were so hesitant to create yet another cop based on experiences with them.

❤️ 1
Jill Mead20:10:37

@nicole.forsythe We decided to throw this on its head. Create experience that overtime was morphed into something special. Honestly - i dont know how this forum would have morphed if we were all co-located. Part of me thinks it gained alot of steam because it was remote.

Joe Arrowood19:10:33

The "Problem Is" that many developers don't understand 1. How to Test 2. Why to Test 3. That it is their responsibility to test. 4. Others will test their code AFTER they have tested their code. Once they experience the POWER of testing their own code, they are hooked. But you have to get them to do it, in order to get them to love it. A CoP could help get them immersed in testing!

❤️ 2
Eric Kramlinger19:10:05

@nepobunceno we thought it was. Start weekly to establish familiarity, routine, and find a sustainable pace

Javier Magaña - Walmart19:10:02

We're about to start a community, and we have them set up monthly... time to revisit that then :thinking_face:

Eric Kramlinger19:10:58

Great point. We had a few stop and starts and found that we needed some initial consistency to gain momentum.

Ganga Narayanan19:10:20

Can relate! We've seen many starts and stops every now and then. People getting busy; reorgs; changing mandates etc. -- and also because these are seen as voluntary.

Klaudia Breslavets - Vanguard19:10:21

@jillmead2018 & @erickramlinger, I feel like CoP done well is an underrated topic. I've seen pockets of CoPs done well in my travels but they are rare!! I have found the hardest part in a CoP, beside momentum to start it, is evolving out of a solely consumption-based participant engagement to a balance of contribution-based. In your experience, have you seen this challenge & any suggestions to reach equilibrium?

👍 6
❤️ 1
Eric Kramlinger19:10:56

@kxbres - great call out. Every CoP featured different members of the organization. We also did "feedback blitzes" where volunteered to show off artifacts and the CoP would ask questions and share feedback. A few examples of how we tried to create a back and forth flow.

Tashfeen Mahmood19:10:27

I often say: I have yet to meet a CoP that works well. I wonder if I need to have personalities like @erickramlinger and @jillmead2018 to make a CoP work well... My question is therefore: Is a CoP model sustainable?

👏 1
Klaudia Breslavets - Vanguard19:10:12

@tashfeen_mahmood one of the CoPs I've seen work really well is with great moderation that believes in drawing out the expertise available in the "hive mind" so-to-speak and enabling many people to share their experiences/SME'ism rather than relying on the moderators/organizers of CoP. Another was feedback from the community & offering enablement from the CoP to address the feedback/needs. Also..rules of engagement that are established & understood so existing and new participants know how best to engage/gain & contribute value as quickly as possible.

❤️ 1
Tashfeen Mahmood19:10:22

and my question is: how many people can do it well? I can tell just by listening to Eric and Jill that these are two extraordinary people, leaders, thinkers, personalities, DevOpsers, Product People... How many of those do you know? By the way: I can tell that you folks at Vanguard are also geniuses. I saw the session from another Vanguard person on Day 1 and I am very impressed!

Jill Mead20:10:14

@tashfeen_mahmood How bought in is your organization to moving to more of product, business agility and/or devops values, ways of working, ect Do you know anyone whom are natural leaders for this movement? Our goal is to create more leaders in this space internally?

Tashfeen Mahmood20:10:23

we are bought in... However, I will also say that if you take 20 executives (from any organization) and ask them what DevOps means, you will get 21 answers. I would like to talk to you more about the cultural topic you brought up related to leadership. IMO - Leadership grows more leaders. That is something we can improve at my company.

Andrew Davis - Copado - DevOps and Testing for Low Code19:10:34

Was there a particular format you used? Did it vary per session?

👀 2
Eric Kramlinger19:10:52

Andrew – varied per session, but through experimentation, found some helpful repeat formats: feedback blitz, stories of impact, and breakout group small discussions.

Andrew Davis - Copado - DevOps and Testing for Low Code19:10:48

So one session would be a “feedback blitz” and another one would be “stories of impact”?

Ganga Narayanan19:10:59

Love the mascots idea; very helpful for branding.

⛓️ 1
👏 1
Patrick S. Kelso19:10:09

How do you create longevity? What happens if you two leave the org at the same time? @jillmead2018 & @erickramlinger

Jill Mead19:10:58

Hey @nl_does We have a group of 6-10 DJ's

❤️ 1
Jill Mead20:10:19

@nl_does One of the key things is that ending a cop isnt necessarily a sign of failure. No need to have a cop for the sake of having one.

Ganga Narayanan19:10:37

So you didn't do book clubs then?

Ganga Narayanan19:10:42

Resonates in many ways. CoP : Everyone wants them; everyone talks about them; easy to start; but difficult to sustain!

💯 4
➕ 3
🎉 1
Eric Kramlinger19:10:14

Hi @ganga.narayanan – no books in this CoP. We have a book club, but found that we wanted to focus on the human experience and hear eachother's stories.

Scott Prugh (CTO - CSG) DOES Prog Committee19:10:20

Yes!! Telling stories is so powerful!! That is one of the reasons I love DOES!

💯 2
❤️ 3
Jill Mead19:10:07

Concepts mean nothing without stories

Carter McHugh (He/Him)19:10:23

Wow, that's great! 39 sessions and growing.

❤️ 1
👏 1
Javier Magaña - Walmart19:10:38


👏 1
❤️ 2
Randy Shoup (Chief Architect eBay; he/him)19:10:04

nps wow

👏 1
🙂 1
❤️ 1
Ganga Narayanan19:10:28

That's amazing! Thank you for sharing!

❤️ 1
Drew Boyer19:10:33

This talk gives me hope for the promise of CoPs!! Stay with it, listen, empathize, adapt, practice vulnerability!

❤️ 3
Ray Padron19:10:38

Now I'm thinking of starting my CoP with some of the great DOES2021 talks!

❤️ 5
Ganga Narayanan19:10:04


❤️ 1
Jennifer Collings19:10:48

Thank you!!

❤️ 1
Jill Mead19:10:16

I will be honest - my past with cops have been gross.


Love those micro videos @jillmead2018

👏 1
❤️ 2

awesome - documentary style

Ganga Narayanan19:10:05

A Product Community of Practice or a practice to build a community around products, or a product that was the community of practice? 🙂 Inspiring video!

Jill Mead19:10:24

We pretty much cover everything related to product

❤️ 1
Eric Kramlinger19:10:57

Great perspective. All of the above. Practice to build a community around our product. I love that!


We celebrate almost everything including failures - EXCELLENT

❤️ 1
👍 2
Brian Smith19:10:54

The more you share, the more you grow. End with a dance party - love it.

❤️ 1
Ann Perry - IT Revolution19:10:00

:kiwifruit: We are very honored to welcome @paul003 and @lbmkrishna here to present, Banking on Flow Metrics – Bank of New Zealand Transformation Journey :kiwifruit:

upvotepartyparrot 3
❤️ 1
Courtney Kissler19:10:10

Super excited for this talk!!

🙏 1

Kio Ora from NZ 🙂 @paul_littlefair

Paul Littlefair19:10:03

Hello everyone!

👋 3
❤️ 1
Patrick S. Kelso19:10:09

Probably the only people here who had to get up earlier than me 😄

😂 2
❤️ 1

ha ha ha 2:30 Everyday - for DOES - It DOES NOT MATTER

❤️ 2
➕ 1
Patrick S. Kelso19:10:04

I have 11am to 1pm blocked in my calendar so if I need a recharge nap I can take it.


I could understand 🙂

Shashi Tippabhatla19:10:34

Amazing talk on CoP!! @jillmead2018 @erickramlinger Learned great ideas....Thank you!

❤️ 3
Jill Mead19:10:07

aww. Thank you!

Roberto Hernandez19:10:57


➕ 1
❤️ 1

Really interesting talk.... got my brain turning on how to bring some of these ideas to my team.

❤️ 1
Jill Mead19:10:29

Of course! A staple lead out song!

Eric Kramlinger19:10:31

Thanks for the conversation! Can't wait to @paul_littlefair. Cheers!

❤️ 1
Tashfeen Mahmood19:10:32

Oops I showed my age!

❤️ 1
😂 3
🙂 1
Brian Smith19:10:32

Phenom! Love

❤️ 2
Joe Arrowood19:10:34

Definitely different from other talks. It was great. Obviously your CoP's are different as well.

❤️ 2
Andy Nelson19:10:36

👏 great stuff @jillmead2018 @erickramlinger - Love hearing about the community you are building at US Bank

❤️ 1
Tashfeen Mahmood19:10:01

@jillmead2018 and @erickramlinger That was just awesome! I thoroughly enjoyed it! I would attend any meeting, any CoP with you guys any day!

❤️ 2
Eric Kramlinger19:10:28

@tashfeen_mahmood thank you! Let's make that happen :)

Michal Jackowski Procter and Gamble19:10:11

Fantastic session!!! Super inspiring!!

❤️ 1
Eric Kramlinger19:10:48

Thank you, Michael!


I want to introduce @paul_littlefair to our DevOps Enterprise Community

👏 4
Courtney Kissler19:10:09

Welcome @paul_littlefair

👏 1
Paul Littlefair19:10:16

Such a great presentation on CoP @jillmead2018 and @erickramlinger. I particularly enjoyed the Dance Jam at the end- but also some great ideas to try out with our teams. Thank you so much :)

❤️ 3
Jill Mead19:10:57

Thanks @paul003 So excited to be listening to your talk! Amazing. And currently being in a bank 🙂

thankyou 1
Mik Kersten (Project to Product, Tasktop)19:10:54

Love the “Digital First, Human When It Matters”!

❤️ 1
👏 7

Banking relies on trust 🙂

💰 1
Scott Prugh (CTO - CSG) DOES Prog Committee19:10:19

Yes!! Assembly and mainframes again!

😂 2
Scott Prugh (CTO - CSG) DOES Prog Committee19:10:29

Note that you can do great things there too!

💯 2

@scott.prugh - Your talks are inspiration for us; Watched all your DOES talks and interacted with you 🙂

❤️ 1
Javier Magaña - Walmart19:10:28

Those things will be addressed later. I need my feature now! /s

😂 2
Javier Magaña - Walmart19:10:31

yes... it is something we often tell people.

Mik Kersten (Project to Product, Tasktop)19:10:37

This diagram is soooo effective

💯 1
Paul Littlefair19:10:54

Absolutely @scott.prugh! All our teams are embracing DevOps, including the Mainframe teams. Learning journey for us all!

jeff.gallimore (CTIO - Excella, he/him)19:10:57

“hire more people. that’ll fix the problem.” 😭

😭 1
😂 2
Tom Wojtusik (Tasktop)19:10:01

So key: Need to be able to see. Digital work is hidden.

💯 2
Craig Cook - IBM19:10:22

Something about Agile, make the work visible?

jeff.gallimore (CTIO - Excella, he/him)19:10:57

the images on these slides are brilliant.

Scott Prugh (CTO - CSG) DOES Prog Committee19:10:57

Sigh. Dependency management, WaitTime(queues) and knowledge all degrade exponentially. Just hiring without understanding these bottlenecks is like lighting money on fire.

🔥 4
💰 1
👍 2
💸 1
jeff.gallimore (CTIO - Excella, he/him)19:10:21

ToC: any optimization not made at the constraint is waste

Mik Kersten (Project to Product, Tasktop)19:10:59

It all goes back to making work visible 🙂

👍 4
💯 2

There we go @paul_littlefair - Bold moves 🙂 and one of the moves

Mik Kersten (Project to Product, Tasktop)19:10:54

The recycling symbol on PMO is quite amusing 🙂

John Willis19:10:12

You have to expose the visibility. I think Team Topologies creates a good framework for exposing visibilities artifacts. Dojo's also go a long way to make work visible.

👍 2
💯 3

It freed up our cash cc - @mik (recycled)

👍 2
Mik Kersten (Project to Product, Tasktop)19:10:33

And accurate I think, as I have seen a lot of PMO leaders become champions of Flow!

👍 1
👏 2
💯 2

I could not agree more - YES

Paul Littlefair19:10:11

Yep @cncook001 So easy to understand the concept, but even in 2021 so few corporates actually doing this. We all report the flow at the end of the system, but so few exec teams look across the whole system.

💯 5
John Willis19:10:35

I'm great at platatudes.

😂 2
😆 1
jeff.gallimore (CTIO - Excella, he/him)19:10:36

i can imagine the removal of the pmo was… bumpy?

🔥 2
⚔️ 1
John Willis19:10:03

Actually, I stole it from Andrew.

John Ediger19:10:11

how did you do that? -- Make time for continuous improvement

👍 4
Paul Littlefair19:10:47

We are so grateful to @jwillis! Even if you are admitting to plagiarism :)

❤️ 4
👏 1
jeff.gallimore (CTIO - Excella, he/him)19:10:14

“reuse”. it’s a virtue.

❤️ 2
John Willis19:10:36

Thanks @paul_littlefair.. hopefully, I can visit you all next year...

Dian Hansen19:10:40

"measuring only one area of the value stream is like only using two inches of a 12-inch ruler" - couldn't love this more!!! ❤️

❤️ 2
Randy Shoup (Chief Architect eBay; he/him)19:10:42

This @jwillis guy sounds pretty smart ...

💯 2
👍 3
Dominica DeGrandis, Author - Making Work Visible, Tasktop19:10:03

We didn't wait to get everything perfect before starting.

☝️ 4
❤️ 1
John Willis19:10:13

@rshoup Miss you buddy

❤️ 1

@rshoup - You are an inspiration. All your thoughts and talks - actions - love it

❤️ 1
Randy Shoup (Chief Architect eBay; he/him)19:10:09

Aww, thanks, BMK! This is a great talk.

🙏 1
John Willis19:10:36

@mik Fyi.. I pushed flow metrics hard in my final report. I do this almost everywhere I go... (big fan)

👏 4
Courtney Kissler19:10:53

YES - alignment with tech and business!!

💯 2
Paul Littlefair19:10:15

It certainly was @jeff.gallimore. A great leap of faith in the opportunity of DevOps and Flow from our business colleagues. Relied on lots of communication and building trust through developing credibility, reliability and intimacy.

Tom Wojtusik (Tasktop)19:10:46

@paul_littlefair Love your emphasis on culture and team happiness.

Craig Cook - IBM19:10:47

Creating a common language is important

Mik Kersten (Project to Product, Tasktop)19:10:48

Happy to hear that @paul_littlefair! And such a key point, don’t wait to restructure Jira, get to 100% Agile, etc. Making work visible is a journey where you want flow, feedback and learning for every step of the way.

Courtney Kissler19:10:54

We need @paul_littlefair to talk to more C levels 🙂

❤️ 3
✔️ 1
John Willis19:10:15

In case anyone is interested.. this is the model I used for the analysis at BNZ..

🙏 4
Mik Kersten (Project to Product, Tasktop)19:10:56

This is my main area of interest right now, CxO and board level reporting. So great that you are sharing out your learnings on this @lbmkrishna and @paul_littlefair. And @jwillis time for us to compare notes again I think!

John Willis19:10:38

@mik love geeking out with you.. anytime...

John Ediger19:10:48

Paul, what were the challenges? @lbmkrishna @paul003


@paul_littlefair another bold move - introducing all together a new role VSA (not in NZ), He was so sure that we need this - I would like to acknowledge Paul's vision for us and a true servant leadership

👏 2
Randy Shoup (Chief Architect eBay; he/him)19:10:06

see, improve, scale; love it

💯 5
jeff.gallimore (CTIO - Excella, he/him)19:10:49

very consistent with what @steve773 preaches. i think it’s “see, solve, spread”

Steve Spear10:10:25

@jeff.gallimore @rshoup That’s 100% right. Those who learn more, faster win by the ability to bring new and useful insights into practical use ahead of rivals. That’s the “high velocity edge.” The basis of all learning is recognizing a problem, swarming on it to investigate it and experiment around it to get to a solution. The huge multiplier is sharing what’s discovered so the individual insight gets systemic application and so the individual discover collides with others to build compounded understanding. That latter part is the basis of Jared Diamond’s Guns Germs and Steel (which I use as the intro case in the chapter on knowledge sharing).

🚀 1
thankyou 1
Randy Shoup (Chief Architect eBay; he/him)14:10:10

Thanks, Steve. Your insights and clarity in this area are fantastic. The Diamond reference has me intrigued; GG&S has been one of my favorite books since before it was a big deal 🙂 .

John Willis19:10:22

If all CTO's were like @paul_littlefair the IT world would be a better place...

😊 1
👏 5
Paul Littlefair19:10:35

Always keen to help @ckissler. The DevOps community has given so much to us, so I'm delighted when I get an opportunity to pay it back.

❤️ 8

Tasktop VIZ is amazing tool - LIVE VSM 🙂

🌈 1
Mik Kersten (Project to Product, Tasktop)19:10:57

Thanks @rshoup, but it’s what BNZ is doing with the insights that is so amazing!


FLOW is connected to Business Results

👍 1
Randy Shoup (Chief Architect eBay; he/him)19:10:15

“Shift focus from short-term gains to long-term value”

💯 4
💪 2
👏 1

That too coming from CTO 🙂

jeff.gallimore (CTIO - Excella, he/him)19:10:57

@mik “and how quickly”… from an outside perspective, what sets them apart from others who don’t move as quickly?

John Ediger19:10:57

Did you track flow distribution (of flow types)?


Yes - we introduce 6 types of BNZ flow items and map them back to Flow Items


This is only to align the existing work types

John Ediger19:10:00

thanks. any samples of that?


you mean the flow distribution graph. Yes - for some team - all green, some are all amber - Making the type of work visible also challenge.

Scott Prugh (CTO - CSG) DOES Prog Committee19:10:43

Argh!!! Don't outsource unless it is a non core capability.

💯 6
Paul Littlefair19:10:52

YES @halfmoondad!!!!!!!

Amri Abuseman - Flatiron Health, Director of Quality Engineering19:10:18

Collect data without committing to change is a good place to start (sets team baseline)

Tom Wojtusik (Tasktop)19:10:45

Unlearn and relearn!

👍 2
👏 1
Drew Boyer19:10:12

Love providing Air Cover for experimentation!

🛩️ 1
➕ 1
Gene Kim, ITREV, Program Chair19:10:21

Thank you so much for your amazing presentation, @paul_littlefair and @lbmkrishna!!!!

❤️ 1
👏 1
Paul Littlefair19:10:27

Distribution is key. We need to educate our business colleagues that the work isn't just features.

💯 3
Mik Kersten (Project to Product, Tasktop)19:10:28

@jeff.gallimore That is a great question and almost seems like a talk topic of its own. Directionally I think it is a very effective pairing of exec leadership and tech leadership, rapidly operationalizing the visibility, and really interestingly to me their organization approach to learning. As BMK alluded they created an entire learning portal for learning Flow!

➕ 4
jeff.gallimore (CTIO - Excella, he/him)19:10:36

next year’s talk topic @lbmkrishna @paul003? 🙏

John Ediger19:10:45

@paul_littlefair I would love to learn how you leveraged that. Did teams make adjustments based on that? (as an aside, I've always wondered why "Continuous VS Improvement" is not a distinct flow type.

Dominica DeGrandis, Author - Making Work Visible, Tasktop19:10:00

We didn't wait for the stars to align:sparkles:


Thank you @genek for the opportunity to share with our community.

Mik Kersten (Project to Product, Tasktop)19:10:05

Actually maybe it’s what Paul just said: “Not waiting for stars to align”. I think that;’s it!

🌠 2
Mik Kersten (Project to Product, Tasktop)19:10:29

Having teams be OK with data problems and visibility, and creating the psychological safety to make that a journey.

👍 1
Mik Kersten (Project to Product, Tasktop)19:10:45

OK, that’s my main take away, “not waiting for stars to align”

👍 1
Tom Wojtusik (Tasktop)19:10:45

Let the data shape the journey. Love this, @lbmkrishna

David Van Couvering - Senior Principal - eBay19:10:50

So, sorry, but this is just an awesome talk, and thanks to @mik for your book and vision

☝️ 1
💯 1
John Willis19:10:29

Read @dominica Making Work Visable. One of my core tools for helping clients.

❤️ 5
👍 3
💯 1
Randy Shoup (Chief Architect eBay; he/him)19:10:07

The book I show to my teams 2nd-most often, right after Accelerate 🙂

Randy Shoup (Chief Architect eBay; he/him)19:10:24

Project to Product is 3rd; sorry @mik

Mik Kersten (Project to Product, Tasktop)20:10:28

Very happy to be in your top 3 @rshoup!


I will tell you once you put the tool like VIZ, that exposes the JIRA and other tools data issues - The teams are in SHOCK;

Gene Kim, ITREV, Program Chair19:10:08

@paul_littlefair It’s been so amazing to interact with @lbmkrishna from the very beginning — I’m so delighted that he’s presenting with you here! THANK YOU!

❤️ 1
🙏 1

Making Work visible and also Make the problems/data issues visible

Jon Smart [Sooner Safer Happier]19:10:39

Thanks @paul_littlefair @lbmkrishna, great talk!


Thank you @jonathansmart1 - The days are not that far - Better Value Sooner, Safer and Happier


Thank you @chris.gallivan421 - please share your feedback 🙂

Courtney Kissler19:10:46


💯 2

Thank you @ckissler for your support

❤️ 1

Kia kaha - Take care folks

❤️ 4

Thanks @rshoup - would love to catchup sometime to talk about "Architecture" 🙂

Randy Shoup (Chief Architect eBay; he/him)20:10:37

I'd be happy to, @lbmkrishna

🙏 1
Drew Boyer19:10:51

That was GREAT!!

❤️ 2

Thank you Drew

🙏 1

Thank you, glad you liked it

Amri Abuseman - Flatiron Health, Director of Quality Engineering19:10:51

Insightful talk - thank you! In my experience, data speaks for itself in showing what's inefficient.

💯 1
Javier Magaña - Walmart19:10:55

Excellent presentation. Thanks

Paul Littlefair19:10:36

Thanks All. So grateful for the opportunity to share. Please let us know how you are going on your journey :)

❤️ 1
Tom Wojtusik (Tasktop)19:10:37

Thank you for sharing @paul_littlefair and @lbmkrishna. Inspiring!

thankyou 5

Thank you @tom.wojtusik for all your support, much appreciated

Mik Kersten (Project to Product, Tasktop)19:10:30

👏 As @jwillis said, this kind of leadership moves mountains. Honored to get to collaborate with you @lbmkrishna and @paul_littlefair.

thankyou 1
👍 1
❤️ 1

Thank you @jwillis, @mik - for all the support

Steve Kurzeja, CTO - Vocus19:10:05

Great to see fellow kiwis presenting at DOES! Well done @paul003 and @lbmkrishna. We've been on the same journey so will definitely compare notes!


Woohoo. another Kiwi here, love it:kiwifruit:


Reminder: The final plenary sessions are starting in 5 minutes. Start making your way back to your browser and join us in #ask-the-speaker-plenary to interact live with the speakers and other attendees.

Tashfeen Mahmood21:10:07

That was great! Thank you @shelby and @lizf

➕ 2
❤️ 1
Joe Arrowood22:10:21

Thank you @allspaw I enjoyed your talk.

Joe Arrowood22:10:29

It was the best DevOps Summit that I have ever attended

❤️ 3
Chris Glover22:10:50

@allspaw in your journeys, what has been the most effective method of publishing incident learnings? Any particular method that was excellent at capturing metadata for future engineers to sort by?

John Allspaw22:10:50

I’d say the content of write-ups is paramount…the specific platform (Google docs, Confluence, etc.) matters very little if whatever indexing/search doesn’t have much to index.

thankyou 5
Vlad Ukis07:05:08

How to judge whether learning really happened?