This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-10-07
Channels
- # ask-the-speaker-track-1 (422)
- # ask-the-speaker-track-2 (356)
- # ask-the-speaker-track-3 (215)
- # ask-the-speaker-track-4 (278)
- # bof-arch-engineering-ops (2)
- # bof-leadership-culture-learning (12)
- # bof-sec-audit-compliance-grc (1)
- # bof-working-with-data (1)
- # demos (7)
- # discussion-main (1182)
- # games (73)
- # games-self-tracker (1)
- # gather (4)
- # happy-hour (38)
- # help (82)
- # hiring (14)
- # lean-coffee (8)
- # networking (20)
- # summit-info (101)
- # xpo-adaptavist (5)
- # xpo-anchore-devsecops (7)
- # xpo-aqua-security-k8s (2)
- # xpo-basis-technologies (2)
- # xpo-blameless (3)
- # xpo-bmc-ami-devops (1)
- # xpo-cloudbees (14)
- # xpo-codelogic-code-mapping (1)
- # xpo-dynatrace (1)
- # xpo-everbridge (2)
- # xpo-gitlab-the-one-devops-platform (1)
- # xpo-granulate-continuous-optimization (1)
- # xpo-instana (1)
- # xpo-itrevolution (9)
- # xpo-launchdarkly (1)
- # xpo-pagerduty (1)
- # xpo-planview-tasktop (3)
- # xpo-rollbar (1)
- # xpo-servicenow (2)
- # xpo-shoreline (2)
- # xpo-snyk (2)
- # xpo-sonatype (7)
- # xpo-split (1)
- # xpo-splunk_observability (8)
- # xpo-stackhawk (2)
- # xpo-synopsys-sig (1)
- # xpo-tricentis-continuous-testing (1)
- # xpo-weaveworks-the-gitops-pioneers (1)
Reminder: The final day is starting now – opening remarks and then plenary talks! Join the conversation in #ask-the-speaker-plenary.
I will be sharing our experience "Banking Flow Metrics" https://devopsenterprisesummitus2021.sched.com/event/mqjw/banking-on-flow-metrics-bank-of-new-zealand-transformation-journey?iframe=no&w=100%&sidebar=yes&bg=no later today ; Track 1; see you all there
Hi All, Looking forward to chatting with you all as we share our journey so far coming up next! We hope you can make it! https://devopsenterprisesummitus2021.sched.com/event/n6QF/nationwide-building-society-measure-for-learning
⚡ Let's get ready to welcome @marc.price and @zsolt.berend, here to present Nationwide Building Society: Measure for Learning ⚡
Inspiring story! Congratulations on being a finalist at the Digital Transformation Project award!
Weaponised (and game-ized) metrics.. This is all-too familiar. Same story around the world!
@marc.price @zsolt.berend when do you start the clock for lead time? At idea creation? Or at time work is allocated to start?
Hi Andy, the clock starts when work started (the first time when a ticket moves from 'backlog' state to any in progress state until the chg implemented in prod
and release lead time is the max of all ticket lead times within the release
do you measure any of the up front portfolio planning or product discovery work? i.e. how the larger level asks get broken down into the given set of stories?
This slide is showing "issue created date" not "status" - could you talk more on that?
@isuanelson we measure alignment - golden thread
%stories that linked all the way up to strategic objectives
@meghan.glass we look to see that when an issue is created, does it instantly link into the golden thread alignment to strategy or not
so golden thread = ideation to backlog and lead time = progress start to release?
golden thread is alignment from stories all the way up to strategic outcomes
so current measures are: does the work align and then the lead time for the work from start to release, but no "lead time" calculations for idea to backlog/work start at this stage
@meghan.glass, we do have a lead time measure starting from the time an issue is created, but it depends of the persona for usage. For example a support team working on incidents might want to start their lead time from the time of creation, so they can choose that metric also
Hi Glenn, We created a bespoke survey that we send out to the teams and followed up a few months later to track how happiness has changed over time.
"None of these should be measured at the expense of the other." Truth!
it is alignment from stories all the way up to strategic outcomes
so whether what the teams are working on are aligned to quarterly, yearly and multi-year strategic outcomes
measure: %stories that linked up to strategic outcomes
we partnered with our value realisation office
so practically in Jira: story - epic - OKRs - yearly/multi-yr outomes
VRO = reinvented PMO. From traditional to supporting the organisation on value (articulated via OKRs)
and we link logically to measure end to end lead time
and SNOW chg requests are created/completed as part of deployment pipelines for the most part with some exception manual ones?
our flow metrics is not reliant on how the changes being created - we link it through fix versions
What was the dashboard / visualization tool you leverage? I think it was just meentioned but I missed it.
NM, just saw it answered here: https://devopsenterprise.slack.com/archives/C015DUDD9C5/p1633625038468400?thread_ts=1633625015.468300&cid=C015DUDD9C5
this accelerate metrics view - is this at the app level? product team? value stream? or multiple different "flight levels"?
Hi Andy, We allow the users to filter to their context. So all levels of the society
Mel was one of our early adopters of our product within the intelligent controls team!
We are using a tool called ThoughtSpot for reporting and self service
2its great to see how Craig has implemented our product within his own context in the payments team
This is great. The lead time in the dashboard, how was it calculated? What tool?
lead time: #days from work started to release in prod (snow)
@berendzs Did you need to standardize on JIRA board columns for this to work or did teams retain some flexibility in identifying their state columns (to visualize the flow of work)?
@mring - see thread - https://devopsenterprise.slack.com/archives/C015DUDD9C5/p1633625123472200
Perfect - Thanks @isuanelson !
@mring on purpose we wanted to give full autonomy for teams to choose their workflow as it is unique so no one size fits all
instead we created a map from all states to todo in progress done - this is an essential step to make the calc. agnostic the workflow states as well as to teams ways of working
Do you find that your metrics are only as good as teh data that people provide in Jira/SNOW?
we have seen that, yes. It requires good discipline in updating stories and change records etc
how do you improve that discipline? how granular are the workflow states?
we are agnostic to data sources as we do mapping within our data model to map to todo, in progress and done so are not reliant on all teams using the same workflow
teams have the ability to reflect their unique flow of work
I find that most do not invest that time... that is just tribal knowledge... people do not care about putting that in metrics
we provide a link back to jira within the reporting etc
what we see is that using VSM data from our dashboards teams go on the journey to make their queues and work states visible
they go from simple todo in progress done boards to boards that reflect their vsm
I like this golden thread approach to measuring productivity. It’s simple yet very effective
Thanks for joining of session today, its been great chatting with you all. Please reach out to keep the conversation going!
👏 @marc.price @zsolt.berend - Great presentation - glad to see BVSSH in action and the value it provides the org
☀️ More from Discover, let's now welcome @sakthikasiramalingam and @bryanpayton here to present, The Shift to a DevOps Model While Building Our Cloud Platform - You Build it, You Run it! ☀️
Is this single backlog & rotation being done in the app teams or is this just for this platform team?
@babita.sood We have an operational rotation 24 hrs a day segmented into 8 hour periods. As they incidents come in, we prioritize them and open an internal response call until resolved. During this we maintain an master station log of activities performed and times. If there is a rotation shift we turn over the master station log between engineers. Escalations for SME's happen through our escalation process, but with the consolidation of Ops/Eng we found we have fewer escalations.
We have weekly refinement and weekly planning/grooming meetings to keep inline with priorities and pivots. We also have weekly incident/issue calls for reflection and creation of backlog work for fix-actions and remediation. The individuals on the operational rotation take 'operational' work from our backlog when issues/incidents are slower and capacity permits. This keeps us progressing both engineering and operational deliverables while providing production response.
@halfmoondad Currently this is down on the platform and product side (e.g Databases, Cache, Messaging, etc). We are not directly involved in the AppDev side, but we have shared our success and lessons with that community. They are adopting.
@graham_mcgregor we have found the 'more the merrier' with this approach as the rotations are less frequent and the perspectives are rather 'fresh'.
@graham_mcgregor True, we had to bring more of our engineers into the rotation schedule to enable this
We're using the concept of paired Eng/Ops teams today in my org that share a common portfolio. It works if the leads are aligned. Did you collapse these into single teams with a single leader?
@brianspo Yea, same objectives. Myself and a Senior Product Owner oversee all products/deliverables within this space. However, each product has its own PO and PA. Operations has a Ops Manager that is consistent and we simply rotate the individuals in and out. All of our leaders are aligned and were involved in this decisions.
I think what is resonating for me here is the product thinking, which means you need to start further upstream. How you're organized to build and deliver is a downstream effect of the upstream org approach.
Agree it’s about the product orientation instead of a project focused approach
@bryanpayton - Are slides for this talk avaialble? I didn't see them in the library.
Yes, they will be available along with your video in the Video Library. If you're okay with that, I'll make sure they're included.
BTW, I'm really appreciative of the presence the Discover folks have had at the conference. I've found your talks really relatable from the standpoint of where I'm at in my (much smaller, academic IT) org.
Appreciate the conversation everyone - thanks for attending our session!! We will get the slides published.
🔵 As we return from the break, we now welcome @cmchugh and @jdirks from TrueBlue, here to present A Force for Good: How TrueBlue is "Accelerating" Connecting People with Work 🔵
Hi Everyone! I'm excited and honored as well to be a part of this great event.
The downward spiral is one we often see in transformations
We really felt "landlocked"...almost held hostage by forces many felt powerless to change.
@scott.prugh Yes, it's a play on TrueBlue's Flywheel of Innovation we discuss with our Board and Investors.
Love "pit of despair" and have seen that movie a few times in 30 years of tech.
The performance metrics in Accelerate have been so critical in many transformations! @nicolefv @genek @cmchugh @jdirks
We wanted to avoid two major impacts here: teams opting out with reasons (e.g. we're legacy, our customers won't allow it, etc) and also avoid gaming the system with proxy/vanity metrics.
Accelerate talks about how important it is that teams set the commitments and pick their tools. We needed to embrace that.
Not setting hard targets for the measures is a great approach. Let the teams measure to learn to then set their own goals.
The comment about some teams being green-field having different goals from some that are legacy is very true. We don't want a comparison between the teams in goals and targets!
One team had really good CI/CD already in place, but were bottlenecked by cumbersome CM policies and procedures.
They set to work with the CM team to define a new, automated "Standard Change" policy that enabled them to deploy daily.
I love the compass analogy! The key is to teach teams to measure to learn to improve and coach them to do that!
Yes - we talk a lot about compasses over maps. Get on the field of play, get more feedback loops and use those for orienteering towards the end goal. Maps tend to promote waterfall approaches.
Some legacy teams found a way to do some even very simple improvements...many legacy procedures were simply done again and again just because no one even questioned them anymore.
But added up, they were spending hours in manual tasks and this created fear, error, and declining motivation.
Yes - we focused thematically on making it easier to get work done -- and automating everything.
The big learning for me, is how shifting into smaller release, suddenly showed up in the other metrics. KTLO came down because there was less to troubleshoot each day, or plan for rollbacks, or other non-value add activities.
We know teams like innovating, and this "broke the logjam". Forces that folks couldn't describe well let alone affect for good, were given clarity in "Accelerate"
Yes. Start the journey where ever you are! Don't get discourage by: 1) the size of the challenge or 2) where others are
Remember: "The best time to plant a tree was 20 years ago. The second best time is today."
We are pushing from the bottom up, but there seems to be a lack of support from the top. This causes our message to get missed sometimes. Everyone is too busy to improve
In my first DOES event in 2016, I think how I felt @jason.cox were so far ahead. How can we even begin. But I learned everyone is on their own journey and start where you are.
@ganga.narayanan The Cobra effect is a frequently cited example of Goodhart's law.
Thanks all! We are happy to share our journey and learn from others along the way!
It's important to be aware...measure many things, avoid universal hard targets. Encourage transparency, learning....a Generative culture.
🎧 Excited to welcome @jillmead2018 and @erickramlinger who will share, From Grass Roots to Brush Fire: Creating Massive Organizational Momentum Using Communities of Practice During a Global Pandemic 🎧
Was going to say waiting for the brush fire to start, but wasn't sure if that would be appropriate 😄
Wait. Is this the conference call we got off of this morning?
Hah. If I had a $1 for every time I heard: "That won't work here" I wouldn't still be doing this DevOps thing.
If I had $1 for every time I heard that, I'd be able to buy a gold chain like the one Eric has!
Yes. Model the behavior you want to see in your people!!
@erickramlinger that is some first-class bling you have going.
@abd3721 - THANK YOU! Of course, my dog had to literally take a bite out of the corner. 🙂
I got involved in this community because of the: 1) learning, 2) energy, 3) hope
@jillmead2018 great to see you presenting again this year. Always very engaging
Shoutout to http://Mural.co and other digital whiteboarding tools
I stopped saying “eat our own dog food” in part because I don’t like dog food…we’ve been using “sip our own champagne”
I say "drink our own champagne" a lot, but that's probably because I rarely sip. 😅
Honestly - I think many lose steam because it is so infrequent. We were consistent to start. We allowed the community to build when it needed to the most.
Did this CoP have any people exclusively dedicated towards its cultivation or was it just part time for everyone?
Jill and I were dedicated to it, and other folks jumped in as volunteers as it grew.
Coming from Target and knowing the crew that went to US Bank, how much do you attribute your CoP to what Target had done?
@nicole.forsythe I think it helped to pull from past target experiences and all the creativity that went into so many things. But I never experienced a super engaged, effective product cop at Target during my tenure there. We were so hesitant to create yet another cop based on experiences with them.
@nicole.forsythe We decided to throw this on its head. Create experience that overtime was morphed into something special. Honestly - i dont know how this forum would have morphed if we were all co-located. Part of me thinks it gained alot of steam because it was remote.
The "Problem Is" that many developers don't understand 1. How to Test 2. Why to Test 3. That it is their responsibility to test. 4. Others will test their code AFTER they have tested their code. Once they experience the POWER of testing their own code, they are hooked. But you have to get them to do it, in order to get them to love it. A CoP could help get them immersed in testing!
@nepobunceno we thought it was. Start weekly to establish familiarity, routine, and find a sustainable pace
We're about to start a community, and we have them set up monthly... time to revisit that then :thinking_face:
Great point. We had a few stop and starts and found that we needed some initial consistency to gain momentum.
Can relate! We've seen many starts and stops every now and then. People getting busy; reorgs; changing mandates etc. -- and also because these are seen as voluntary.
@jillmead2018 & @erickramlinger, I feel like CoP done well is an underrated topic. I've seen pockets of CoPs done well in my travels but they are rare!! I have found the hardest part in a CoP, beside momentum to start it, is evolving out of a solely consumption-based participant engagement to a balance of contribution-based. In your experience, have you seen this challenge & any suggestions to reach equilibrium?
@kxbres - great call out. Every CoP featured different members of the organization. We also did "feedback blitzes" where volunteered to show off artifacts and the CoP would ask questions and share feedback. A few examples of how we tried to create a back and forth flow.
I often say: I have yet to meet a CoP that works well. I wonder if I need to have personalities like @erickramlinger and @jillmead2018 to make a CoP work well... My question is therefore: Is a CoP model sustainable?
@tashfeen_mahmood one of the CoPs I've seen work really well is with great moderation that believes in drawing out the expertise available in the "hive mind" so-to-speak and enabling many people to share their experiences/SME'ism rather than relying on the moderators/organizers of CoP. Another was feedback from the community & offering enablement from the CoP to address the feedback/needs. Also..rules of engagement that are established & understood so existing and new participants know how best to engage/gain & contribute value as quickly as possible.
and my question is: how many people can do it well? I can tell just by listening to Eric and Jill that these are two extraordinary people, leaders, thinkers, personalities, DevOpsers, Product People... How many of those do you know? By the way: I can tell that you folks at Vanguard are also geniuses. I saw the session from another Vanguard person on Day 1 and I am very impressed!
@tashfeen_mahmood How bought in is your organization to moving to more of product, business agility and/or devops values, ways of working, ect Do you know anyone whom are natural leaders for this movement? Our goal is to create more leaders in this space internally?
we are bought in... However, I will also say that if you take 20 executives (from any organization) and ask them what DevOps means, you will get 21 answers. I would like to talk to you more about the cultural topic you brought up related to leadership. IMO - Leadership grows more leaders. That is something we can improve at my company.
Was there a particular format you used? Did it vary per session?
Andrew – varied per session, but through experimentation, found some helpful repeat formats: feedback blitz, stories of impact, and breakout group small discussions.
So one session would be a “feedback blitz” and another one would be “stories of impact”?
How do you create longevity? What happens if you two leave the org at the same time? @jillmead2018 & @erickramlinger
@nl_does One of the key things is that ending a cop isnt necessarily a sign of failure. No need to have a cop for the sake of having one.
Resonates in many ways. CoP : Everyone wants them; everyone talks about them; easy to start; but difficult to sustain!
Hi @ganga.narayanan – no books in this CoP. We have a book club, but found that we wanted to focus on the human experience and hear eachother's stories.
Yes!! Telling stories is so powerful!! That is one of the reasons I love DOES!
This talk gives me hope for the promise of CoPs!! Stay with it, listen, empathize, adapt, practice vulnerability!
A Product Community of Practice or a practice to build a community around products, or a product that was the community of practice? 🙂 Inspiring video!
Great perspective. All of the above. Practice to build a community around our product. I love that!
We celebrate almost everything including failures - EXCELLENT
:kiwifruit: We are very honored to welcome @paul003 and @lbmkrishna here to present, Banking on Flow Metrics – Bank of New Zealand Transformation Journey :kiwifruit:
3ha ha ha 2:30 Everyday - for DOES - It DOES NOT MATTER
I have 11am to 1pm blocked in my calendar so if I need a recharge nap I can take it.
Amazing talk on CoP!! @jillmead2018 @erickramlinger Learned great ideas....Thank you!
Really interesting talk.... got my brain turning on how to bring some of these ideas to my team.
Definitely different from other talks. It was great. Obviously your CoP's are different as well.
👏 great stuff @jillmead2018 @erickramlinger - Love hearing about the community you are building at US Bank
@jillmead2018 and @erickramlinger That was just awesome! I thoroughly enjoyed it! I would attend any meeting, any CoP with you guys any day!
I want to introduce @paul_littlefair to our DevOps Enterprise Community
https://www.canstar.co.nz/online-banking/canstar-awards-bnz-as-bank-of-the-year-2020-online-banking/
Such a great presentation on CoP @jillmead2018 and @erickramlinger. I particularly enjoyed the Dance Jam at the end- but also some great ideas to try out with our teams. Thank you so much :)
Thanks @paul003 So excited to be listening to your talk! Amazing. And currently being in a bank 🙂
1Love the “Digital First, Human When It Matters”!
@scott.prugh - Your talks are inspiration for us; Watched all your DOES talks and interacted with you 🙂
Absolutely @scott.prugh! All our teams are embracing DevOps, including the Mainframe teams. Learning journey for us all!
Sigh. Dependency management, WaitTime(queues) and knowledge all degrade exponentially. Just hiring without understanding these bottlenecks is like lighting money on fire.
There we go @paul_littlefair - Bold moves 🙂 and one of the moves
"We took out our PMO"
You have to expose the visibility. I think Team Topologies creates a good framework for exposing visibilities artifacts. Dojo's also go a long way to make work visible.
And accurate I think, as I have seen a lot of PMO leaders become champions of Flow!
Yep @cncook001 So easy to understand the concept, but even in 2021 so few corporates actually doing this. We all report the flow at the end of the system, but so few exec teams look across the whole system.
how did you do that? -- Make time for continuous improvement
We are so grateful to @jwillis! Even if you are admitting to plagiarism :)
"measuring only one area of the value stream is like only using two inches of a 12-inch ruler" - couldn't love this more!!! ❤️
We didn't wait to get everything perfect before starting.
@rshoup - You are an inspiration. All your thoughts and talks - actions - love it
@mik Fyi.. I pushed flow metrics hard in my final report. I do this almost everywhere I go... (big fan)
It certainly was @jeff.gallimore. A great leap of faith in the opportunity of DevOps and Flow from our business colleagues. Relied on lots of communication and building trust through developing credibility, reliability and intimacy.
Happy to hear that @paul_littlefair! And such a key point, don’t wait to restructure Jira, get to 100% Agile, etc. Making work visible is a journey where you want flow, feedback and learning for every step of the way.
In case anyone is interested.. this is the model I used for the analysis at BNZ.. https://www.infoq.com/presentations/qda-qualitative-data-analysis/
This is my main area of interest right now, CxO and board level reporting. So great that you are sharing out your learnings on this @lbmkrishna and @paul_littlefair. And @jwillis time for us to compare notes again I think!
@paul_littlefair another bold move - introducing all together a new role VSA (not in NZ), He was so sure that we need this - I would like to acknowledge Paul's vision for us and a true servant leadership
very consistent with what @steve773 preaches. i think it’s “see, solve, spread”
@jeff.gallimore @rshoup That’s 100% right. Those who learn more, faster win by the ability to bring new and useful insights into practical use ahead of rivals. That’s the “high velocity edge.” The basis of all learning is recognizing a problem, swarming on it to investigate it and experiment around it to get to a solution. The huge multiplier is sharing what’s discovered so the individual insight gets systemic application and so the individual discover collides with others to build compounded understanding. That latter part is the basis of Jared Diamond’s Guns Germs and Steel (which I use as the intro case in the chapter on knowledge sharing).
1Thanks, Steve. Your insights and clarity in this area are fantastic. The Diamond reference has me intrigued; GG&S has been one of my favorite books since before it was a big deal 🙂 .
If all CTO's were like @paul_littlefair the IT world would be a better place...
Always keen to help @ckissler. The DevOps community has given so much to us, so I'm delighted when I get an opportunity to pay it back.
Thanks @rshoup, but it’s what BNZ is doing with the insights that is so amazing!
@mik “and how quickly”… from an outside perspective, what sets them apart from others who don’t move as quickly?
Yes - we introduce 6 types of BNZ flow items and map them back to Flow Items
you mean the flow distribution graph. Yes - for some team - all green, some are all amber - Making the type of work visible also challenge.
Argh!!! Don't outsource unless it is a non core capability.
Collect data without committing to change is a good place to start (sets team baseline)
Thank you so much for your amazing presentation, @paul_littlefair and @lbmkrishna!!!!
Distribution is key. We need to educate our business colleagues that the work isn't just features.
@jeff.gallimore That is a great question and almost seems like a talk topic of its own. Directionally I think it is a very effective pairing of exec leadership and tech leadership, rapidly operationalizing the visibility, and really interestingly to me their organization approach to learning. As BMK alluded they created an entire learning portal for learning Flow!
@paul_littlefair I would love to learn how you leveraged that. Did teams make adjustments based on that? (as an aside, I've always wondered why "Continuous VS Improvement" is not a distinct flow type.
We didn't wait for the stars to align:sparkles:
Thank you @genek for the opportunity to share with our community.
Actually maybe it’s what Paul just said: “Not waiting for stars to align”. I think that;’s it!
Having teams be OK with data problems and visibility, and creating the psychological safety to make that a journey.
OK, that’s my main take away, “not waiting for stars to align”
So, sorry, but this is just an awesome talk, and thanks to @mik for your book and vision
Read @dominica Making Work Visable. One of my core tools for helping clients.
I will tell you once you put the tool like VIZ, that exposes the JIRA and other tools data issues - The teams are in SHOCK;
@paul_littlefair It’s been so amazing to interact with @lbmkrishna from the very beginning — I’m so delighted that he’s presenting with you here! THANK YOU!
Making Work visible and also Make the problems/data issues visible
Thank you @jonathansmart1 - The days are not that far - Better Value Sooner, Safer and Happier
Thank you @chris.gallivan421 - please share your feedback 🙂
Thanks @rshoup - would love to catchup sometime to talk about "Architecture" 🙂
Insightful talk - thank you! In my experience, data speaks for itself in showing what's inefficient.
Thanks All. So grateful for the opportunity to share. Please let us know how you are going on your journey :)
Thank you @tom.wojtusik for all your support, much appreciated
👏 As @jwillis said, this kind of leadership moves mountains. Honored to get to collaborate with you @lbmkrishna and @paul_littlefair.
1Great to see fellow kiwis presenting at DOES! Well done @paul003 and @lbmkrishna. We've been on the same journey so will definitely compare notes!
Reminder: The final plenary sessions are starting in 5 minutes. Start making your way back to your browser and join us in #ask-the-speaker-plenary to interact live with the speakers and other attendees. https://devopsenterprise.slack.com/files/UATE4LJ94/F01D34MC2KS/image.png
@allspaw in your journeys, what has been the most effective method of publishing incident learnings? Any particular method that was excellent at capturing metadata for future engineers to sort by?
I’d say the content of write-ups is paramount…the specific platform (Google docs, Confluence, etc.) matters very little if whatever indexing/search doesn’t have much to index.
5