and FYI for @s.nikiforov

haha, you’ll get used to it, just a matter of staying longer at home on one hand, at the same time, accept how it flows the second time, also on stage for real you’ll make these mistakes. DOES is not the apple event

My squad is called Razee. https://en.wikipedia.org/wiki/Razee We spend a few hours scrolling through a list of Nautical terms to find it. We think the definition is quite fitting for our team.

We completely replaced Jenkins for our code delivery. We used LaunchDarkly in conjunction with our Razee CD tools (http://razee.io, shameless plug)

We still use Jenkins for other tasks (mostly operational)

OK; Thnx!

@mckaymic: is this Razee the tool you mention in Phase 4? The one built on top of LaunchDarkly and used by the devs to manage the changes and integrate with SNOW?

It's an internal app we call RazeeFlags. Since we have created it, LaunchDarkly has provided a similar integration with SNOW so eventually we will sunset RazeeFlags

okay, cool. I was about to ask if you were willing to share the source for RazeeFlags haha. One of the big pain points in my org today is SNOW tickets and how to deal with it when moving to DevOps, etc. Therefore I am always interested when someone mentions SNOW in a DevOps talk and some kind of integrations

We could open source it. One hurdle is that our app doesn't communicate with SNOW. We have another service within IBM that provides us an API to create and manage SNOW tickets.

I see, so there is another component in the middle. Gotcha. Would be interested to see an architecture diagram of all the things you talked about here (my solution architect hat speaking now 😉 )

@rodriguessemensati.e Feel free to reach out to me (<mailto:mckaymic@us.ibm.com|mckaymic@us.ibm.com>). I'd be happy to talk about this more with you directly.

sure, will definitely do! Thanks!

Are you asking about it when the service being managed is async, or the feature management tool itself?

The services being managed - ideally one service per feature, though that's not very realistic, but it would look like an ideal scenario to me. If features are implemented in async services, feature flagging would seem to be more straightforward to manage. Many tightly coupled sync services depend improperly on sequence of service calls to work. Feature flagging one of those services might break the chain and cause undesired side effects.

There are certainly approaches out there which leverage deployment and validation of feature changes through routing of traffic to versioned services, in which I do think that this becomes quite relevant, but I think as long as good service contracts are established that feature flags can be leveraged effectively whether sync, async, or monolith. When a feature change impacts multiple services, the flags let you release the changes to each part of the system individually and then rolled out in a coordinated manner to avoid those side effects

I think that when you get legacy code in the conversation, good service contracts (or really any artifact that clearly expresses intent) are the first thing out the window. It's really a lot of small judgment calls until your code is stable enough to be played with more. I am very interested in using feature flagging with a product that I'm working with, but this may not be the first thing we do. Thank you for your thoughts. I am enjoying the presentation

Thats fair, even with all of the tools available the cultural aspects of maintaining strong contracts are challenging. As long as you have an effective implementation, I find that Feature Flags are almost always a safe first step though (of course, I'm biased haha). Effectively any change you are making in the code (in a single service or across services) can be made safer by placing the ownership of its release on the developer or team who wrote it

Since the changes are going to happen anyway 🙂

At least there aren't any bad habits to fix in existing telemetry!

always keeping a positive attitude, I like this!

Adventures in DevOps! https://podcasts.apple.com/us/podcast/adventures-in-devops/id1475784710 We discuss a wide range of topics facing DevOps teams, from organizing DevOps teams, release management, cloud services, and the latest security trends. (also replied in the other thread)

Hey, Bryant from HCL Software DevOps here! We absolutely do, stop by our booth and watch the demo of HCL Accelerate. The relationship between complex data is key to drive advanced metrics.

You are going to be able to see end to end and then take action on those items day to day with actually seeing your Value Stream and using it.

Yes! You can see the constraints, waste etc but it takes experience to know what to do next

Sometimes the truth is so bad, that it demoralizes the org - it’s too big to face

That's where outside help can be a major boost - there's always something small to start with and build from. If you can run a mapping session, there's probably enough enthusiasm to make some progress.

Its really about the culture of having that conversation. There is a million things that teams can go do, lets get those prioritized!

Hopefully it isn't being simplifying that far.

Yeah that would be a big mistake :\

We want to simplify the data gathering so you have more time to focus on the culture and improvements. Then be able to track if those improvements are working

Companies (ahem some vendors)....of course they are. VSM is much more than that of course.

I just helped write one this week.

Almost every day! It's my full time job!

We do a VSM with every team with do our embed/bootcamps with. They really help bring up waste and make it more visible for the whole team to see what is going on and helps drive better conversations with the teams.

Just recently have we started doing this. Implemented with vendor assistance. Very informative to expose gaps and waste in current processes.

We’ve done several

I do not know how we did software without value stream!

Value stream mapping is definitely becoming the norm, but it's not a one time event. The process is iterative and the map will change (for the better) overtime as gaps/problems are identified.

That is a great point. we do a current vsm at the start and then also give the team a to-be vsm of the items that our team would focus on to remove waste and what that vsm would look like and then drive change toward the to-be vsm and then document the changes of time. Its also a good way to be able to call out the victories the teams make toward CI/CD

The big eye opener for most teams is when you calculate their Value and non-value times along with the flow efficiency of their current vsm.

Thats a great way to do it, get a baseline then set a goal of where you want to be in 6,12,24 months. Then track it and make sure you meet those goals.

Yeah. We even take it a setup farther and help them code the delivery pipeline on one of their repos. We just finished one that took them from getting from pr to stage in 5 days down to 40 minutes.

PS: if anyone's interested in chatting more about this stuff, come out to the lean coffee on value streams soon!

hes actually sitting at a piano absolutely ready to riff once this session is over...

LOL

😆

Thanks @richard.winslow 🙂

Thanks for taking the question. Sorry for the Garter-speak. Former company was big on G so I drank a lot of kool-aid...

The broader the view, the more likely you are to find the real bottleneck, so 'end-to-end' is a key factor

There's a breadth view and depth view. The breadth view is what we want to enable everyone to see and we can best enable handoffs between stages. That's the full end-to-end value stream. However, we don't want to get too caught up in the depth view. This can often derail a value stream mapping session when the specifics aren't important to the great vision. (Example: How many reviews are required to merge a PR?) These are good areas for those specific team leads to hyper focus and refine their own processes.

Absolutely, you get diminishing returns from micro focus, and you risk the attention and energy of the participants

Once the team understands how to use the framework, they can dig in where they want more detail

I'll cover for Steve while he's presenting....we find when modeling value streams, each product team needs one core value stream defined. For larger teams, it's often helpful to have child value streams (subteam or squad level) and/or functional value streams for teams that may cut across multiple product teams.

thanks for the great response!!! extremely helpful.

thanks for the great response!!! extremely helpful.

Even better if continual engagement with security, risk, compliance throughout the product development lifecycle.

i.e. from risk assessment to release

Absolutely Myles! Imagine getting developers involved in things like threat modeling so they can better understand exposure and risk.

and imagine security better understanding risk from a business perspective and not just vulnerable/not vulnerable.

Agreed - at my previous place of work we established cross functional "Safety Teams" including all governance, risk and compliance support for Value Streams

I agree 100% with this. And it’s not just “take this security tool and make devs run it in CI”. You have to set things up so that it saves time in the end — doesn’t just add another hurdle. Doing this at Dev time needs to make the security review easier. Bonus points if your tools can help devs be more productive (e.g. by flagging non-security issues like reliability / crashing problems, etc.)

And then we could have a dashboard of dashboards!

recursively! and if it gets written in Clojure...

This is interesting as I feel we've generally heard that "VSM is a process problem, not a tools problem". However, I agree there are situations where devs where can't perform because they don't have the tools to support them. From personal experience, these are types of issues come to light during retrospectives or the value stream mapping process itself. Once it's tracked and monitored, then the finer adjustments can occur.

Interesting. I wasn’t thinking about developer tools. I build Products for internal team members and i definitely hear a lot about “not having the right tools to do function x” and when we do discovery with these same customers, i see an abundance of domo cards and dashboards.

What dashboard is your CEO interested in seeing? Just a list of release management requirements? Or the possibly the set of epics that were/weren't delivered?

It worked out over time, that was the joke at the time was "don't tell me the good, tell me who I need to talk to to get things fixed", it was an education for him over time.

We integrate with over 50 tools today in the SDLC! I am currently working with a customer on Aha! and the use cases. Lots of exciting features around tracking larger initiatives in the Value Stream.

Grab some candy...

That is a good question - is certainly lets you know about popularity within the organization. Personally, from a security perspective, I would want to know how many VULNERABLE components were being reused over and over again.

true points I wanted to pivot thinking toward the semantic flexibility in creating value stream components fostering reuse and along your ides better oversight in terms of security management because of less components overall.

Reuse in general lends itself well towards robust and more individuals with insight into how to manage and troubleshoot the process. For example, standardizing around a AppScan proces means there's something the entire organization can rally behind and more easily understand. Whether that's interpreting the results or the standard of High/Medium issues to rectify before release. It sounds like you are starting to reach towards a DevOps or VSM "scorecard" which is an exciting are to think about. How do you quantify best practices down to a number or an A/B/C grade? You should keep an eye on HCL Accelerate. 😉

There is great value in "build once, use multiple times"

Currently we are doing the metric analysis to provide those key insights to the business but yet to put dollar signs to that. We have it on our roadmap and would love to discuss more.

for some of our Ops side development we have started to put in counters to see how often some automation is triggered and adverted downtime, so that we can put some dollars to that, and then get that automation added to Product Development so that we can show the value of the defect reduction.

To add on to that, you can check out more info about HCL Accelerate in the https://doesvirtual.com/hclsoftwaredevops.

thanks @elise-yahner

Trends over time is key to seeing ROI on these initiatives, we can help out there!

Yes, ROI, is an estimate till it is realized…

As a type of silo, we wrote our own to add clarity to other teams VSM.

One of the key first steps is getting everyone in the room together to talk about the Value Stream. Make sure key stakeholders from those silos are there from end to end. Do it on a white board first, see which stages need some help.

Hello João - that is a really important part of the overall process. The short answer is that it depends on how you set up your stream and the phases that you include in it. As a basic example - people can have a "backlog" phase and then track how long it takes for things to move from the backlog to something else. That process likely involves many of the things you mention. If you find that it is taking a long time - maybe you then add in additional phases to break it down and get better understanding of where those processes are getting stuck.

I actually imagined it would be good to have a separated tracking of the analysis process from the implementation process

just because they have different dynamics

It certainly can be, and you are right that the dynamics are different. The point here is being able to have the flexibility and visibility to make better decisions. I have not seen a lot of Value Streams that specifically call out the stakeholder discussions and decisions to date - but you potentially COULD if you found that there were significant delays there.

it's a great point Ben - have you found a good way to make unplanned and planned work more visible together?

No yet. Mostly because our ITIL Ops tool is "tuned to" Ops work and our Dev Kanban tool is "tuned to" Dev.

I kind of feel like 2020 has been more of the "W" shaped human... up and down and up and down and...

facepalm we really don't need a term like 'versatilist' from Gartner. I agree @brad499, it's multi-disciplinary teams with T-shaped (or comb-shaped) people, as we've known since the 1980s in Japan in Product Development (The New New Product Development Game, 1986)

@jonathansmart1 I know... I find the term cringeworthy, though that's not the first neologism G has come up with to repackage and brand existing information. I love the references you come up with and share.

You absolutly should then you can show the business with a metric like Distribution what is really happening in the SDLC. My favorite story is helping customers bubble up these technical task that they think are "free" 😉

ha, got featured live, awesome, thanks @helen.beal. I was multitasking and then heard my name and I was like "what, is that me? did I ask something and forgot?" 😄

If we are measuring individual developers by story points, there's more basic things to fix than testing. 😄

totally agree @bryan.finster

Lol - I love that show :)

The old ways are incredibly resilient

Especially when they are roles and not tasks

check

yeah, I like it too, but I have to stop my brain from asking about what powers their movements... ow ow ow ow ow

https://www.youtube.com/watch?v=5Z6EvBFblvg - for whatever reason this comment reminded me of this

Honestly during covid it has become more real

Yes, yes it has

mindless hordes

the pandemic has exposed a lot of "interesting" features of human psyche and systems...

fodder for future analysis for years to come

In some ways human brains are the holographic principles representation of the zombies feeding off of them - chaotic, untrackable in ways, irrational, greedy, maximizing input for unknown output,....:rolling_on_the_floor_laughing:

centralized command and control vs decentralized individual choice and control - which experts do you listen do... quality of data sources...

how did "buy toilet paper" become a thing? lol

Value Stream is looking at the flow of value rather than a methodology. The learnings from Value Stream should help drive what Agile methodologies your team uses. Metrics and stage times are going to give you visibility if your methodology is working for the team. Kanban is the dream right?

Indeed. Seems like if I manage via Kanban, it would be easier for HCL to illuminate things for me

A value stream is the flow of value end-to-end, agnostic of role silos. A kanban board can be used to visualise the value stream. There are points where work is being worked on and work is waiting. A kanban board is helpful to visualise the knowledge work in the system of work

Most kanban boards (irrespective of methodology that are team are using to get work done) is usually a narrow window on the end to end flow (e.g. just the IT bit)

It's not about a methodology. It's understanding and visualising the system of work

On a factory floor you can see it

With knowledge work you can't

Kanban boards are great, but as Jon points out they only cover part of the value stream and they also require manual updating so are often out of sync with reality

Usually cover part of the value stream

I set them up to cover end to end

This shines a light on the lack of flow upstream with PMO and Finance and Steering Committees

e.g. one org that was saying yes to twice as much work as the throughput of the system of work. They were oblivious to it, until visualised on a giant kanban board / VSM

that's a good strategy to have end-to-end kanban boards, although without connecting the kanban status with additional data like CI/CD and quality data it's difficult to answer questions like should I release this code, where are we slow and what versions are in each environment. By building relationships across all of our tooling, we can start to not only answer these questions but help proactively manage our work.

Yes. Optimise for Better (Quality), Value, Sooner (Flow), Safer, Happier.

Not one at the expense of others

@jonathansmart1 I recall images from youre previous talks of maps that took up a whole room. Easy to do if everyone is physically present. In the model where everyone is WFH, how do you visualize and communicate such complexity?

A large virtual board.

I use Mural for this, but I always try to make maps that only cover 15-20 steps to keep it usable and minimize the cost. You get diminishing returns from detail, and in my experience, a simpler map with more data is better than a huge map where you can't collect as much.

I also create maps at different levels, so there's a separate map for release and one for a single team sprint, and each team has their own map

Virtual boards are much, much better than physical whiteboards. The team can contribute in real time, vote anonymously, add comments, and zoom in where they like, then afterwards it can be exported instantly or shared with anyone in the org.

@jonathansmart1 your point about upstream visibility is so important - it's always the biggest surprise and the most wasteful part, because it's never considered as part of the process

@steveelsewhere https://youtu.be/R-fol1vkPlM

Yes! Saw it live, still one of my favs :)

Anyone interested in talking more about all this, come out to the lean coffee on value streams soon!

a value stream map is a great place to start. once you get an understanding of process and tooling you can start to model it in a VSM tool (like HCL Accelerate). You can get started with just a work item mgmt tool like Jira and then add CI/CD, quality, security and others as you go for more visibility.

@robert.cuddy My quote for the day. 🙂

It was pretty solid. Well done. It reminded me of one of my favorites from Rick Warren - "People don't change when they see the light. They change when they feel the heat".

that is awesome

never heard this term, what does it really mean?

Fuzzy front end is the concept, define and design folks

Everything before the coding

Typically varies wildly from company to company

got it, thanks

Don Reinertsen talks about managing the Fuzzy Front End on this podcast: https://projecttoproduct.org/podcast/don-reinertsen-part-1/ - it’s super insightful!

Love Don Reinertsen !

There is a talk by @halfmoondad later in the conference on the fuzzy front end

True that! I'm thinking I need to start a list...they just roll out because there are SO MANY SILOS

What toy store is he going to with only $10?

Sounds interesting but must prioritize session talks and slack catch-up over demos while they are happening...

And when you are done that step, keep looking, asking questions, challenging the existing. There is always room for improvement when you look at the end to end flow as every system has at least one constraint.

Thanks lovely! 😁

Thanks, @steveelsewhere

True story!

Well said. I feel like test environment management (or lack of) is a major Achilles Heel of any DevOps transformation

Yes, In addition to the Test Environments such as servers, middleware, configuration etc. I think a lot of organization miss the data aspects of Test Environments - have proper set of test data that you can consistently and quickly reuse is a critical.

That's a constant struggle, as well, and lack of data causes app dev teams to do everything possible to skip good testing, or pole vault over one test env into another.

That pole vaulting is something I've seen cause immense amounts of pain. Test data management is an art. Phrases like "data masking" get thrown around like it is easy.

It's one of the harder things with larger complex environments - especially when you have multiple applications and your micro-services are stateful - creating sets of data and being able to reset the data becomes complex. It's hard to resist "cheating"

So if the SDLCs from the vendor and the internal team is in sync its possible to excel in the main DevOps KPIs? Any advice or specific Anti-patterns you have seen when using an external SW Factory?

Hey! (still very new to the devops industry) - but i'm interested in why you'd say that?

I believe that having the separate team called "DevOps" team has nothing to do with the DevOps methodology. It's like having the "Agile team". Or "TDD team". If the organisation spends money on something called "DevOps team" - chances are that I won't be enjoying working there.

@nick.kritsky Have you seen much of that in the wild?

@nickeggleston yes, it is a quite popular concept. As many people mentioned here - usually it is just a fancy name for the team that manages developer tools and/or PaaS/IaaS clusters.

What role titles are u seeing for those focused on the cultural/organization transformation in the wider sense of DevOps?

Very good question ^^^ I'm also interested what a true DevOps position would be called or entail.

Nick, I liked this one: "DevOps coaching team".

Ah, right. That makes sense. That wasn't necessarily visible in the diagram....

I agree, and I have seen engineers apply that know the tools but do not understand the concepts or the culture of DevOps..

True, it seems that it is replacing Release Engineering role titles.

Platform team, Tooling team. Yes. SDLC stack team.

I use Developer Tooling team, although the SDLC Stack team sounds great.

I like to define the team that supports the teams pushing code to production (and creating the business value) as Developer Tooling team [although someone suggested SDLC Stack team]. The team running the DevOps methodology is the business facing team. I normally like to see the management of the Developer Tooling working with the Operations and Information Security to refine the devops process - and a sucessfuly framework around that is something like BSIMM which help sframe the security aspects of it

Great, i really appreciate your response! Makes a lot more sense.

@nickeggleston I asked a similar question to what you were wondering in one of the earlier threads. Hope this helps.

you can look into this that i used in the past : https://www.parasoft.com/products/parasoft-soatest/

Thank you!

https://www.sonatype.com/software-supply-chain-2020

Thanks Dessie

thank you!

@t.girouard

Hi Sean, I think this video will help explain it a bit: https://www.youtube.com/watch?v=Lr1_kEvY-aE&amp;list=PL_mckFOFnjHmB5xgonrUlaICMFOR-fTyo&amp;index=3&amp;t=50s

Thank you Tim, appreciated.

@e.chisholm

@w.hayden ^^ can you provide details to @ali.zahor’s questions above?

@ali.zahor Thank you for this question! Our LiveCompare technology can detect changes in an SAP environment and we are working to extend this capability to other technologies. However, at this time, we have not yet extended it to Java and .Net environments.

thanks @e.chisholm and @w.hayden

You’re welcome, @ali.zahor!

Thank you!

We are using ServiceNow for End-to-end automations and creating self-healing applications based on tickets. We have automated the Change tickets too.....

That's great. One of our observations in creating Transposit is that ServiceNow actually did a great job (especially given when it was created) around managing the communications and business processes. But then with the rise of agile and DevOps, we see the more developer side of the house not really having the same sort of tools for process and tracking. So we really want to bridge that gap.

Thank you!

These are so great @laurel that I shared them in the #happy-hour channel too! Thanks for all of your hard work!