Fork me on GitHub
#ask-the-speaker-track-3
<
2021-05-18
>
Ann Perry - IT Revolution11:05:09

Welcoming @ashley.noble in a couple of minutes, joining us from Australia today!

Alex Broderick-Forster, IT Revolution, Event Staff11:05:14

I see 82 watching live! Post comments and questions here for @ashley.noble

Matt Cobby (NAB)11:05:33

I didn't quite catch there - are the templates runnable applications or examples to get started

Matt Cobby (NAB)11:05:59

Do each of the templates encapsulate the whole stack or are they also available by component e.g. logging

Ashley Noble12:05:31

@matthew.cobby they represent the full stack.

Gus Paul - Morgan Stanley (he/him)12:05:35

They seemed to be language/tech stack based...do the templates also include the data layer

Ashley Noble12:05:18

@matthew.cobby, so if you create a C# WebApi pipeline, when it finishes creating, you have an endpoint that you can hit that is deployed into QA.

Tim Wyatt12:05:52

@ashley.noble Is this a "build to attract teams in model" or "force teams to use model"?

πŸ‘€ 2
Matt Cobby (NAB)12:05:11

Central build system has access across all cloud subscriptions? What was the security split? Non/Prod?

Ashley Noble12:05:19

@gus.paul they provide some basic abstractions like repository pattern as a guide for teams, but not the actual DB stack, as teams generally have a preference for how they want to talk to there dbs

πŸ‘ 1
Matt Cobby (NAB)12:05:42

How do teams stop drift with each pipeline over time?

βž• 1
Matt Cobby (NAB)12:05:56

Forgot to use threads πŸ™‚ What do I mean? You create a pipeline that is compliant with AutoMate. Day 1 - all is good. Then over time, teams make a change here, make a change there. Somebody hire a DevOps contractor that goes wild and changes it all. Is that kept in line with a Pipeline of Pipelines approach or through a software clean room (Capital One) governance model or maybe it is just natural evolution

Ashley Noble12:05:32

@matthew.cobby, sorry missed the thread. (great to have another Oz person here as well πŸ™‚ ). So the pipelines themselves don't drift, and when we provide big updates they "re-execute" their pipelines and that makes sure that they still work the same way. There is drift in the code, and perhaps the frameworks they're using for development, but since the pipelines "just work" for them, they don't usually touch them and so very little drift.

πŸ‘ 1
Ashley Noble12:05:56

@tim.wyatt Build to attract... if it's not easier, we're not winning. Teams can do it themselves, but we cut their time down dramatically

πŸ‘ 2
Tim Wyatt12:05:29

This is perfect, and what I am trying to push in our org!

Ashley Noble12:05:39

@matthew.cobby Can you expand the question please?

Andreas Baernthaler - TNG Technology Consulting12:05:40

It's probably about how you manage template changes. If devs start out with an old template and you change the teamplate afterwards, you probably want to keep the artifacts created from the template up to date as well. Is this the point where the script that you mentioned comes into play?

Gus Paul - Morgan Stanley (he/him)12:05:41

Which security scanning tools are you using? We've not had a lot of success in that space

Ashley Noble12:05:32

@gus.paul Coverity (static code), BlackDuckHub (OpenSource), Twistlock (Container)

πŸ‘ 1
Gianluca Manzi12:05:33

Are all developers internal? #ask-the-speaker-track-3

Robert Ruzitschka - DevOps Guild Lead12:05:17

Did Teams generally like the push for standardization or was there resistance?

Ashley Noble12:05:20

@manzi.g yes, this is for internal developers, of which we've got quite a few

πŸ‘ 1
Tim Wyatt12:05:01

Interesting point on the "contributions" Jono Bacon would talk about building communities

Christian Rudolph (TUIGroup - Head Of DevOps Transformation)12:05:58

we started to build a dashboard who contributed to it

Christian Rudolph (TUIGroup - Head Of DevOps Transformation)12:05:53

and so drove behaviour to show this in newsletters and events

Ashley Noble12:05:10

I like that idea... I'll use that!

Gus Paul - Morgan Stanley (he/him)12:05:34

We have similar league table of inner source contributors...it definitely works

πŸ‘ 2
Tim Wyatt12:05:20

i.e. how do you incentivise teams to contribute?

πŸ‘ 1
Ashley Noble12:05:39

@robert.ruzitschka most people liked it and went with the flow. 5% wanted to swim up stream, and I was inclined to let them and spend time helping the teams that wanted help... generally the 5% ended up sinking and came back later on.

πŸ‘ 2
πŸ˜‚ 2
Ashley Noble12:05:19

@robert.ruzitschka I like your title, is that "Spotify" like?

Bernard Voos (FedEx)12:05:50

Hi @ashley.noble! Did you use the Google 4 Keys to build this dashboard?

Ashley Noble12:05:58

@tim.wyatt sometimes we gave out internal awards, but mostly they just got their "feature" in quicker.

Matt Cobby (NAB)12:05:05

That is a powerful motivator - we have an Inner Source programme to enable those changes into the platform in a formal way

Ashley Noble12:05:43

How do you prioritize your work and theirs? We found they often didn't align πŸ™‚

Matt Cobby (NAB)12:05:39

We find conversations work well. They do the dev work for their feature - to our dev standards

Robert Ruzitschka - DevOps Guild Lead12:05:24

@ashley.noble I am leading an internal community of DevOps guys - goal is to exchange knowledge and learn but also to act as a platform for alignment.

πŸ™Œ 1
Ashley Noble12:05:55

@bernard.voos mostly we took it from Accelerate, but I think they're pretty close

πŸ‘ 1
Chris Shuknecht12:05:16

What do the 'jump starts' consist of?

Ashley Noble12:05:26

A deep dive into all the pieces that make up a pipeline, and all the technology involved. A big challenge, given many hadn't ever worked with containers before.

Chris Shuknecht12:05:22

I bet. Thanks for the clarity!

Robert Ruzitschka - DevOps Guild Lead12:05:54

Metrics: Are you measuring on team level or aggregate? And how do you measure cycle time? We try to measure but there are endless discussions about start and end points πŸ™‚.

Andreas Baernthaler - TNG Technology Consulting12:05:06

@ashley.noble It sounds very familiar that creating self-service tools does not always work out as expected.

Tim Wyatt12:05:08

Isn't the start always form the point you merge to mainline and end is when you are running in prod? Everything else is a potential bottleneck to measure

Robert Ruzitschka - DevOps Guild Lead12:05:06

Tim, I agree. But unfortunately in our case teams do not generally have a useful definition of done = in production.

Ashley Noble12:05:50

@tim.wyatt Agreed, but measuring the middle points is useful because you can see the batching (from "The Goal" Book)

Ashley Noble12:05:37

@robert.ruzitschka Since we owned the whole pipeline, we built "tags" throughout every tool to measure when a commit made it to that stage, so we could measure it at each point.

πŸ‘ 1
Robert Ruzitschka - DevOps Guild Lead13:05:24

@tim.wyatt, @ashley.noble: I wanted to mention something, forgot it and now it came up again πŸ™‚. If you want to focus on the reduction of cycle time, is it meaningful to start measuring with the commit? The story could already have been in progress for weeks (including the respective feature branch). We tried to convince team that the relevant KPI starts with the state "In Progress". What do you think?

Tim Wyatt13:05:28

@robert.ruzitschka I think there are a number of things to measure: β€’ flow of work requests (stories, requirements, etc) into a teams backlog β€’ Flow of work from backlog into "To Do" β€’ Getting work done (from the point an item enters "in-progress" to "done" β€’ Lead time from merge to prod (IMO)....

Tim Wyatt13:05:46

Quite a lot of this is covered in Mik Kersten's work (http://flowframework.org)

πŸ™ 1
Ashley Noble12:05:44

@robert.ruzitschka we measure at all levels (individual, team, Release Train, Organization). We don't publish Individual. We found proxies for lead time that were merge to master (start) to when that commit was deployed to production (end). We also measured the intermediate points so we could demonstrate batching from Ops.

πŸ‘ 2
Gus Paul - Morgan Stanley (he/him)12:05:18

Yep. We are doing the exact same

Robert Ruzitschka - DevOps Guild Lead12:05:54

Agree, the only meaningful approach.

Gus Paul - Morgan Stanley (he/him)12:05:09

Do you have a change management approval function? Have you been able to automate that?

Tim Wyatt12:05:16

there are many other things to measure prior to this point (esp. in SAFe 🀒) like flow of work into teams....(but not really "devops")

Tim Wyatt12:05:27

Nice talk @ashley.noble

Robert Ruzitschka - DevOps Guild Lead12:05:44

Thanks, Ashley. Great and very realistic insights. Highly appreciated!

Ashley Noble12:05:15

pleasure great questions, thanks

Chris Shuknecht12:05:56

great content @ashley.noble. Thank you for sharing your experiences. I'll be watching that again!

πŸ™ 1
πŸ‘ 1
Adrienne Shulman12:05:56

thank you @ashley.noble ! great talk

Ashley Noble12:05:59

@gus.paul yes, mostly we have dashboards to show go/no-go, but we do also have some manual steps from other functions that we're working to build trust in our testing in order to allow us to deploy

πŸ™ 1
Ashley Noble12:05:38

Pleasure thankyou.

Ann Perry - IT Revolution12:05:15

Thank you, @ashley.noble! Coming up next, @tod.bickley

Bryan Kemp12:05:20

Yes, thank you @ashley.noble

Jason Clark12:05:42

@ashley.noble What sorts of points do you check to determine go/no go?

Bernard Voos (FedEx)12:05:45

Thanks for a great talk, @ashley.noble !

Ashley Noble12:05:56

@clarkjas, mostly all the items we ran as part of the pipeline, so make sure the security tools ran on that commit, make sure that the functional tests ran and passed, make sure the performance tests ran and passed.

πŸ‘ 1
Matt Cobby (NAB)12:05:50

"Plan, Build, Run and 5 Execs to make a decision" - sounds too familiar.

Tod Bickley12:05:26

@matthew.cobby Very painful!

Katharine Chajka (Tasktop)12:05:38

love that you articulated this opportunity you had!

Tod Bickley12:05:51

@katharine.chajka Thanks!

Katharine Chajka (Tasktop)12:05:56

"it starts with people"

Matt Cobby (NAB)12:05:50

How did you get the infra teams to trust the dev teams, even enough to let them in the door?

mark rendell12:05:27

@me1208 and @matthew thank you for your talk. Great to get an update on Team Topologies and the great work you are doing. One question - do you have any good examples of Team API definitions and/or templates? Is there a http://swagger.io yaml template for example? (Because everyone needs the chance to use more yaml πŸ˜‚)

Matthew Skelton (co-author of Team Topologies)12:05:31

Hi @mark.rendell have you seen the GitHub repos? This one has a lightweight example of a Team API https://github.com/TeamTopologies/Team-API-template

mark rendell12:05:27

Ah sorry no @matthew I'd missed that and only looked at the shapes repository. Looks useful, thanks! Anyone out there got some examples that they fancy sharing please?

πŸ‘ 2
Christian Rudolph (TUIGroup - Head Of DevOps Transformation)12:05:31

@tod.bickley how has the IAM team reacted to the general change?

Tod Bickley12:05:39

@matthew.cobby First we had leaders from App Dev come to Infrastructure. Second, we showed people results. The journey with the rest of the Infra team continues...

πŸ‘ 1
Tod Bickley12:05:08

@christian.rudolph Actually very well. A few outliers of course, but most everyone has leaned in to the change.

πŸ‘ 3
Helen Beal12:05:16

Great to see such clarity around the product manager and owner roles! What are their reporting lines?

πŸ‘ 3
πŸ‘€ 1
Tod Bickley12:05:13

@helen.beal We all report in the Technology Org. Split between Risk Management and Infrastructure. We have a pretty solid 1:1 match

Katharine Chajka (Tasktop)12:05:18

is there a certain cadence you use or any special meetings to keep everyone aligned?

Tod Bickley12:05:25

@katharine.chajka Yes. We have daily 15 minutes standups, quarterly Big Room Planning and monthly Town Halls. Try to make them quick and to the point

πŸ‘ 1
Marcin PrΔ…czko12:05:45

@tod.bickley How did you achieved solutions that everything should be in Jira (only) related with UserStory?

Katharine Chajka (Tasktop)12:05:53

how often were flow metrics reviewed?

Katharine Chajka (Tasktop)12:05:42

great slide, very helpful

Tod Bickley12:05:56

@marcin.praczko1 We essentially made it a requirement. If people talked about work they were doing that wasn't in JIRA (outside of OPS work in Service Now) we pushed it all to Jira

Tod Bickley12:05:12

We review FLOW metrics bi-weekly

πŸ‘ 1
Marcin PrΔ…czko12:05:05

@tod.bickley What about emails - requests coming directly via chats / emails / etc - did you have some person who put those requests to Jira?

Tod Bickley12:05:45

@marcin.praczko1 For the most part yet. If they were "quick hits" maybe 10 mins or so, we would do them. If it's more that an hours, it's a JIRA card

Helen Beal12:05:56

Sorry - did I miss this - do you automate the collection of your flow metrics?

Tod Bickley12:05:11

And really this is more of an art than a science. A loosely defined set of rules empowering engineers

πŸ‘ 1
Katharine Chajka (Tasktop)12:05:40

love the transparency into financials

Tod Bickley12:05:52

@helen.beal We somewhat automate. Ours TDP's still manually pull the metrics to put into a biweekly communication

Helen Beal12:05:54

Thanks Tod - have you looked at integrating the DevOps toolchain for visibility or adding a value stream management solution on top?

Philipp BΓΆschen, TUI, DevOps Coach, (he/him)12:05:56

@tod.bickley how do you avoid your JIRA becoming a complicated disaster where it takes ages to create and move tickets around? :thinking_face:

βž• 1
Chris Leeworthy (he/him)12:05:28

what is meant by a TDP please?

Katharine Chajka (Tasktop)12:05:43

related to velocity - do you use flow metrics for predictability or still more estimation/story points, or both?

Marcin PrΔ…czko12:05:43

@tod.bickley So you still had '5 min task (in your case 10 min)' to deal quickly (ad-hoc) without Jira. Make sense.

Tod Bickley12:05:50

@philipp.boeschen650 They key is having the TDP's triage these. They are such a key role

πŸ‘ 1
Tod Bickley12:05:25

TDP Is the Technology Delivery Professional. They are scrum master like, but are also technical. They manage cards AND pull cards

πŸ‘ 1
❀️ 1
Tod Bickley12:05:55

@katharine.chajka Flow metrics, not story points for us

πŸ‘ 3
Katharine Chajka (Tasktop)12:05:08

super interesting about the TDP role - I feel like it was how SM used to be back in the day haha, funny to see so many FT SM now

Philipp BΓΆschen, TUI, DevOps Coach, (he/him)12:05:19

That's an interesting approach, I like that πŸ™‚ Have experienced super positive things with similar roles inside of gaming a few years back

Katharine Chajka (Tasktop)12:05:09

knowledge sharing sessions for sure on the PO not clear

Matt Cobby (NAB)12:05:11

I've been silent listening to this talk as every single moment is so full of really interesting insights and experiences. This is possibly the single most dense presentation in terms of golden nuggets of experience. Can you just come work for us please? @tod.bickley ? This is the problems so many of use face with Infrastructure teams and the shift the value flow/devops/digital transformation.

😁 2
Katharine Chajka (Tasktop)12:05:29

re the agile vs technical this is a struggle for sure

Tod Bickley12:05:37

@matthew.cobby Thanks Matt!

Bernard Voos (FedEx)12:05:49

Thanks, @tod.bickley!

Tod Bickley12:05:50

Thanks all! Really glad for the questions.

Clarissa Lucas, IT Audit Director, Nationwide Insurance12:05:51

@tod.bickley - Great Presentation!

πŸ‘ 1
Chris Leeworthy (he/him)12:05:09

Thank you, very interesting stuff

Khyati Ghatalia12:05:36

Thanks @tod.bickley - The presentation was quite informative!

Matthew Skelton (co-author of Team Topologies)13:05:44

Enjoy the talk from me and @me1208, everyone πŸ™‚ Looking forward to the discussion here soon :thumbsup:

1
Ann Perry - IT Revolution13:05:58

Welcoming our friends and IT Revolution authors, @matthew and @me1208!

πŸ™Œ 2
πŸŽ‰ 1
Manuel Pais, speaker, co-author Team Topologies13:05:33

Hi everyone! Looking forward to your questions πŸ™‚

James Simon13:05:39

Linke to the academy?

James Simon13:05:41

Thank you!

πŸ‘ 3
Ann Marie Fred - IBM13:05:30

I like this Team API concept

πŸ‘ 1
πŸ’― 2
Robert Ruzitschka - DevOps Guild Lead13:05:52

Team Topologies Core Concepts seem to be already well established in the community as there are not too many questions :-)

Christoph Hagedorn13:05:43

Can't speak for others. But its new to me

πŸ‘ 2
Filip Berlikowski, CTO13:05:18

And delivered at super rapid pace πŸ˜„

1
🏎️ 1
Robert Ruzitschka - DevOps Guild Lead13:05:48

The book is highly recommended!πŸ™‚

πŸ™ 1
Ann Marie Fred - IBM14:05:22

Adding the book to my reading queue πŸ™‚

πŸ˜„ 1
Filip Berlikowski, CTO14:05:43

ugh, it ages on my pile of shame 😊

2
πŸ˜€ 1
Christoph Hagedorn14:05:21

"reading queue" - that's great

Manuel Pais, speaker, co-author Team Topologies14:05:45

Now you can attend the academy and in 3h you can pretend you've read the book Filip :rolling_on_the_floor_laughing: https://academy.teamtopologies.com/courses/team-topologies-distilled

Filip Berlikowski, CTO14:05:37

Thanks Manuel πŸ™‚ sounds like a prerequisites for this presentation πŸ˜‰

πŸ˜„ 1
Kevin Littlejohn14:05:29

cognitive load has been hugely useful as a tool for me in looking at our teams, and highly predictive - overloaded teams fragment into specialists, it's a clear indicator of where to go looking for fracture lines and break teams up or reduce load

πŸ™Œ 1
Vlad Ukis14:05:14

did you do any structured measurements?

Kevin Littlejohn14:05:39

honestly, no - the teams in question are very obviously overloaded :face_palm:

😲 1
Manuel Pais, speaker, co-author Team Topologies14:05:14

Nice heuristic @kevin986 ! Hadn't thought about it in that way, but makes sense: "over specialization as a smell for too high cognitive load"

πŸ‘ 1
Vlad Ukis14:05:44

Basically, it is not quite a team but a set of specialized individuals with not enough cohesion. Right?

πŸ‘ 2
Helen Beal14:05:31

I think we need cranial probes for cognitive load assessment

😁 3
Matthew Skelton (co-author of Team Topologies)14:05:06

Will you be the first to volunteer, Helen? πŸ˜„

Manuel Pais, speaker, co-author Team Topologies14:05:37

Elon Musk might be able to help πŸ™‚

Helen Beal14:05:01

:rolling_on_the_floor_laughing:

Helen Beal14:05:16

I'll check in with Elon and see what he suggests πŸ™‚

2
Helen Beal14:05:35

Seriously though - you've given me an idea... I'm reaching out to some of the work neuroleaders at the moment on another initiative (like David Rock and Britt Andreatta) - I shall see if they have any nuggets. As you both know I am a tad obsessed with this subtopic of yours!

πŸ‘ 2
Matthew Skelton (co-author of Team Topologies)14:05:11

That would be great. Manuel and I are exploring how to scale assessments for team cognitive load - we have some PhD level expertise involved, too 🧠

Bryan Finster - DoD Platform 1 (Speaker)14:05:33

I love that definition of "Platform"!

βž• 2
πŸ™Œ 2
Kevin Littlejohn14:05:00

also helped explain to a new manager why his team was behaving the way it was, and why "share the knowledge" wasn't working within the team

🎯 2
Ffion Jones (Partner, PeopleNotTech)14:05:56

"The worth of certain leaders can depend on how many people report to them" We need to address power structures, indeed.

πŸ‘ 1
Manuel Pais, speaker, co-author Team Topologies14:05:49

Such a hard problem to solve 😞

Adrienne Shulman14:05:19

sadly a common problem πŸ˜•

Saulius Bartkus14:05:14

I wonder what are the Flow metrics ( any good examples)?

πŸ‘€ 1
Gianluca Manzi14:05:00

check out DORA metrics

David Orsi - RTE, NatWest Group14:05:14

How long does something sit with nothing happening to it?

🎯 1
Ann Marie Fred - IBM14:05:27

Like from Domenica DeGrandis?

βœ… 1
Manuel Pais, speaker, co-author Team Topologies14:05:32

We were referring to the Accelerate / DORA metrics and we often recommend to look at flow efficiency at least

Saulius Bartkus14:05:28

cycle times, velocity ?

Bernard Voos (FedEx)14:05:51

Hi @matthew and @me1208! How do you see the dynamic between strongly opinionated platforms and the optional approach to using the platforms?

πŸ‘ 1
Matthew Skelton (co-author of Team Topologies)14:05:35

A platform can be strongly opinionated without requiring anyone to use it

Matthew Skelton (co-author of Team Topologies)14:05:49

The two things are separate... or should be

Matthew Skelton (co-author of Team Topologies)14:05:15

The opinionated platform MUST be informed by the needs of teams builing software in that organization, however.

Matthew Skelton (co-author of Team Topologies)14:05:29

Opinions are no good if they are not suited to the context

Robert Ruzitschka - DevOps Guild Lead14:05:37

My experience is that in many cases teams severely underestimate the effort to manage and maintain things that could be put into a platform. This reduces their capacity to provide business value. That is why cognitive load is a very useful concepts to frame these discussions.

πŸ’― 1
Robert Ruzitschka - DevOps Guild Lead14:05:11

Of course platforms must cater to the teams needs and must not hold them back or slow them down.

Manuel Pais, speaker, co-author Team Topologies14:05:22

Platform is a product, and most products succeed because they are opinionated in a way that meets users needs (i.e. provides differentiation vs other similar products).

Manuel Pais, speaker, co-author Team Topologies14:05:52

In internal platforms we need to be looking much more at: product adoption lifecycle and value proposition (differentiators) for internal teams.

Manuel Pais, speaker, co-author Team Topologies14:05:15

If it's less cognitive load to use AWS directly than the internal infra platform, we've done something wrong.

Manuel Pais, speaker, co-author Team Topologies14:05:30

Mandating the platform only exacerbates the problem.

Bernard Voos (FedEx)14:05:40

Do you see a role on the platform team to be almost a marketeer/salesperson for the platform?

πŸ’― 1
Sophie Weston14:05:32

Sadly, mandating use of the platform is what I see too often - devs are compelled to use it, rather than it being a compelling product that they choose to use.

😒 2
πŸ‘ 1
Matthew Skelton (co-author of Team Topologies)14:05:03

https://devopsenterprise.slack.com/archives/C0150HQB6UX/p1621347040138500?thread_ts=1621346631.119600&amp;cid=C0150HQB6UX Absolutely, @bernard.voos - a skill often lacking in platform groups is marketing of the platform to internal customers.

Michael Bird - Senior Architect, UPMC14:05:09

At some point, if you have resistance to the platform that interferes with the creation of a cohesive product, persuasion no longer works. How do you combat that?

Matthew Skelton (co-author of Team Topologies)14:05:15

Why is there resistance to the platform? Is the platform meeting teams' needs?

Matthew Skelton (co-author of Team Topologies)14:05:51

What incentives are in place to help guide the teams towards something more coherent?

Michael Bird - Senior Architect, UPMC14:05:28

Our platform includes our messaging system, common UI libraries, ... In general we get pushback on the UI pattern. Many time it comes down to "prove that this is the best solution". The answer that is is a "good enough solution" that we can iterate on over time is sometimes not well received.

Manuel Pais, speaker, co-author Team Topologies14:05:03

Don't look at platform adoption as a "combat", you shouldn't be convincing teams, you should be making their life easier (and market how you do it internally).

Matthew Skelton (co-author of Team Topologies)14:05:28

If they have no need for it or have other needs that are already met, what is the benefit in "standardisation"?

Matthew Skelton (co-author of Team Topologies)14:05:06

It's an important trade-off that needs to take into account the sense of "ownership" in the teams

Michael Bird - Senior Architect, UPMC14:05:25

We don't have time to convince everyone, and in order to get our separate domain services to play well together, we need some standards for how they are built. The last thing we want is 50 different ways to attempt to make things work together. It slows the other teams down when trying to integrate and makes it more difficult for someone to change teams.

Matthew Skelton (co-author of Team Topologies)14:05:12

Have you tried changing the incentives around onboarding and integration?

Matthew Skelton (co-author of Team Topologies)14:05:41

Add "ease of onboarding" and "ease of integration" to each team's balanced scorecard

Michael Bird - Senior Architect, UPMC14:05:51

We're not up to score cards yet. We are still getting up to speed with new teams, new pipelines, devops...

πŸ‘ 1
Matthew Skelton (co-author of Team Topologies)14:05:57

Ultimately, though, some needs are not being met by the platform as it is: maybe DevEx/DX, docs, support, etc.

Michael Bird - Senior Architect, UPMC14:05:25

Sometimes it's just a person. Some people have very strong opinions that don't align with the rest of the teams. If that is a strong personality compared to others in a team, you get this kind of situation.

Michael Bird - Senior Architect, UPMC14:05:02

It's more a personnel issue than a technical one.

Michael Bird - Senior Architect, UPMC14:05:56

We are moving towards a set of architectural requirements that allow the artifacts from the different teams to work smoothly together in a consistent way. We can iterate on those requirements based on feedback and the emergence of new technolologies.

πŸ‘ 2
Manuel Pais, speaker, co-author Team Topologies14:05:42

That should help, just be careful requirements are not "abused" in the sense you can make them so low level that you're basically telling people how to do their work πŸ™‚ It's a fine line, obvisouly.

Michael Bird - Senior Architect, UPMC14:05:36

Yes, and we don't want that. Ours are more communication patterns between team artifacts that make them work together easily and consistently. This reduces integration time since you know what to expect from another team.

πŸ’― 1
πŸ‘ 1
Nick Eggleston, IT Architect (open to offers)14:05:57

Cognitive Load is quite an eye opener when you haven’t heard of it before. Thanks for putting a spotlight on it.

πŸ™ 2
Craig Cook - IBM14:05:58

Netflix focuses on this concept for their teams

Manuel Pais, speaker, co-author Team Topologies14:05:17

Indeed. Netflix used the DevOps Topologies internally extensively: http://devopstopologies.com

Manuel Pais, speaker, co-author Team Topologies14:05:34

These are in fact the precursors to Team Topologies πŸ™‚

Nick Eggleston, IT Architect (open to offers)18:05:48

Do they? And yet "Netflix doesn't do DevOps" :rolling_on_the_floor_laughing:

Manuel Pais, speaker, co-author Team Topologies22:05:16

I'd rather say they do DevOps without calling it DevOps because... they don't need to.

Manuel Pais, speaker, co-author Team Topologies22:05:04

Yet they look at available approaches and what can be useful in their context. Very different from so many orgs who just want to put a "DevOps stamp" over their image

Olivier Jacques, DXC14:05:03

"Complicated sub system teams": this is a notion which I frankly try to put on the side and not use. In fact, I have not seen a place where I needed those.

πŸ’― 3
πŸ‘ 1
βž• 1
Vlad Ukis14:05:57

In a monolithic architecture, you are going to have that.

Matthew Skelton (co-author of Team Topologies)14:05:02

Great - keep it that way, Olivier! That's what re recommend in the book. Use a CS team only if truly needed.

Kevin Littlejohn14:05:05

We have one or two, but they're very infrequent.

Chris Gallivan, Stellantis, Value Stream Architect14:05:05

in an automotive ECU - I see some need for complicated subsystems

Manuel Pais, speaker, co-author Team Topologies14:05:14

Can we hire you as "evangelist" @olivier.jacques? πŸ˜„

πŸ’― 1
Gianluca Manzi14:05:42

For the anti-patterns related to sub-teams / internal teams, is a good proxy to measure number of tickets to raise from epic/feature to code to Production? #ask-the-speaker-track-3

Kevin Littlejohn14:05:19

value stream map! :)

😁 1
Gianluca Manzi14:05:21

True...map it to simplify

Matthew Skelton (co-author of Team Topologies)14:05:13

The number of tickets needed to get some code into Production would be a in interesting measure, @manzi.g - ideally, the number of tickets needed would be 0, zero, zilch, null, none, ...

Helen Beal14:05:01

But it's ok if the stream-aligned team is offering a service (rather than a product) to customers? It's just when they are torn in multiple (and internal) directions? Can we say value stream aligned?

πŸ‘€ 2
Matthew Skelton (co-author of Team Topologies)14:05:11

The service/product thing is a bit blurred, but the key thing ehre is that a Stream-aligned team should have a single focus (customer), not trying to serve two or more customers.

πŸ‘ 1
Pulak - Accenture DevSecOps Practice Lead14:05:36

@olivier.jacques I was running a project last year (terraforming and "ansibilizing" SAP builds) with 4 teams broken down by cognitive load and the complicated subsytems team was clearly SAP expertise

πŸ‘‹ 2
πŸ’‘ 2
Pulak - Accenture DevSecOps Practice Lead14:05:30

I would say sensibilizing now that we have most SAP products e.g. S4 HANA end to end on any major cloud stood up from scratch to functional working app in under 3 hours . Anyone who has done it by hand would tell you it takes weeks usually even in 2021 to build anything close to prod grade and scale. Thanks to TT we did it pretty well

πŸ‘ 1
Manuel Pais, speaker, co-author Team Topologies14:05:08

That's awesome. Sounds like this would make a wonderful industry example if you're up for it @pulak.a.agrawal;)

Manuel Pais, speaker, co-author Team Topologies14:05:38

Awesome, will ping you later via email or Linkedin.

Olivier Jacques, DXC14:05:12

One of my favorite TT resource is the drawio/diagrams.net library at https://github.com/TeamTopologies/Team-Shape-Templates#drawio--diagramsnet

πŸ’₯ 1
πŸ‘ 2
Chris Gallivan, Stellantis, Value Stream Architect14:05:48

in automotive the regulatory requirements are driving high levels of cognitive load

Chris Gallivan, Stellantis, Value Stream Architect14:05:05

more time spent on regulations than on the customer

Matthew Skelton (co-author of Team Topologies)14:05:51

Is there an opportunity to simplify the detail of the regulations via some kind of platform, @chris.gallivan278?

Chris Gallivan, Stellantis, Value Stream Architect14:05:21

there is an industry std architecture called Autosar that attempts to abstract some of this away - this has a side effect of dumbing down the engineers imho

James Simon14:05:57

because they have not automated inspection and audit

Andreas Baernthaler - TNG Technology Consulting14:05:09

Regarding compliance as code: GitOps seems to solve some of the problems in this area. Any thoughts on this?

Chris Gallivan, Stellantis, Value Stream Architect14:05:33

there is an attempt to use modeling and code generation tools to an extent

Manuel Pais, speaker, co-author Team Topologies14:05:36

I'd try to start with a regulations enabling team.

Manuel Pais, speaker, co-author Team Topologies14:05:08

A group of experts (likely not from IT) that could help most stream teams raise their understanding of the fundamental constrainsts in the regulations.

πŸ’― 1
Manuel Pais, speaker, co-author Team Topologies14:05:39

At the same time, look for opportunities to automate some of these checks. But sounds like fundamentally teams don't have enough awareness of regulations.

Manuel Pais, speaker, co-author Team Topologies14:05:59

In fact, they need to increase their "germane cognitive load" on that front it sounds like.

Manuel Pais, speaker, co-author Team Topologies14:05:25

(yes, cognitive load is not always bad as we explain in the book and in our Academy)

James Simon14:05:46

Would love to hear more about Gitops Compliance as Code

Christian Kullmann, Eurowings Digital, Automation Professional (He/Him)14:05:47

Thank you for the insights @matthew, @me1208

βž• 1
πŸ™ 1
πŸ‘ 1
Helen Beal14:05:50

Thanks you two - fabulous to get an update - thanks for sharing as always!

πŸ™Œ 2
πŸ™ 1
Manuel Pais, speaker, co-author Team Topologies14:05:42

Great questions as always Helen! Nice to hear from you!

Randy Shoup (Chief Architect eBay; he/him)14:05:25

I'm struggling with the anti-pattern around a stream-aligned team providing a service. Are you saying that any internal service would be provided by a platform team then?

βž• 2
Vlad Ukis14:05:43

we are having a notion that any team in the org can provide an enterprise-wide and / or external service delivered according to a common service standard

Vlad Ukis14:05:50

Does anyone else have that as well?

Matthew Skelton (co-author of Team Topologies)14:05:31

The difficulty comes when a team ends up with multiple customers because that leads to priority clashes.

Vlad Ukis14:05:26

Yes, absolutely. If a team decides to provide a service, they take the full product ownership for the service.

πŸ’― 1
Vlad Ukis14:05:44

Do you see this leading to issues down the road?

Matthew Skelton (co-author of Team Topologies)14:05:11

We agree with people like Marty Cagan that a Stream-aligned team (what Marty might call a product team) should have a single real customer. As soon as that team starts providing (say) some data "as a service" to other internal teams, they then have more than one customer. Far better to expose that data via an API< have a platform ingest that data, and then provide the data back out as apart of a proper platform experience.

Manuel Pais, speaker, co-author Team Topologies14:05:51

Even for functionality the same problem applies. It's quite hard for a team of up to 9 people to do proper service discovery (which is part of a stream team's responsibilities) if they see every stakeholder as a customer, both internally and externally.

Matthew Skelton (co-author of Team Topologies)14:05:14

OR... give the Stream-aligned team enough time and space to truly provide a service to other internal teams, ... But that can get messy very quickly.

Manuel Pais, speaker, co-author Team Topologies14:05:23

Separate concerns require separate teams. Otherwise, you'll likely see external customer needs always prioritized over internal customer needs.

Lucek14:05:46

Is there a limit in separating concerns? We ended up with a single, well-grained service, that is consumed by both internal and external customers. It's so small, that there is no chance to further distill it. Building a platform for it looks like an overkill...

Vlad Ukis14:05:05

@lucjan.giza is there a dedicated team that owns the service completely?

Matthew Skelton (co-author of Team Topologies)14:05:01

If there is no overhead on the team from this, then (by definition) you have a decent solution, @lucjan.giza πŸ™‚ Just look out for the situation where the scale changes and a Platform is needed. BUT - as always - only introduce a platform when flow is impeded and/or team cognitive load is too high.

πŸ‘‹ 2
Vlad Ukis14:05:14

That is what we do as well. Seems to be working well for the moment being.

πŸ‘ 1
πŸ‘ 1
Bernard Voos (FedEx)14:05:49

Thank you @matthew and @me1208!!! That was great and the music transitions were sweet

πŸ™Œ 1
πŸ‘ 1
🎺 1
Manuel Pais, speaker, co-author Team Topologies14:05:07

The music bits you have to thank @matthew the trumpet player :)

Matthew Skelton (co-author of Team Topologies)14:05:47

🎺 🎡 πŸ˜„

πŸ₯³ 1
Olivier Jacques, DXC14:05:01

πŸ‘ Thanks for the talk Manuel & Matt - that was great. Lots of announcements there too

πŸ’₯ 1
πŸ‘ 1
Manuel Pais, speaker, co-author Team Topologies14:05:43

What better place than DOES to announce new initiatives? πŸ™‚

Katharine Chajka (Tasktop)14:05:14

great talk, thank you!

πŸ‘ 1
πŸ™ 1
Robert Ruzitschka - DevOps Guild Lead14:05:24

I see no big difference between stream aligned and platform teams. The have different customers. But many of the the "product" team principles apply to both.

πŸ‘€ 1
Matthew Skelton (co-author of Team Topologies)14:05:51

The "Platform" is really a viewpoint, a wrapper around a different set of needs or focus

πŸ’― 1
Matthew Skelton (co-author of Team Topologies)14:05:44

In the TT book we clearly say that inside the Platform there should be mostly Stream-aligned teams... just focus on on platform-y things (data? infra? design?) rather than end-customer services.

Randy Shoup (Chief Architect eBay; he/him)14:05:49

@matthew @me1208 Great talk! I really appreciate all of the additional tools and resources you are providing. You're making a community effort out of this, which makes it so much more likely to gain traction and be transformative. Thanks!

πŸ™Œ 3
πŸ™ 2
Manuel Pais, speaker, co-author Team Topologies14:05:34

Reading that really makes it worthwhile, honestly! Thank you!

1
Olivier Jacques, DXC14:05:48

@cornelia will have a talk Thursday on GitOps and business benefits. I don't if she'll touch on the compliance as code as a topic.

πŸ‘ 1
Andrew Davis14:05:52

Hello everyone ... Looking forward to discussing about DevOps for Salesforce with you!

πŸ‘ 2
Ann Perry - IT Revolution14:05:54

Thank you so much, Matthew and Manuel! We now welcome @abd3721, coming up next!

πŸ‘ 2
Adrienne Shulman14:05:55

Excited for this talk @abd3721, i'm already believer devops principles can be widely applied to so many other areas

πŸ™Œ 1
πŸ’― 1
Adrienne Shulman14:05:54

raise your hand if you are old enough to know what a rolodex is. i'll start πŸ™‹

πŸ˜† 1
1
Manuel Pais, speaker, co-author Team Topologies14:05:51

Thank you everyone, insightful questions as always! If you need any info on the new resources and tools we mentioned, just drop us a DM here or LinkedIn or email: <mailto:info@teamtopologies.com|info@teamtopologies.com> or <mailto:academy@teamtopologies.com|academy@teamtopologies.com>

πŸ™Œ 3
πŸ™ 1
Andrew Davis - Copado - DevOps for Salesforce14:05:26

Anyone here already managing a development lifecycle on Salesforce?

Adrienne Shulman14:05:32

not me personally , but i know our salesforce team has been working on a big bang data project and struggling to launch it successfully

Adrienne Shulman14:05:06

i'm sure they could benefit from an agile, devops mindset

πŸ’― 1
Andrew Davis - Copado - DevOps for Salesforce14:05:20

I'd be happy to connect with your team at some point if that would be useful

πŸ™ 1
Adrienne Shulman14:05:13

thanks.... it definitely may be.. will reach out to you if it makes sense to connect about it

Lucek14:05:01

@me1208, @matthew - will you be available for discussion in Gather zone / Birds of a Feather?

Matthew Skelton (co-author of Team Topologies)14:05:35

We have an AMA later today - 18:25 UK time

1
Joachim14:05:13

Yes, good fun - like 1999. πŸ˜‰ Will you provide your view on SF DX?

Andrew Davis - Copado - DevOps for Salesforce14:05:02

I cover Salesforce DX (somewhat briefly) a little bit later in the talk

Joachim14:05:18

What, no Copado vs. DX comparison? πŸ˜†

Joachim14:05:17

I am having this discussion surprisingly often for the last 2 years.

Andrew Davis - Copado - DevOps for Salesforce14:05:59

Kinda sorta. The DX initiative basically opened up a huge number of new improvements that makes things better for everyone. There are things that only Salesforce can improve.

πŸ‘ 1
Andrew Davis - Copado - DevOps for Salesforce14:05:53

Nice that you're up to speed on the options @joachimsammer. I obviously get a skewed perspective, but this topic of "DevOps" has become a bigger and bigger deal for Salesforce customers. It's top of mind for most big Salesforce teams

Dmitry Luchnik /// adidas Data Analytics architect14:05:21

@abd3721 maybe a bit offtopic - but the similar challenges exist in the SAP world as well. And as I'm from the data domain - just wanted to ask: Do you have any insights/experience on "DevOps for BW?" /4HANA, or DWH Cloud, or SAC - whatever

Andrew Davis - Copado - DevOps for Salesforce14:05:29

I don't have direct experience on SAP, but Copado is starting to work on that. You could connect with them in the xpo-copado channel

πŸ‘ 1
Andrew Davis - Copado - DevOps for Salesforce14:05:52

This is an actual XML merge in Salesforce

4
Andrew Davis - Copado - DevOps for Salesforce15:05:11

You'd love it @matthew Merge works fine as long as there aren't any repetitive elements. But, a 100,000 line long XML file is basically filled with repetitive elements.

Andrew Davis - Copado - DevOps for Salesforce15:05:34

You've got to parse it and merge it intelligently if you want to manage that XML at scale

πŸ’― 1
Andrew Davis - Copado - DevOps for Salesforce14:05:07

Database Monoliths for the win! @me1208 @matthew

2
Matthew Skelton (co-author of Team Topologies)14:05:50

Manuel and I once worked on a monolithic database system where it took over 24 hours to build the new code - this was in 2016

Matthew Skelton (co-author of Team Topologies)14:05:00

A developer VM in the cloud took 40 minutes to start

Matthew Skelton (co-author of Team Topologies)14:05:42

Almost all the business logic was in the database, but not even accessible as normal SQL - it was generated from models in a proprietary language - woo!

Matthew Skelton (co-author of Team Topologies)14:05:57

Needless to say, development there was not rapid...

Andrew Davis - Copado - DevOps for Salesforce15:05:47

The term "database monolith" won my heart when I saw that in team topologies, since it describes Salesforce and the challenge of modularizing Salesforce. Miraculously, they've engineered things such that migrating and updating the Salesforce data model is super fast, and is not the limiting factor at all. The limit is on your ability to modularize your applications since they all get intertwined with the big database

πŸ‘ 1
Matthew Skelton (co-author of Team Topologies)15:05:01

That is good to hear. As always, the focus should be on a fast flow of change... and supporting technologies to enable that. :thumbsup:

πŸ‘ 1
Andrew Davis - Copado - DevOps for Salesforce14:05:17

Note the vision for separating stream-aligned teams from platform teams

πŸ‘ 2
Gene Kim, ITREV, Program Chair14:05:16

Thank you so much for this fantastic tutorial on the Salesforce ecosystem, @abd3721!!

❀️ 1
πŸ‘ 1
Andrew Davis - Copado - DevOps for Salesforce14:05:43

Very grateful for your invitation to share @genek!

Joachim14:05:38

Aren't packages just a "cosmetic layer" on top of the SF data model?

Andrew Davis - Copado - DevOps for Salesforce14:05:01

Salesforce packages all become part of the common Salesforce data model. So they're not isolated the way you might expect on some other platforms

Andrew Davis - Copado - DevOps for Salesforce14:05:21

That's what makes it tricky to decouple metadata into packages

Joachim14:05:22

Yep, so just a way of structuring your code and config. The underlying architecture is still the same SF core and add-ons.

Andrew Davis - Copado - DevOps for Salesforce14:05:35

Yes. There are some hard distinctions between packages - commercial apps can protect their IP, and get additional runtime (governor) allowances

πŸ‘ 1
Andrew Davis - Copado - DevOps for Salesforce14:05:27

But even though Salesforce packages don't segregate things very deeply, they at least provide some formal statement of where boundaries exist

Joachim14:05:02

Yes, at least there is a boundary. πŸ˜‰

πŸ‘ 1
Andrew Davis - Copado - DevOps for Salesforce14:05:22

That reduces the risk of confusion about who needs to change what, or where the risks of a change may lie

πŸ’― 1
Gene Kim, ITREV, Program Chair14:05:40

cc @jakob.knutsson657 ^^^ regarding SAP, Salesforce, etc.

Gene Kim, ITREV, Program Chair14:05:58

Just curious, @abd3721: How much time do you have to spend trying to convince Salesforce developers to use version control? cc @jason.cox

πŸ˜‚ 2
Andrew Davis - Copado - DevOps for Salesforce14:05:01

... cuz you know, who needs to remember history, right?

1
Andrew Davis - Copado - DevOps for Salesforce14:05:47

Salesforce teams like the idea, it's just sometimes daunting if they feel they have to go to the command line, if they didn't come from a coding background

Andrew Davis - Copado - DevOps for Salesforce14:05:26

And Salesforce metadata doesn't make it easy. It opens up some problems that most code projects don't struggle with

Matthew Skelton (co-author of Team Topologies)14:05:13

VSCode is pretty good for Git integration, to be fair. There is even an official Salesforce extension pack! https://marketplace.visualstudio.com/items?itemName=salesforce.salesforcedx-vscode

Matthew Skelton (co-author of Team Topologies)14:05:43

With VSCode, there is (often) no need to drop to the command-line.

πŸ‘ 1
Andrew Davis - Copado - DevOps for Salesforce14:05:27

The introduction of VS Code to the Salesforce community has been a huge win for those who write code on the Salesforce platform. The challenge remains that you have a lot of non-coders collaborating on changes, and they also need to manage their changes alongside the coders.

πŸ‘ 1
Dmitry Luchnik /// adidas Data Analytics architect14:05:33

@genek101, we had a similar challenge in Data Warehouse domain. Convincing DWH Developers to use git was a starting point. But you know - understandable Diff + Blame made the magic :)

πŸ˜† 1
πŸ‘ 1
Andrew Davis - Copado - DevOps for Salesforce14:05:22

So you have to span the skill boundary between the click-based folks and the coders, maintain a shared history, and be able to deploy stuff that's not just code.

πŸ‘ 1
Dmitry Luchnik /// adidas Data Analytics architect15:05:06

Also testing in between. We invested our focus into a fast CI - which validates the whole data warehouse model with all dependencies on each commit.

πŸ‘ 3
Andrew Davis - Copado - DevOps for Salesforce15:05:07

Yeah, it's amazingly powerful when you can get such automation in place. Version control is the only big hurdle

Matthew Skelton (co-author of Team Topologies)15:05:27

Agreed. Version control is hugely powerful but it's not hugely accessible... yet. Interesting discussion/rant here: https://twitter.com/matthewpskelton/status/1387521384226643968

πŸ‘ 1
Gene Kim, ITREV, Program Chair20:05:43

This is so good! Thank you!!!

❀️ 1
πŸ‘ 1
Gene Kim, ITREV, Program Chair20:05:37

PS: in confidence, I miss the simplicity of SourceSafe β€”Β the ability to share files was so awesome. (forget the occasional database corruption issue.)

😱 1
πŸ™‚ 1
Dmitry Luchnik /// adidas Data Analytics architect13:05:59

@matthew, oh, I've been in exactly the same discussions :) Unfortunately a lot of tools do not provide first-class version control. Take Office365 - saves all versions automatically. But where is the diff, pull request, etc.? Finding out what changed is very tricky. In my eyes it slows down collaboration dramatically. "What am I looking into, what's the story someone else wanted to tell by modifying 5 slides...." Have no recipe how to get git accepted for WYSIWYG guys...

πŸ‘ 1
Matthew Skelton (co-author of Team Topologies)14:05:39

I intend to write up some of this stuff but the short version is that the solution here seems to lie in custom visual diffs based on file types. So an AutoCAD file gets special 3D visualisation diff An Excel file gets a custom column-based diff etc.

Joachim14:05:51

Thank you, Andrew.

Mike McDaid - Copado14:05:28

Head over to #xpo-copado for more Q&A with Andrew Davis (Author of Mastering Salesforce DevOps)

πŸ‘€ 1
πŸ‘ 1
1
πŸ‘ 1