Fork me on GitHub
Sean D. Mack12:10:17

👋Good morning folks! Excited to get started here at DOES '21!

❤️ 7
Sean D. Mack12:10:09

My session, "DevOps for Pandemics" kicks off at 12:20 EDT. Hope to see you all there.

👍 5
Bramley Maetsa13:10:48

Im excited to me here

👍 1
Raghu Tumuluri14:10:50

Great themes for sure. please share more on breaking down the silos. how did you organize your product teams and not let the matrix structure come in the way.

👏 1
👍 1
Christian Kullmann, Eurowings Digital, Automation Professional (He/Him)14:10:17

I like that you speak of guests rather than customers. Has this always been this way or did it involve a cultural shift lately or some time ago?

💯 2

Love the “Demos over memos” 👍

👍 3
👏 1
Paul Murphy, TeamLeader, Edward Jones15:10:38

For Vanguard - does monolith=mainframe?

Christina Yakomin16:10:26

in many cases, yes, our monolith has mainframe dependencies

Paul Murphy, TeamLeader, Edward Jones15:10:55

For Vanguard - does monolith=mainframe?

Tashfeen Mahmood15:10:58

Vanguard Team - Great Presentation! That really resonated!

❤️ 1
Maria Luisa Polo15:10:29

Great Presentation!! Many Thanks for sharing!!

❤️ 1
Andre Prado15:10:12

That was great!! Thank you for sharing.

❤️ 1
Bill Orr15:10:14

Does Vanguard have regularly scheduled Chaos testing that impacts the full enterprise or are they more configured for a smaller targeted group? For example do you stress just a subset of applications or shared services?

Christina Yakomin16:10:19

the majority are very targeted. We have small-scale chaos experiments happening in vanguard's non-production environment every single day. Larger-scale chaos experiments happen less frequently, because of the coordination effort required, but we're certainly tackling this a few times a year with aspirations to get even better at this once we have fully activated our SRE operating model

Bill Orr16:10:53

For the Large-scale chaos testing what metrics do you look to collect? For example, total apps that failed, degraded service, latency?

Bill Orr16:10:19

My apologies for the detailed questions but very fascinated in how you approach these types of tests.

Christina Yakomin16:10:37

We go in with a set of hypotheses for every test we run. Which metrics/SLOs we look at depend on the type of test we’ve crafted. It changes every time! But everything you’ve listed has been used at some point, yes.

René Lippert15:10:51

Vanguard Team: Thanks for sharing! This was really great. I will definitely watch it again.

❤️ 1
Bill Orr15:10:23

What specifically do the SRE Leads provide to the Product teams? For example do they just explain policy to provide clarity? Do they review monthly metrics with the teams or recommend best practices?

👍 1
Christina Yakomin16:10:51

SRE Leads will consult on infrastructure/architecture decisions, ensure alert portfolios and SLIs/SLOs are being reviewed at the appropriate frequency, facilitate post-incident reviews across the product teams, and co-ordinate and conduct chaos and performance testing for the products in collaboration with the engineers on the teams. It's a role that is a combination of hands-on technical and facilitation

Bill Orr16:10:31

In other words, the SRE Lead workload is a balance between chaos testing and production app delivery

Bill Orr16:10:53

How does Vanguard approach the organic process of tuning alerts? Is there a strategy you recommend? Do you use the number of false positives to drive improvement? Curious if your approach is different other than just having teams add the work to their backlog and address it as time and capacity permits?

Christina Yakomin16:10:56

I recommend quarterly alert portfolio reviews at a minimum, but I was once on a team that did alert reviews weekly! I always suggest that people use Cory Watson's CASE methodology for alert construction as a guide for tuning alerts, and actually calculate their signal-to-noise ratio and track it.

Christina Yakomin16:10:28

Now, that's quite a bit of overhead, so not every team is doing this today, but I'm hopeful that adding SREs into the mix will help to ensure ALL teams have the available bandwidth to prioritize this critical work!

Bill Orr16:10:10

Fascinating. This is the first I heard of Cory Watson's CASE methodology. I will study up on that. Thank you

Christina Yakomin16:10:54

@rdaitzman and I followed along in the #ask-the-speaker-plenary channel during the talk. I'm going through this channel now to catch up on any questions we may have missed that were directed here!

👍 1
Bill Orr16:10:09

How long has Vanguard been using OpenTelemetry and how many applications are using it?

Christina Yakomin16:10:13

Switched to mobile and forgot to reply as a thread

Christina Yakomin16:10:26

I don’t know the exact numbers, but it’s at least a hundred so far! We started using it about a year ago, evaluating it a year and a half ago. In combination with Honeycomb for visualization of traces, it has been a total game changer!

Ann Perry - IT Revolution16:10:29

🎉 Let's get ready to welcome @smack from Wiley, presenting DevOps for Pandemics 🎉

🙌 1
Sean D. Mack16:10:09

Thanks @annp! Excited to be here.

Daniel Cahill - Engineer - Ontario Systems16:10:01

It sounded like you were scaling to meet demand as the pandemic started. With Wiley being used in online learning, were there already major lessons in scaling from summer to school year that helped you be prepared for the need to scale?

Sean D. Mack16:10:43

Yes. Most definitely. We have cyclical patterns which we prepare for all year round Most notably the back-to-school period in fall.

Sean D. Mack16:10:32

Of course, we had that + a pandemic this past year which was something we had never seen before 🙂

Gene Kim, ITREV, Program Chair16:10:08

Another great story of vast mobilizations due to COVID. Thank you @smack!

👍 1
Eric Mosher, NSA16:10:28

Hah! "Socially distance your applications." Really like that way of describing it

👍 1
Gene Kim, ITREV, Program Chair16:10:55

I love these stories of saves like this.

Sean D. Mack16:10:41

Not only responding but accelerating additional improvement!

Gene Kim, ITREV, Program Chair16:10:44

Graph showing fantastic drops in response times —

Hoda Alshami- Speaker- DevNetOps Journey, Nationwide Insurance16:10:41

Can you share sample of transparency and observability dashboards?

Sean D. Mack16:10:53

Sure @alshah1 The Business Continuity Dashboard was a prime example of this. We shared metrics about system performance and business performance across the business.

Sean D. Mack16:10:02

It's easiest to see this in contrast to siloed tooling where teams only have access to their data. Network teams have access to network data, database teams have access to db data, etc. Our approach was to fundamentally change this and get out of these information silos

Ganga Narayanan16:10:51

Loved it! Thank you!

👍 1
Jennifer Petoff16:10:31

Thank you, @smack. Enjoyed your talk and really appreciated the emphasis that you placed on training and the importance of a learning culture.

👍 1
Ann Perry - IT Revolution16:10:50

📣 We now welcome the team from ING – @aurel-george.proorocu @mihai.roman2 @misupriest1 presenting How's Your Bank Working From Home 📣

Sean D. Mack16:10:55

Thanks for joining for "DevOps for Pandemics"!

Mihai Popa16:10:09

We are here :)

Hoda Alshami- Speaker- DevNetOps Journey, Nationwide Insurance16:10:33

@smack sounds exactly what we are trying to do in my team, what kind of tools you have used for building Business cont. dashboard, aggregation from other monitoring tools, my guess, like Grafana

Sean D. Mack16:10:39

@alshah1 For this we used Power BI but we also use Grafana for more of day-to-day operational information.

Sean D. Mack16:10:10

Thanks for joining!

Geri O’Toole17:10:12

Im so intrigued by these trends and what other orgs experienced

Geri O’Toole17:10:56

days where we have no meetings! Ive heard others did this.

Meaghan Dop17:10:55

I think we had one or two official "no meeting days".. but wasn't a long term thing

👍 1
Tom Wojtusik (Tasktop)17:10:13

We've seen a trend toward fewer formal meetings throughout the week, many more smaller, impromptu Slack "meetings" as needed, AND blocks of "flow time," blocked on the full teams' calendars during which no meetings are allowed. Sort of a hybrid approach.


NOAA/NESDIS has started doing this on Fridays... but now I find that everybody's M-T are so full that when we need a new meeting to discuss something one-off meeting... then it ends up being on Friday because that's the only time everyone is free.

Tom Wojtusik (Tasktop)17:10:40

Too much work in progress, a chronic symptom for organizations to address.

Joe Waid - Manager, Delivery Engineering - Columbia Sportswear17:10:32

Walking one on ones are a great tool, I love that.

👍 4
George Proorocu17:10:02

Yes, this was something very appreciated by our engineers because people were really missing social interaction and in some cases they were not very comfortable to discuss different sensitive subjects on Teams. In my case I also had some people that were hired at the beginning of the pandemic, so we never managed to see each other face to face before our "walking 1:1".

Ganga Narayanan17:10:51

Good point about paying attention to "micro actions" with the camera on! As long as we can avoid Zoom fatigue!

👍 3
Geri O’Toole17:10:12

zoom fatigue is real.

Geri O’Toole17:10:55

I think that it's also real due to so much context switching if we are going from back to back. in which case, stop that. right?

👍 1
Ganga Narayanan17:10:54

Indeed it is! Maybe a 50-50 or whatever ratio works for camera on vs off? For example in 1:1's and workshops, camera on by invitation..

Mihai Roman17:10:03

unfortunately, the zoom fatigue is something we, all, have to learn to deal with it in the same way as we have learned to work from home for long periods.

Ganga Narayanan17:10:15

I also turn the live-captions on. Easy on the ears as well. 🙂

Mihai Roman17:10:13

@ganga.narayanan, not success recipe for on vs off camera. The context of the discussion/meeting is the one that decides most of the time for us. As human interaction is an important factor, I would say for 1:1 should be on 😉

Geri O’Toole17:10:49

yah,, the old "80 % of communication is nonverbal" is real too.

Geri O’Toole17:10:38

I have zero poker face so most know if I turn mine off it's to hide if I"m vehemently disagreeing with the conversation and composing myself. lol

💯 2
😉 1
Mihai Popa17:10:47

It's like everyone knows about it but we all forget about it

👍 1
Ganga Narayanan17:10:34

True true! Same with me - zero poker face! But we haven't forced people to turn their cameras on. A lot of people keep them off most of the time..

👍 1
Matt Ring (he/him) - Sr. Product/Engineering Coach, John Deere17:10:47

We struggle with this too. It is a common discussion point with both our team leaders on Scrum Masters. We gently encourage in subtle and fun ways, but don't push it hard.

Keith Langenberg17:10:11

How did ING deal with all the time zones in 6 countries when working a normal work day?

👍 1
Mihai Roman17:10:06

working 9 - 5 is “old school”, flexibility remains the key. Having the right environment and the space will solve any time zone differences.


I don't know if I agree with "9-5" is old school. The only way I can maintain a proper work/home-life balance is to keep strong start/end times to my work from home day. I strongly set 9-5. When my day is over my laptop is shut and work is OVER. There is no checking e-mails after the kids are in bed or anything like that for me.

Tom Wojtusik (Tasktop)17:10:23

Will the "new normal" at ING result in returns to physical offices? To what degree?

Geri O’Toole17:10:47

Thank you for sharing your story @misupriest1 and team

👍 2
Mihai Popa17:10:50

Good one and not an easy one. We make sure to plan the meetings so that most of us can align.

👍 1
George Proorocu17:10:51

Thank you all for joining our talk! Please let us know if you have any questions and we will reply asap.

Mihai Popa17:10:43

@tom.wojtusik not so much. 2 days a week in the office and the rest at home. The teams are asked to align so that in those 2 days everyone is in the office

thankyou 1
Ann Perry - IT Revolution18:10:24

🌟 Coming back from the break, let's welcome @bryan.finster486 on How to Misuse and Abuse DORA Metrics 🌟

👏 2
Gene Kim, ITREV, Program Chair18:10:59

Hey, @bryan.finster486 — looking forward to hearing about weaponizing DORA metrics!!! 😆

😂 5
Virginia Laurenzano NSA/MARFORCYBER18:10:22

defense unicorns ❤️ might have to rename my desk partner

😂 1
Trac Bannon (Speaker)18:10:13

To be read in a Brady bunch voice: "Dora, dora, dora"!

Mark Peters18:10:23

Hey Brian, how you doing? Weaponizing depends on from how far away I can launch my metric...

😆 2
Mark Peters18:10:03

Although, the details for weaponization are there, repeatable set, with clear results, that deliver a static response

Brian Smith18:10:26

I don't understand that last comment. Product teams and pipelines over scaling frameworks. Why not both?

Bryan Finster - Defense Unicorns (Speaker)18:10:53

Coupling teams was the opposite of our goals

Brian Smith18:10:50

What does that mean?

Brian Smith18:10:01

Coupling teams?

Bryan Finster - Defense Unicorns (Speaker)18:10:25

Release trains are designed for teams delivering together. We didn’t want that. It was slowing us down.

Brian Smith18:10:08

I am not sure that is accurate understanding of "Continuously Deliver and Release on Demand"

Geri O’Toole18:10:19

people were happier.. what a concept. 🙂

Ferrix Hovi - Head of DevOps - Siili18:10:30

@bryan.finster486 I expected more critique there for scalable frameworks 😉

😂 1
Brian Smith18:10:05

You started with deliverying one app: Jigsaw. That is not the same thing as delivering large integrated value at scale. What happens when you need the teams alligned to deliver value.

Gene Kim, ITREV, Program Chair18:10:10

I have so much respect for the efforts inside USAF to drive DevSecOps — so I couldn’t help but read this: Hope that the mission goes on, @bryan.finster486!

Geri O’Toole18:10:26

I read this too.

Bryan Finster - Defense Unicorns (Speaker)18:10:45

We are driving the mission forward. I had dinner with him after this. The mission continues.

Gene Kim, ITREV, Program Chair18:10:11

So good and so reassuring to hear this, @bryan.finster486 !!!

Bryan Finster - Defense Unicorns (Speaker)18:10:36

I only work where the mission lives. 😄

Trac Bannon (Speaker)18:10:09

This was a really poignant post by Nic. And Bryan is right... the mission, or should we say missions, continue!

Trac Bannon (Speaker)18:10:42

@genek - Nic served a great role as a change agent.

Gene Kim, ITREV, Program Chair18:10:57

So great to hear this, @tbannon!!!

Skylar Rudolph18:10:04

Coverage correlation with quality/effectiveness seems to rear its head time and time again. Relatively recent article around this topic… not peer reviewed, but purely observational.

👍 1
Javier Magaña - Walmart18:10:05

My takeaway is that the expectation needs to be a sentence. As a ... I want ... so that... Instead of just a couple of words that are easier to interpret however I want.

Daniel Cahill - Engineer - Ontario Systems18:10:53

I was working on making some better metrics yesterday. How do you improve the quality and quantity of metrics about how much time is spent on helping in different areas without logging minute by minute? Is there an interval that has worked well? I'm wanting to enable other teams to get better outcomes but trying to show both that it is where I'm spending time and actually proving improvements.

Geri O’Toole18:10:30

Dojos. wicked fast learnings.

👍 2
Topo Pal - Programming Committee Member18:10:36

Watching @bryan.finster486’s talk

Gene Kim, ITREV, Program Chair18:10:15

“handed out six cases of books, handing it out to people, saying, ‘please read!’” 😆

Brian Smith18:10:17

Accelerate is a great book

Brian Smith18:10:28

I love this book.

👍 2
Gene Kim, ITREV, Program Chair18:10:56

“…it all started out well…“. hahaha

Virginia Laurenzano NSA/MARFORCYBER18:10:02

LOVE the Princess Bride reference

Jim Tranquill18:10:14

inconceivable 🙂

Topo Pal - Programming Committee Member18:10:40

Productivity went through the roof yesterday!

Brian Smith18:10:44

This is an incredibly frustrating problem, "People don't read books" I read this book and could not get others to read it in the AF.

Deanna Stanley18:10:46

All my sponsors use this to prove what teams are "good" or not

Javier Magaña - Walmart18:10:45

Well it is easy, and simple to point and go 😄

Mark Peters18:10:55

Like the, Don't focus on metrics, metrics are a measurement, not a goal

Gene Kim, ITREV, Program Chair18:10:15

Is anyone else covering their eyes? Haha

🙈 3
Geri O’Toole18:10:16

these make my head hurt, @bryan.finster486

👀 1
Virginia Laurenzano NSA/MARFORCYBER18:10:36

I'm having some nightmarish flashes for sure

👍 2
Jim Moverley18:10:46

Can't lie.. i've become a @bryan.finster486 fan! 😄 always speaking great truths..

👍 1
Mark Peters18:10:42

Can' ask for context with metrics? What kind of heresy is that? Apples to apples not oranges to elephants

👍 1
Bryan Finster - Defense Unicorns (Speaker)18:10:02

And for the love of all that’s holy, READ PAST PAGE 19!!

👍 2
Gene Kim, ITREV, Program Chair18:10:19

@bryan.finster486 So for those fictitious dashboards (ahem), what was the (fictitious) context? In what way were they misrepresenting the situation?

Bryan Finster - Defense Unicorns (Speaker)18:10:04

“We are awesome. Look how many times we shipped!”

Bryan Finster - Defense Unicorns (Speaker)18:10:35

Strange that MTTR and lead time were so out of whack.

Geri O’Toole18:10:47

happier teams. what a concept. 🙂

Topo Pal - Programming Committee Member18:10:53

Some of the numbers were for 10 applications and some are 200 @genek

Gene Kim, ITREV, Program Chair18:10:16

OMG. Hahahah, @topo.pal @tapabrata.pal

Deanna Stanley18:10:31

"Culture? I don't care about culture! Just give me a pipeline!"


oh no, please not

Deanna Stanley18:10:06

And then they wonder why they're not a high performing org. OBVIOUSLY they need new contractors...

Ferrix Hovi - Head of DevOps - Siili18:10:05

Here's what I've been thinking. How many of the Five Ideals do the four metrics cover. Not a lot.

Gene Kim, ITREV, Program Chair18:10:12

“some of them were actually finishing things”

👀 2
Daniel Cahill - Engineer - Ontario Systems18:10:38

Is there a place to watch for when this is open sourced?

Brian Smith18:10:50

Page 19- Table 2.2 and 2.3?

Bryan Finster - Defense Unicorns (Speaker)18:10:22

Yep, read that and you’re done :rolling_on_the_floor_laughing:

Brian Smith18:10:57

Okay, your saying Don't stop there. Got it.

Geri O’Toole18:10:04

there it is, Bryan! you've got Deming in there for me.

❤️ 1
Scott Kellerman (DevEx Product Owner, Vanguard)18:10:18

This is a great talk! My organization has absolutely had a similar experience. We've learned that DORA metrics are great indicators but two "next level" measures are Value Stream Flow and Business Outcomes!

👍 2
Daniel Folkes - Carmax18:10:57

How do you get hose WIP radiator?

Pete Heller18:10:07

Love this - I've been screaming from the rooftops that no organization should ever be going faster for the sake of going faster. What is the downstream benefit to actual business outcomes!!

Geri O’Toole18:10:25

im exhausted of screaming. aren't you? lol

Bryan Finster - Defense Unicorns (Speaker)18:10:30

Yes! We are solving a business problem!

💯 1
Gene Kim, ITREV, Program Chair18:10:19

This reminds me of the quote: > I made up the term ‘object-oriented’, and I can tell you I didn’t have C++ in mind > -- Alan Kay, OOPSLA ’97 “This is not what we had in mind when we wrote Accelerate.” 😆

❤️ 1
Topo Pal - Programming Committee Member18:10:51

There are "DORA Metrics Ready" tools these days. I am not kidding

Jim Moverley18:10:55

haha you beat me to it @kapoor.vaidik

Gene Kim, ITREV, Program Chair18:10:07

Oh, and DORA metrics showed up in GitLab S-1 filing!!! Wild!!!

🎉 2
😃 1
Jim Moverley18:10:54

supplying the cardboard for the cargo cult mob!

Gene Kim, ITREV, Program Chair18:10:16

It’s part of the lifecycle of ideas. I have equanimity about this. 🙂

Topo Pal - Programming Committee Member18:10:57

I know team doing once a month deployment use DORA metric dashboard

Gene Kim, ITREV, Program Chair18:10:02

Oof. Hoefully it only took a couple of minutes that wire that dashboard…. 🙂

Bryan Finster - Defense Unicorns (Speaker)18:10:07

I’ve talked to some of them.

🤐 1
Pete Nuwayser - IBM18:10:12

This slide right here, about metric relationships, is everything.

💯 3
Pete Nuwayser - IBM18:10:02

I say that because the DORA metrics have to be traceable to business outcomes in order to avoid them becoming proxy/vanity metrics.

Graham McGregor18:10:31

What have you found effective to implement OKRs so they don't devolve into just a new form of arbitrary deadlines?

Bryan Finster - Defense Unicorns (Speaker)18:10:57

Jon Smart’s got it nailed. He helped kick them off in Platform 1

Mark Peters18:10:07

"flow metrics from teams moving cards" Ive had this conversation, our pipeline is based on when the install happened, time on keyboard, not when someone moved the JIRA ticket the following monday

Trac Bannon (Speaker)18:10:15


❤️ 1
Geri O’Toole18:10:22

why is it everything you are saying resonates?

Mark Peters18:10:25

I was going to say it was the large room

😂 1
Seth Tager - Salesforce18:10:21

Education is essential

Craig Cook - IBM18:10:40

Yes, provide education like Coaching and Dojo's

Pete Nuwayser - IBM18:10:51

I might have missed it, but did @bryan.finster486 mention that the throughput metrics are about size of change vs. speed of change?

Daniel Cahill - Engineer - Ontario Systems18:10:00

What would your official training instead of hobbyists look like? For now, I'm at a job where hobbyists is the main way I am teaching myself 🙂

Brian Smith18:10:29

Aw man, I got to read it again?

Brian Smith18:10:47

Good thing I have it on Kindle and Audible.

Tashfeen Mahmood18:10:10

This talk is so profound! I feel like I will need to listen to it a few times. There are so many nuggets in each sentence

👏 4
Geri O’Toole18:10:39

I want to speak it word for word at my org.

👍 1
Geri O’Toole18:10:18

DONT Measure people. invest in them.

❤️ 3
👍 2
👏 1
Stephanie DeGuire18:10:40

This is so timely for us. We're trying to figure out what we could/should measure to show our dojo adds value to the organisation . I'm happy to see I'm not the only one looking at typical DevOps metrics and wondering if focusing on them could become problematic.

Brian Smith18:10:51

A good portion of SSH in the IT rev library. There is a link in the reading room.

Jim Moverley18:10:52

ace resources 😄 thanks @bryan.finster486 amazing talk!

👏 5
🙌 1
Mark Peters18:10:19

What's that? SSH into the IT Rev library?

😂 1
Geri O’Toole18:10:26

Id love to see those suggestions too, Bryan.

Scott Kellerman (DevEx Product Owner, Vanguard)18:10:46

Does anyone have any recommendations for how to measure your Value Stream Flow.? Pipelines often have many tools which make measuring end to end flow challenging. Any recommended dashboarding or measurement tools and or techniques?

Gene Kim, ITREV, Program Chair18:10:55

Thank you!!! Keep collecting stories, @bryan.finster486 — let’s figure out what to do with this next year!!! And keep up the great work at USAF!!!

💯 2
Ferrix Hovi - Head of DevOps - Siili18:10:04

@bryan.finster486 Thanks. Some slides are going to be stolen 😄

Jim Moverley18:10:41

hahah was thinking same!

Jim Moverley18:10:50

love the iceberg..

jeff.gallimore (CTIO - Excella, he/him)18:10:08

defense unicorns!!! :unicorn_face:

Andy Nelson18:10:10

👏 thanks @bryan.finster486 that was a dense meal right after lunch

😆 1
Brian Smith18:10:12

Thanks Brian.

Sujay Solomon18:10:16

superb session @bryan.finster486 - thank you so much!

Mark Peters18:10:21

Great talk Bryan!

Jason Trent18:10:34

This was a great sessions! Thanks @bryan.finster486!

Dian Hansen18:10:45

@bryan.finster486 phenomenal talk!!

Stephan Stapel18:10:54

Thanks @bryan.finster486, great talk!

Tashfeen Mahmood18:10:58

@bryan.finster486 - That was cool!

Donald Fischer - Tidelift (Speaker)18:10:13

Great stuff @bryan.finster486!

Bryan Finster - Defense Unicorns (Speaker)18:10:41

Thanks so much everyone. I’ll be at the bar later. 🙂

🍻 4
👏 12
🍺 3
🎉 2
Seth Tager - Salesforce18:10:46

Bryan, great talk, but it is just a starting point. The devil is in the details! Is there a BOF for talking details?

👍 1
Bryan Finster - Defense Unicorns (Speaker)18:10:15

We should do that. I agree. It’s just the beginning. I wanted to say so much more.

Trac Bannon (Speaker)18:10:43

In 25 minutes, we can only scratch the surface!

👍 1
Seth Tager - Salesforce18:10:58

I'll look for folks in the bar later

Matt Ring (he/him) - Sr. Product/Engineering Coach, John Deere18:10:00

I vote for a BOF, Happy Hour and then After-Happy-Hour Happy Hour session @bryan.finster486. Just let me know when and where. 😃

Bryan Finster - Defense Unicorns (Speaker)19:10:21

Hanging out at the empty table at BOF

Brian Smith18:10:16

Can I have that bitly link for continuous Delivery again?

Ann Perry - IT Revolution18:10:30

And now, we're honored to have @dff here to present: Thinking Upstream About White House Cybersecurity Executive Order 14028 

👏 2
Donald Fischer - Tidelift (Speaker)18:10:16

Thanks @annp, great to be here!

Trac Bannon (Speaker)18:10:14

Looking forward to this talk, @dff!

👋 1
👏 4
Donald Fischer - Tidelift (Speaker)18:10:05

Great to see you here @tbannon 👋

Virginia Laurenzano NSA/MARFORCYBER18:10:54

anyone here participate in any of the NIST solicitations?

Virginia Laurenzano NSA/MARFORCYBER18:10:19

full transparency: I did. just curious about others

Donald Fischer - Tidelift (Speaker)19:10:26

Tidelift’s specific comments Re: Software Bill of Materials Elements and Considerations here:

👍 1
Donald Fischer - Tidelift (Speaker)18:10:47

I also appreciated Google’s comments in the NTIA process, which included this

Donald Fischer - Tidelift (Speaker)19:10:50

Here’s a link to the 2021 Tidelift open source maintainer survey referenced in the talk:

Donald Fischer - Tidelift (Speaker)19:10:51

And here’s the companion survey of organizations that build with open source, also cited in the talk--

Donald Fischer - Tidelift (Speaker)19:10:21

If you’re interested in following up on anything in the talk or finding out more about Tidelift, my contact info: <|>

👍 1
👏 1
Craig Larsen - he/him - Solution Design Group Mpls19:10:13

Love this talk, @dff! One of my concerns with paying the maintainer is that would maintainers intentionally add bugs (there was a great article on this regarding the U of M and open source software). I think we can trust the maintainers because there should be a number of them with checks and balances. Do you have any thoughts around this?

Donald Fischer - Tidelift (Speaker)19:10:48

My perspective--it’s all in how you setup the incentive system. For example, Tidelift doesn’t pay a bounty per issue resolved, instead we pay maintainers who agree to work with us to ensure their software meets specific security, licensing, and maintenance standards.

💯 1
Brian W. Spolarich - Cal Poly19:10:31

What kind of project coverage does Tidelift have?

Donald Fischer - Tidelift (Speaker)19:10:17

TLDR: Broad coverage of language-level application development packages in JavaScript, Java, PHP, Ruby, Python, .NET and emerging coverage of Go, Rust

🎯 1
Glenn Wilson, Author of DevSecOps19:10:44

How scalable is managed open source? There are some orgs that use a very high number of open source dependencies

Donald Fischer - Tidelift (Speaker)19:10:31

Yep, typically the organizations we’re working with are using tens of thousands of discrete open source dependencies. You can see the breadth of our current coverage at

👍 1
Craig Cook - IBM19:10:20

Does a donor specify which projects they want to support, or is it a "general pool"?

Donald Fischer - Tidelift (Speaker)19:10:16

The more Tidelift’s paying subscribers use a particular open source package in their applications, the more income that partnered maintainer receives. Think kind of like Spotify artists paid based on song play counts.

Gene Kim, ITREV, Program Chair19:10:58

Wow! What a big idea! Thank you @dff — I will definitely be watching your talk later this week! I was so much looking forward to your presentation! And thanks for everything you’ve done to advance important pieces of the software ecosystem, such as Akka, etc. Looking forward to great interactions ahead!

❤️ 1
Jim Moverley19:10:50

brilliant talk thanks @dff

❤️ 1
Christopher Pryce19:10:09

What does Tidelift do to enforce that critical vulnerabilities are addressed?

Christopher Pryce19:10:01

• Sure, but in that process, I see a maintainer can: Create an exception - The vulnerable release stays approved in your catalog.

Christopher Pryce19:10:10

There doesn’t seem to be a way to enforce a maintainer to fix a vulnerable transient dependency? Or am I missing something?

Donald Fischer - Tidelift (Speaker)19:10:58

That’s referring to the fact that as an organization using Tidelift, you can choose to override Tidelift’s maintained-based guidance — in other words if you want to “force approve” a release where we’ve flagged a security issue.

Donald Fischer - Tidelift (Speaker)19:10:17

For transitive dependencies, we work jointly with our maintainer network — sometimes a few packages need to be independently updated in coordination (say, one to fix the vuln itself and another to update a version-locked dependency on it)

Ann Perry - IT Revolution19:10:12

🔆 Excited to now introduce @tbannon, here to talk about DevOps’ Missing Link: Data 🔆

👏 2
🎉 2
Gene Kim, ITREV, Program Chair19:10:25

What’s so difficult about reproducing errors?! This is so good, @tbannon 😆 😆

😁 1
Gene Kim, ITREV, Program Chair19:10:08

State management. 😆 This hurts. cc @stephen

😆 4
Ferrix Hovi - Head of DevOps - Siili19:10:32

I know a myriad of teams who have this problem but are not spending time on it in a way that they'd regard it as a priority... So hard.

Gene Kim, ITREV, Program Chair19:10:26

“I frown on using production data because… we’re not masking it. Less than 50% mask their production data” 😱

🤐 2
Trac Bannon (Speaker)19:10:24

Build. borrow, and buy!

Ferrix Hovi - Head of DevOps - Siili19:10:06

Hey, you could steal it from a competitor and mask it to your needs... with 1:2 odds 😄

Ferrix Hovi - Head of DevOps - Siili19:10:37

The other option seems to be gatekeeping teams from any test data at all.

Trac Bannon (Speaker)19:10:01

At a minimum, it can't be an afterthought.

👍 1
💯 1
Ferrix Hovi - Head of DevOps - Siili19:10:49

"We can't test because we ran out of fake social security numbers."... I am so near to shellshock right now.

😀 2
Scott Jaffa (Principal Engineer, ValidaTek)19:10:37

My old favorite was “we don’t have good enough fake data, so we won’t test at all” from long ago.

Gene Kim, ITREV, Program Chair19:10:54

😱 This is so great, @tbannon

💯 4
Trac Bannon (Speaker)19:10:47

Thanks @genek! This is not the sexiest topic AND yet still needed.

👍 1
Gene Kim, ITREV, Program Chair19:10:20

What? It’s about correctness, testing and data. Doesn’t get much more interesting than that! 🙂

Trac Bannon (Speaker)19:10:55

I guess quality is sexy... eh?

Pedro Jordan19:10:54

Nice talk !

❤️ 1
Joe Arrowood19:10:08

Thank you Tracy, great talk.

Dian Hansen19:10:28

This made explicit an area that is often not talked about - thanks! @tbannon

❤️ 1
Topo Pal - Programming Committee Member19:10:29

"We have full test automation" - "How do you manage test data" - "Hmmm. We always have problem with that"

Trac Bannon (Speaker)19:10:58

right?! Can't have one without the other!

Jim Moverley19:10:31

awesome talk @tbannon test data security is an interesting area for sure!

👏 1
Ferrix Hovi - Head of DevOps - Siili19:10:02

Thanks @tbannon. A great collection of ground rules. A lot of people need this revelation.

👏 2
Dave McNierney19:10:44

Excellent presentation... thanks @tbannon!

👍 1

Reminder: The plenary sessions are starting again in 5 minutes. Start making your way back to your browser and join us in #ask-the-speaker-plenary to interact live with the speakers and other attendees.

Andy Nortrup - Director of PM at Tanium21:10:52

Drat, not writing fast enough that was: 1. Competance 2. ? 3. Accountability 4. Delegated Authority.


2. Character and integrity

jeff.gallimore (CTIO - Excella, he/him)21:10:59

@andy744 all the action is happening in #ask-the-speaker-plenary now 🙂