Fork me on GitHub
#ask-the-speaker-track-2
<
2020-06-25
>
Jess Meyer - IT Revolution (she/her)10:06:30

After break, welcome @lewir7 @lucasc5 for Q&A!

πŸ‘‹ 1
Bryan Finster - Walmart (Speaker)10:06:41

Looking forward to this one. "How do you level up Audit?" is a frequent conversation we have in the Dojo Consortium.

πŸ‘ 1
Matt Cobby (Director of Engineering, Deloitte)10:06:24

Great paper! It's such an interesting topic.

Areti Panou10:06:14

I have shared this paper so many times I lost count

John Willis10:06:32

One of the biggest problem with audit in most organizations is that it is hard to reconcile cloud-native activities with ITSM processes.

John Willis10:06:32

One of the biggest problem with audit in most organizations is that it is hard to reconcile cloud-native activities with ITSM processes.

Giulio Vian, Unum10:06:15

interesting concept that separation of duties is between software and human

πŸ‘ 1
John Willis10:06:23

I find the mis-match starts with an inaccurate CMDB. If audit relies on CI/Service Owner this becomes an issue. Typically Cloudnative activities start with git evidence. Typically organizations don't have callbacks between CI and git evidence.

πŸ’― 2
John Willis10:06:23

I find the mis-match starts with an inaccurate CMDB. If audit relies on CI/Service Owner this becomes an issue. Typically Cloudnative activities start with git evidence. Typically organizations don't have callbacks between CI and git evidence.

πŸ’― 2
Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW10:06:40

Absolutely. It’s been crucial for what we’re building.

John Willis10:06:17

You need evidence in an attestation store that comes directly from the pipeline not the chage record.

Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW10:06:49

Yes. We link the pipeline to the change record - actually raise and approve the CR automatically from the pipeline.

Matt Cobby (Director of Engineering, Deloitte)10:06:04

A common pattern I see is that the CMDB is "precious" and so only "trusted" people (aka Service Management) is allowed to update it. This in turns means that the people with the correct information can't update it and it gets out of date. As it gets out of date it becomes less valuable and less used and so less updated. Repeat and rinse

John Willis10:06:44

In my experience I have not met a company that has better than a 40% accurate CMDB

Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW10:06:20

We are getting better. Incentivising data quality in the CMDB is hard, but we are getting there.

John Willis10:06:39

It all goes back to service owners and CI's.

πŸ‘ 1
Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW10:06:42

As much automation as possible - and cloud CIs 100% automated & dynamic.

John Willis10:06:06

It get worse with microserices and even worse with serverless... even worse than worse with Service Mesh activities.

πŸ˜€ 2
John Willis10:06:34

Evidence needs to be objective not subjective. Or said as attestations need to be objective not subjective. The previous mentioned paper suggests all attestations in a chain of immutable signed lists.

John Willis10:06:34

Evidence needs to be objective not subjective. Or said as attestations need to be objective not subjective. The previous mentioned paper suggests all attestations in a chain of immutable signed lists.

Bryan Finster - Walmart (Speaker)10:06:48

An immutable BOM attached to each artifact is my goal.

Giulio Vian, Unum10:06:58

which can be seen as the other side of the coin of repeatable builds

Giulio Vian, Unum10:06:39

(or reproducible)

Bryan Finster - Walmart (Speaker)10:06:44

Yes, but I want a freight bill in production with full traceability of what is in it, how it was approved, how it was tested, etc.

Giulio Vian, Unum10:06:02

I am also pushing org in that direction as we have some disconnection between tools that requires manual (thus arbitrary) step to glue activities and complete a process

Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW10:06:53

This is all super familiar - it’s exactly what we’re doing :)

πŸ‘ 2
Bryan Finster - Walmart (Speaker)10:06:59

Pipelines that enforce controls are key.

πŸ‘ 4
Bryan Finster - Walmart (Speaker)10:06:59

Pipelines that enforce controls are key.

πŸ‘ 4
Stephen Magill [Sonatype]10:06:38

And then collecting the right evidence from key controls. So glad to see so many people doing this or looking to!

Matt Cobby (Director of Engineering, Deloitte)10:06:04

This is what our team built to help with Compliance Attestations linked to CI/CD pipelines https://www.youtube.com/watch?v=ll50dAiKPoI

πŸ˜€ 1
thankyou 1
Bryan Finster - Walmart (Speaker)10:06:52

@lucasc5 how to you audit the pipelines in a reasonable amount of time?

Bryan Finster - Walmart (Speaker)10:06:52

@lucasc5 how to you audit the pipelines in a reasonable amount of time?

Clarissa Lucas, Author and IT Audit Leader10:06:57

@bryan.finster Thanks for the question! Our typical audits take about 8-12 weeks from start to finish. We are increasing our use of analytics and automated testing (fewer manual tests), which is helping to reduce the amount of time spent testing.

Bryan Finster - Walmart (Speaker)10:06:53

What controls do you have automated in the pipelines today?

Clarissa Lucas, Author and IT Audit Leader10:06:09

@bryan.finster As an organization, many processes and controls are still manual. We are moving to automation of certain controls to increase efficiencies. One example is automated vulnerability scanning. @lewir7 and I can do a little more digging and follow back up with a few more examples if you're interested.

Bryan Finster - Walmart (Speaker)10:06:13

Yes, I'd love to know what you have as requests automate in the pipeline. I work in the CD platform area and keep telling Audit that if it's not in the pipeline, it's probably not happening. I'm very interested in what others are trying to automate to make compliance a reality.

πŸ‘ 2
John Willis10:06:20

@matthew.cobby I've seen that work... It's great stuff.

John Willis10:06:20

Enforcement is not Evidence IMHO

John Willis10:06:58

A better question is what exactly doe the audit look for. I find that the answer is basically these 5... 1. Change record service owner 2. Time window for deploy 3. backout activities

πŸ‘ 1
John Willis10:06:58

A better question is what exactly doe the audit look for. I find that the answer is basically these 5... 1. Change record service owner 2. Time window for deploy 3. backout activities

πŸ‘ 1
Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW10:06:05

Good auditors will look for evidence of what you specified you would do in your formal process documentation :)

πŸ‘ 1
Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW10:06:37

So if you can update that to be cloud native then you’re on the right track

Vlad Ukis10:06:54

5 points or 3 points?

Clarissa Lucas, Author and IT Audit Leader10:06:53

@jwillis A lot of these in the list are related to risks and controls (e.g., not meeting an SLA for time window to deploy could impact the production environment). Auditors should be seeking to understand how the risk is mitigated, rather than sticking to a list of pre-defined controls.

πŸ‘ 3
John Willis10:06:24

this is hard to reconcile with cloudnative activities..

John Willis10:06:24

this is hard to reconcile with cloudnative activities..

Clarissa Lucas, Author and IT Audit Leader10:06:40

@jwillis I'd love to connect with you and discuss how this could apply to cloud native activities.

Stephen Magill [Sonatype]10:06:38

Great talk! This idea of using DevOps pipeline evidence to streamline compliance is really getting traction. Great to hear the success stories.

❀️ 2
John Willis10:06:58

fyi... We are doing a second version of the Devops Automated Governance paper this summer with a specific focus on policy... policy as code, policy error budgeting.

πŸ’― 4
πŸ™Œ 1
John Willis10:06:58

fyi... We are doing a second version of the Devops Automated Governance paper this summer with a specific focus on policy... policy as code, policy error budgeting.

πŸ’― 4
πŸ™Œ 1
Areti Panou10:06:55

@lucasc5 and @lewir7 Thanks for sharing your experience. I always found that talking to auditors saved us a lot of work.

πŸ’― 2
thankyou 1
βž• 2
❀️ 1
Areti Panou10:06:55

@lucasc5 and @lewir7 Thanks for sharing your experience. I always found that talking to auditors saved us a lot of work.

πŸ’― 2
thankyou 1
βž• 2
❀️ 1
Rusty Lewis, Specialist - IT Auditor10:06:32

Thank you! If I could impart any additional advice, it would be to continue, or even begin your journey today in cultivating a culture of collaboration with your auditors. This can have a tremendous impact in the overall risk management process!

Matt Cobby (Director of Engineering, Deloitte)10:06:24

@lucasc5 @lewir7 - Great talk, thank you for sharing.

❀️ 2
David Jungwirth10:06:06

Hello and welcome everybody to our session!

Jess Meyer - IT Revolution (she/her)10:06:15

Welcome @david.jungwirth @max.ehammer ! Thank you @lucasc5 and @lewir7 !

David Jungwirth10:06:35

"We will do e-Commerce - whatever it takes"

1
David Jungwirth10:06:50

"Not many of us had experience in such dynamic environments, where requirements change constantly"

David Jungwirth10:06:27

"A typical food retail use-case is that you at least have 50 products in your cart, often more than 100 items..."

Arnab Nandi10:06:55

@david.jungwirth and @max.ehammer didn't get the point of regulation? Could you elaborate please? in the context of eCommerce, regulations should help...?

Max Ehammer11:06:44

The aspect of regulation is with respect to comply to local laws that restrict you somehow how you can deliver goods - for example you are not allowed to mix up meet and fruits in one delivery item

πŸ‘ 1
David Jungwirth11:06:45

"some business processes touch up to 20 IT systems"

Joachim11:06:08

This is a great case study from outside the early adopter agile and DevOps bubble. Thanks for sharing!

πŸ‘ 3
David Jungwirth11:06:17

"Alone with improvements in process and culture we achieved a deployment time improvement from one week to one day"

Joachim11:06:57

Sorry that isn't very inclusive (article in German): https://www.derstandard.at/story/2000116456865/spar-zu-online-shop-bestellungenkapazitaetsgrenzen-erreicht - maybe you will get to this - how do align tech to business outcomes?

Joachim11:06:57

Sorry that isn't very inclusive (article in German): https://www.derstandard.at/story/2000116456865/spar-zu-online-shop-bestellungenkapazitaetsgrenzen-erreicht - maybe you will get to this - how do align tech to business outcomes?

Joachim11:06:48

Or to put this differently - how do you align e-commerce with the logistical aspects (which seem to have been the bottleneck here).

David Jungwirth11:06:55

@joachimsammer - this is an article regarding COVID times, which created a huge demand on all kind of e-Commerce retailers. Max will explain business impact of the conduced improvements in general in a few seconds...

thankyou 1
Max Ehammer11:06:59

The logistic cost is the biggest painpoint in food e-commerce, however, our business is trying different aspects of lowering the cost of logistics. This starts with the commissioning of the goods, delivering the goods, and return of goods - all with support of proper IT solutions

Max Ehammer11:06:50

Until there is no appropriate business model, the growth is limited that's for sure

βž• 1
Roman Lublinsky11:06:44

@max.ehammer what was exactly the cultural issues?

Max Ehammer11:06:00

The cultural issues were mainly related to different work behaviour. Coming from mainly waterfall driven projects it's a huge change adopting agile methodologies

Roman Lublinsky11:06:57

Within IT only or at business side as well?

Max Ehammer11:06:41

Definitely both sides - if you ask me today, business is still lacking behind although they are catching up

Max Ehammer11:06:04

Business and IT are getting closer πŸ™‚

Arnab Nandi11:06:03

πŸ‘Great talk, thank you!

Joachim11:06:12

Thanks πŸ‘

David Jungwirth11:06:56

Thanks for joining the talk!

David Jungwirth11:06:23

We are still around some more time in case you have further questions - just let us know...

Jess Meyer - IT Revolution (she/her)12:06:22

Welcome to our next speaker @stephen!

Stephen Magill [Sonatype]12:06:01

Hi everyone!

πŸ‘‹ 8
Thomas Williams12:06:39

Stephen, are you guys doing the same open source study for 2020, that you and Gene presented on yesterday? Have you already done the research?

Thomas Williams12:06:39

Stephen, are you guys doing the same open source study for 2020, that you and Gene presented on yesterday? Have you already done the research?

Stephen Magill [Sonatype]12:06:29

Yes, although this year we’re focusing on the consumer side. Currently analyzing the results of a survey we sent out in January to assess the impact of various practices on secure use of open source.

πŸ™Œ 1
Thomas Williams12:06:23

Got it. Looking forward to seeing that data. Are you guys using Datomic after-all?

Thomas Williams12:06:32

I'm sure you guys are close to done, but, please let me know if there is any way I can contribute, or help.

πŸ‘ 1
Stephen Magill [Sonatype]12:06:41

The survey data is small enough that we can analyze it locally using Python with pandas and scikit learn (some of my favorite tools). But I know @genek101 is still using Datomic for some cool related analysis of open source usage data.

Thomas Williams13:06:07

Sweet. Pandas is great.

Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW12:06:18

Continuous Everything again. Theme of the conference.

πŸ’― 3
Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW12:06:18

Continuous Everything again. Theme of the conference.

πŸ’― 3
Pete Nuwayser - IBM12:06:10

We need a talk called Continue-ish to Continuous

πŸ’― 2
πŸ˜€ 2
Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW12:06:01

Also: β€œfree up humans to do other things” - is the flip side of the automation coin that gets missed so much

πŸ‘† 3
Stephen Magill [Sonatype]12:06:45

The β€œbeacons” in Rasmus’s talk from this morning are another great example of this approach.

πŸ’― 1
Pete Nuwayser - IBM12:06:58

"It's a government effort but there are community elements." And thank goodness for that.

Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW12:06:50

This is why I love SonarQube these days - fix new code now vs fix old code over time.

Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW12:06:50

This is why I love SonarQube these days - fix new code now vs fix old code over time.

Stephen Magill [Sonatype]12:06:50

And check out Muse for a SonarQube-type approach but for deeper security and reliability bugs πŸ˜‰

πŸ˜€ 1
Thomas Williams12:06:27

I was gonna ask, is it better than MuseDev? ;)

Ann Marie Fred - Red Hat12:06:00

One nice thing I’ve seen from Sonarqube: it’s possible for a single developer to fix dozens of security and code smells in 1-2 days, because it gives good, concrete fixes to implement.

πŸ’― 1
Pete Nuwayser - IBM12:06:25

"Developers never want these results in the first place." 😭

Stephen Magill [Sonatype]12:06:34

Tell me if you have particularly good stories about the dev / security conflict.

Pete Nuwayser - IBM12:06:13

This talk is so timely - exactly what I'm working on rn

πŸ‘ 1
Stephen Magill [Sonatype]12:06:49

Very much an inspiration for what we’re doing at MuseDev, btw.

Matt Cobby (Director of Engineering, Deloitte)12:06:22

We run a GitHub Enterprise for a few thousand devs and I've long believed that it's our biggest collection of untapped intelligence about how our company works.

Andy Sturrock12:06:02

Our CISO sees DevOps and automated tools as the way to keep the company safe. So credit to our Digital Security team, they help build things like CheckMarx into our pipelines. So it's way less of a conflict these days.

πŸ˜€ 1
Andy Sturrock12:06:02

Our CISO sees DevOps and automated tools as the way to keep the company safe. So credit to our Digital Security team, they help build things like CheckMarx into our pipelines. So it's way less of a conflict these days.

πŸ˜€ 1
Stephen Magill [Sonatype]12:06:43

It’s so effective when security works with devs to provide tools and services that integrate with their workflows.

πŸ’― 3
πŸ‘ 1
Stephen Magill [Sonatype]12:06:33

Agreed. They do such great things with their pipeline there!

Andy Sturrock12:06:18

How many of their teams are at that level? I'd say our very best teams are close to that, but they are a bit unicorn-y compared to most of our teams!

Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW12:06:54

I believe that this is their standard pipeline. But Topo Pal isn’t around this year to answer, I don’t think.

Chris12:06:08

Here a bit the opposite we initiated it and proposed our central sec team to use as well as relieving some of their manual work. But we’re still at the beginning and have to support our dev in fixing results from the chkmrx scans

Andy Sturrock12:06:22

That's very impressive if so.

Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW12:06:56

Yeah, it’s a 4-5 year aspiration for us. We have around 8-10 of the 26 checks implemented.

Thomas Williams12:06:05

@stephen is there a timeframe for a release of MuseDev "MDP"?

Thomas Williams12:06:05

@stephen is there a timeframe for a release of MuseDev "MDP"?

Stephen Magill [Sonatype]12:06:55

Yep, our private beta is available now to DOES attendees (https://does.muse.dev) and full public launch in the coming weeks.

Thomas Williams12:06:38

Any info on pricing when it's released? We can take it offline as well.

Stephen Magill [Sonatype]12:06:08

Yep, I’ll DM you that!

John Willis12:06:32

This is a pattern i've seen with high performers. They add as many tools as possible.

John Willis12:06:12

One org let their devs push a jar to their system and it spit out a report. The system had about 20 analysis tools.

John Willis12:06:12

One org let their devs push a jar to their system and it spit out a report. The system had about 20 analysis tools.

Thomas DuBuisson12:06:54

Oh neat. Is this Jar to report system home grown?

Thomas DuBuisson12:06:37

And how do you interact? Is this an editor integration, a brower? So many questions on this one.

Stephen Magill [Sonatype]12:06:25

95% of reported bugs fixed when surfaced in code review.

Stephen Magill [Sonatype]12:06:42

(vs batch mode long after the code was written, which never works well)

John Willis12:06:25

yes.. but it included a lot of OSS and non OSS tools

John Willis12:06:40

the main point is that more analysis higer effecacy

πŸ‘ 1
John Willis12:06:44

Still wondering 95% of what... not all dependancies vulnerabilities...

John Willis12:06:44

Still wondering 95% of what... not all dependancies vulnerabilities...

Stephen Magill [Sonatype]12:06:33

95% of code errors (in the application code, not dependencies) fixed when analysis was deployed in code review at Google.

Stephen Magill [Sonatype]12:06:50

(code errors surfaced by the tools that is β€” 95% β€œfix rate”)

Stephen Magill [Sonatype]12:06:05

there are surely still undetected errors in there

John Willis12:06:27

code errors.. thanks...

John Willis12:06:05

Another looming issue is container build and run polymorphic attacks.

Tom Ayerst12:06:12

Quality in many axes, automate to the max but recognise the unique value people actively poking things brings

πŸ‘ 2
Tom Ayerst12:06:12

Quality in many axes, automate to the max but recognise the unique value people actively poking things brings

πŸ‘ 2
Andy Sturrock12:06:54

Add an automated test for anything they find though so it doesn't reoccur.

πŸ‘ 1
Tom Ayerst12:06:12

@rohrersm Bolton is Brilliant!

πŸ’― 1
Stephen Magill [Sonatype]12:06:55

Yes, @tom.ayerst! This is why multiple tools / platform approach is important. So many axes of quality and security it’s hard for one tool to provide everything you need.

Thomas Williams12:06:26

Thanks for the great resources @stephen @rohrersm.

πŸ™ 4
πŸ‘ 1
Stephen Magill [Sonatype]12:06:07

And NIST’s effort is on GitHub and open to contributions from everyone: https://github.com/usnistgov/ACVP https://github.com/cisco/libacvp

πŸ‘ 2
Stephen Magill [Sonatype]12:06:33

call out to @jwillis, @tapabrata.pal, @samgu, @john_z_rzeszotarski, and all my co-authors on that report!

Tom Ayerst12:06:55

Wow, SO MUCH INFORMATION πŸ™‚

🀯 1
Stephen Magill [Sonatype]12:06:44

The trick is to leverage the CAB for good πŸ˜„

Thomas Williams12:06:44

Yep, working on that with some of our clients. Been difficult... Bank in rural TX.

Marcello Marrocos12:06:49

As well as the production readiness checklist, having it automated as possible instead of manual checklists.

JiΕ™Γ­ Klouda12:06:31

I love how research scientists are so good at providing references πŸ˜„

John Willis12:06:32

more importantly they are getting policy ppl writing policy as code (human readable) that drives the attestation and enforcement pipeline models

Stephen Magill [Sonatype]12:06:17

Rusty and Clarissa’s talk on DevOps and Internal Audit is also a great success story for this sort of integration of controls and evidence collection / audit for those controls.Β There’s some great discussion around that and related efforts in the Track 2 Slack this morning.

Pete Nuwayser - IBM12:06:21

Does continuous assurance "mitigate" the need for IV&V?

Pete Nuwayser - IBM12:06:21

Does continuous assurance "mitigate" the need for IV&V?

Stephen Magill [Sonatype]12:06:41

I think that’s the right question to ask first. I think there’s probably still aspects of quality / risk mitigation that IV&V is well-positioned to solve, but continuous assurance should be able to shift and narrow their focus on things that can’t be automated / handled by tools.

Pete Nuwayser - IBM12:06:47

It was there when I arrived. Trying to eliminate it from the value stream entirely, but probably won't be able to do so until the dev team has what they need to improve quality more quickly.

πŸ‘ 1
Pete Nuwayser - IBM12:06:06

A ridiculous number of low/medium defects are being carried over each week, as found by an end-to-end static analysis tool. What Stephen's talk indicated is that it's quality tools not tool singular.

πŸ‘ 1
Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW12:06:53

A trick I’ve used before is to ratchet up the automated bar slowly

1
Simon Rohrer, [Sooner Safer Happier contributor] Saxo Bank, Head of EA and WoW12:06:23

Fix as much as you can, then accept no more new medium defects as an automated bar

Stephen Magill [Sonatype]12:06:14

Yes! I love @samgu’s phrase from his talk today: β€œget clean, stay clean”.

πŸ‘ 2
Pete Nuwayser - IBM14:06:42

@rohrersm sage advice - I like it

πŸ™ 1
John Willis12:06:56

also, all the CAB has is the change record where the only reference in the change record is an immutable chain of digitally signed attestaions.

John Willis12:06:34

great presentation

βž• 3
Tom Ayerst12:06:07

@stephen Really great presentation! πŸ˜„

Tom Ayerst12:06:36

Killer ending!

Thomas Williams12:06:47

Adorable, indeed!

Deepak Ramchandani Vensi12:06:48

best way to close a talk ever! 🎀

Bryan Finster - Walmart (Speaker)12:06:49

Best. Ending. Ever @stephen

πŸ˜€ 2
βž• 2
Matthew Joyner12:06:52

Thank you for the presentation @stephen πŸ‘

Stephen Magill [Sonatype]12:06:57

Thanks! If you have stories, please reach out to me. We’re revising the automated governance paper this summer and would love to incorporate more case studies.

❀️ 1
Jose Mingorance12:06:57

She was much clearer than her dad!!! πŸ˜‚

1
πŸ˜‚ 2
Andreas MΓΌller12:06:59

Nice idea, absolute cute!

Naren Yellavula12:06:05

The climax was so cute @stephen

Thomas Williams12:06:11

Thank you, Stephen!

Bogdan Babalau-Maghiar12:06:22

@stephen your little princess is so sweet!

Brian Martin12:06:35

Awesome ending @stephen.

Stephen Magill [Sonatype]12:06:52

She’ll love all the kind words β€” thanks everyone!

Sarah Hager12:06:56

Awesome cameo appearance at the end! #nextgeneration

πŸ‘ 2
Michael Kalbermatter12:06:06

@stephen thank you! Great talk!

Duena Blomstrom, Psychological Safety Dashboard CEO, Author PeopleBeforeTech12:06:34

Careful @stephen - mine half demanded he is mentioned in the second book because he was in the first πŸ™ˆπŸ˜‚

3
Duena Blomstrom, Psychological Safety Dashboard CEO, Author PeopleBeforeTech12:06:34

Careful @stephen - mine half demanded he is mentioned in the second book because he was in the first πŸ™ˆπŸ˜‚

3
Stephen Magill [Sonatype]12:06:10

Haha β€” good point. I’ll be careful πŸ˜„

Juan Carlos Albarran12:06:44

Thanks @stephen. I really enjoyed your presentation!! Awesome ending, so cute!!

πŸ’― 2
Jeff McAffer12:06:14

<!here> good morning/afternoon/evening. Hope you enjoy the talk. Let me know if you have questions

πŸ‘ 6
Marcello Marrocos12:06:42

It is noticeable the change of Microsoft in last years regarding open source. And the acquisition of GitHub was pretty strategic.

Daniel Cahill - Engineer - Ontario Systems13:06:06

One of the problems my company has ran into is that if we have too much of our code exposed to the internet, then we will lose our competitive edge. This has resulted in only 2 repos being open sourced. How do I help continue to build more momentum to open source more of our tools?

Daniel Cahill - Engineer - Ontario Systems13:06:06

One of the problems my company has ran into is that if we have too much of our code exposed to the internet, then we will lose our competitive edge. This has resulted in only 2 repos being open sourced. How do I help continue to build more momentum to open source more of our tools?

Matt Cobby (Director of Engineering, Deloitte)13:06:08

We were discussing this today in quarterly planning. I don't think it should be a volume games, more about maintaining quality.

Jeff McAffer13:06:24

Yes, that makes a lot of sense. It always bothered me when people would come out with stats comparing open source engagement

Jonathan Garzon13:06:29

Hi @dacahill7, Adam will be able to answer that in more detail. Come to our booth https://doesvirtual.com/sonatype we can open dialogue on this topic. πŸ™‚

Jeff McAffer13:06:17

That's a key point. I'll talk about it in a second in the talk but it really comes down to understanding the "value" of your company and what you produce

πŸ‘ 2
Jeff McAffer13:06:47

and relating that to your competition. Why do people buy your products vs your competitors.

Tom Ayerst13:06:21

My answer sounded rather rude. See if @jeffmcaffer’s is the same πŸ™‚

Jeff Gallimore (CTIO - Excella)13:06:12

@jeffmcaffer i’ve talked to other enterprises with open source programs. one of the drivers was desire to attract technical talent. how much of that do you see at Microsoft and other places?

Matt Cobby (Director of Engineering, Deloitte)13:06:31

What tools did you have to build @jeffmcaffer? And how do you find the half a million new instances of OS every month?

Jeff McAffer13:06:52

@jeff.gallimore that is a common desire/direction. If that is the driver then typically I've seen that fail in the end. There needs to be a strong product/business need. engaging with open source communities is a durable activity that needs sustained attention.

Jeff McAffer13:06:52

@jeff.gallimore that is a common desire/direction. If that is the driver then typically I've seen that fail in the end. There needs to be a strong product/business need. engaging with open source communities is a durable activity that needs sustained attention.

Jeff Gallimore (CTIO - Excella)13:06:29

so maybe a driver but not the top driver. :thumbsup: . i can imagine if β€œattract talent” is at the top, the program (and the open source products) would atrophy

Jeff McAffer13:06:37

exactly. I would see it more as a consequence or outcome. Where it was a key driver, the company would attract folks but they would leave in a year because the engagement didn't run deep

πŸ‘ 1
Jeff McAffer13:06:10

@matthew.cobby we ended up writing a lot of tools to support that scale. Ultimately instrumenting the builds with tools that "detect" open source is key. Then having an automated way of relating policy to those discovered uses.

thankyou 1
Matt Cobby (Director of Engineering, Deloitte)13:06:09

Thanks, did you ever open source those playbooks? πŸ˜‰

Jeff McAffer13:06:45

@matthew.cobby No. We wanted to but there was too much company context in there. I'm sure they could be abstracted but we did not end up doing it

Jeff Gallimore (CTIO - Excella)13:06:11

that browser extension sounds cool and super helpful

πŸ‘ 1
Jeff McAffer13:06:51

yeah. that was a real "ah ha" moment

Jeff Gallimore (CTIO - Excella)13:06:55

@jeffmcaffer sounds like you’re bringing in β€œsoftware supply chain” concepts there

Jeff McAffer13:06:36

yes, it's all about the supply chain (for consumption of open source).

πŸ‘ 2
Jeff McAffer13:06:33

Even in producing open source, the more you pay attention to the needs of the supply chain (your consumers), the more adoption.

Jeff McAffer13:06:59

ClearlyDefined for example would not be needed if all the compliance info was readily available

Jeff McAffer13:06:59

ClearlyDefined for example would not be needed if all the compliance info was readily available

Jeff Gallimore (CTIO - Excella)13:06:20

@jeffmcaffer what would help with the compliance tracking? more YAML? πŸ˜‰

Jeff McAffer13:06:38

there are lots of tools out there , commercial and open for tracking. Automation is key. It's so easy for a team to bring in 1000 components with one command (think npm install )

Jeff McAffer13:06:44

Integration into the engineering system is key. This notion of Proficient is really where most people should strive for where all the compliance and security work is automated

Jeff Gallimore (CTIO - Excella)13:06:19

@jeffmcaffer indeed on both points. accessing one oss product is just the tip of the iceberg with all the dependencies that come with it. and yes, we’ve integrated as much as we can into the pipeline.

Jeff McAffer13:06:02

Thanks all for listening and the great questions

Stephen Magill [Sonatype]13:06:34

Great talk, thanks! πŸ‘

πŸ‘ 1
πŸ‘ 1
Brian Martin13:06:15

Microservices Deathstar! Brilliant.

πŸ˜ƒ 4
Brian Martin13:06:15

Microservices Deathstar! Brilliant.

πŸ˜ƒ 4
Jonathan Evason13:06:50

Wrapping DLL hell in a layer of HTTP

1
Karl Marfitt13:06:04

@kolton - there's definitely no such thing as 100% availability forever, always a case of outage/downtime when not if but unfortunately many large enterprises (and project silos within them!) are still pushing ambiguous (or even missing!) ops requirements that don't consider this properly and not even aware of what the nines really mean. I have to drop for a call but definitely going to watch this later. Thanks for raising awareness!

πŸ‘‹ 1
Karl Marfitt13:06:04

@kolton - there's definitely no such thing as 100% availability forever, always a case of outage/downtime when not if but unfortunately many large enterprises (and project silos within them!) are still pushing ambiguous (or even missing!) ops requirements that don't consider this properly and not even aware of what the nines really mean. I have to drop for a call but definitely going to watch this later. Thanks for raising awareness!

πŸ‘‹ 1
Kolton Andrus13:06:51

Ya, that is the remainder of the talk, the trade offs from different levels of investment. LMK if you have questions when you catch it later πŸ™‚

πŸ‘ 1
Rosalind13:06:34

There are systems that measure their up time in terms of years, some are at 15 years already and going. There are systems that can’t go down.

Karl Marfitt13:06:58

Even for a few milliseconds? Mother nature is likely going to force that test before DevOps does then πŸ˜‰

Karl Marfitt13:06:39

'Nine nines' allows for 31.56 milliseconds every year apparently: https://en.wikipedia.org/wiki/High_availability#Percentage_calculation πŸ™‚

Rosalind13:06:44

visa net has not been down

Rosalind13:06:20

we have 7 nines in the z hardware which helps, 7 nines in the hardware and multiple systems to handle floating the load

πŸ‘ 1
Karl Marfitt13:06:44

Wild scalability and portability (ilities!) for floating the load are useful approaches for risk assess/management approach. Many tier 1 data centres (certainly some of the smaller cloudy ones...) can't really offer five nines effectively?

Rosalind13:06:08

These are usually running in companies Data Centers - multiple - geographically dispersed

Ann Marie Fred - Red Hat13:06:18

How do you decide when to excuse people from the incident call? For example, what if your service is down but it’s something you can’t do anything about. And once the thing you depend on is fixed, your service will recover automatically. (Like a multi-site Cloud infrastructure outage.) Do you excuse that person from the call?

Ann Marie Fred - Red Hat13:06:18

How do you decide when to excuse people from the incident call? For example, what if your service is down but it’s something you can’t do anything about. And once the thing you depend on is fixed, your service will recover automatically. (Like a multi-site Cloud infrastructure outage.) Do you excuse that person from the call?

Ann Marie Fred - Red Hat13:06:37

Another example is when the authentication service is down; apps that depend on it can’t do much on their own.

Kolton Andrus13:06:01

It's a judgement call. Once you can eliminate their service from question is often the place to let them go. i.e. you're dealing with an outage and it's clear it's related to the interaction between two services, maybe we identified a change that is suspect and the group feels confident we know what's happened. That's a good spot to let other folks go. Often that is the recovery time when the 'fix' is going out or taking effect.

Ann Marie Fred - Red Hat13:06:50

OK. I think we’re bad at letting people go back to sleep. πŸ™‚

Andreas MΓΌller13:06:37

What APM tooling are you using and which works best?

Andreas MΓΌller13:06:37

What APM tooling are you using and which works best?

Kolton Andrus13:06:40

Lots of good APM tools out there. πŸ™‚

Andreas MΓΌller13:06:47

Yes, that is why I’m asking^^

Nick Eggleston03:09:04

@kolton it depends on your use-case 😊

Jesse Cafarelli13:06:25

I feel like I would love to see a new book come out that is something like the life of brent: how to realize you are him, and how to change it

πŸ‘ 1
3
Jesse Cafarelli13:06:25

I feel like I would love to see a new book come out that is something like the life of brent: how to realize you are him, and how to change it

πŸ‘ 1
3
Chris Leeworthy (he/him)13:06:34

I want to give this 1 million thumbs up!

Matt McKee13:06:38

Is the number of 9’s ok as part of a service deployment eg. 2 9’s MVP 3. 9’s as we add more users or go external?

Matt McKee13:06:38

Is the number of 9’s ok as part of a service deployment eg. 2 9’s MVP 3. 9’s as we add more users or go external?

Kolton Andrus13:06:42

Ya, I think that's part of the trade off. If you're working on a POC and it's early days, a few outages may be OK. Once it becomes a core part of the system that teams and customers rely on, it's worth the investment to improve it.

Matt McKee13:06:21

This should then also allow a more rapid route to live as you have a minimum set of controls/criteria needed - if the team knows this it should lead to less handoffs (minimum guardrails)

Rosalind13:06:03

In your experimentation with games testing incident response, do you make sure you vary the people available - to make sure their is not that critical one person?

Rosalind13:06:03

In your experimentation with games testing incident response, do you make sure you vary the people available - to make sure their is not that critical one person?

Kolton Andrus13:06:00

There's a few approaches you can take, whether you inform the teams before you run the drill, or whether you 'surprise' them. Either way, I think you want to rotate through so everyone gets an opportunity to practice.

Rosalind13:06:43

I know one team that did this with declaring who’s dead πŸ™‚. virtually of course

πŸ˜„ 4
Kolton Andrus13:06:20

There was a great talk by Dave Rensin, Director of SRE at Google, speaking about testing the knowledge that people have and chaos engineering our organizations.

Vlad Ukis13:06:10

is there a recording of that talk by Dave Rensin?

Davy Kenis13:06:46

what’s the precise difference between a game plan and a fire drill?

Davy Kenis13:06:46

what’s the precise difference between a game plan and a fire drill?

Kolton Andrus13:06:25

A Gameday focused on testing the services and technologies, where a Fire drill focuses on the team and their response.

Kolton Andrus13:06:53

The Gameday is often planned and widely communicated, where the Fire drill might be run as a surprise, or with limited knowledge beforehand.

Ann Marie Fred - Red Hat13:06:54

I love this presentation so much.

❀️ 2
😊 1
Karl Marfitt13:06:08

'availability theatre' - reminds me of someone who once argued with me that 'we never test that failure scenario as it's never happened before' evidence of past failure (or lack of it!) in complex systems does not always closely correlate with future reliability/failures, similar concern with testing complex systems, as there are simply too many combinations of possible outcomes that we ask how many nines test coverage is acceptable. Which failure test scenarios are more likely? Arguably of all possible global disasters, was a pandemic in the top ten most likely to happen, regardless of how we might have prepped and tested readiness to recover? Absolutely loving this talk, thanks for sharing @kolton

Karl Marfitt13:06:08

'availability theatre' - reminds me of someone who once argued with me that 'we never test that failure scenario as it's never happened before' evidence of past failure (or lack of it!) in complex systems does not always closely correlate with future reliability/failures, similar concern with testing complex systems, as there are simply too many combinations of possible outcomes that we ask how many nines test coverage is acceptable. Which failure test scenarios are more likely? Arguably of all possible global disasters, was a pandemic in the top ten most likely to happen, regardless of how we might have prepped and tested readiness to recover? Absolutely loving this talk, thanks for sharing @kolton

Kolton Andrus13:06:27

Thank you very much. Yeah, it's a prioritization effort of both what you think is likely (based on past outages or your own analysis), and things that might be rare but very impactful when they occur (black swan events)

Brian Martin13:06:24

Great talk. Thanks @kolton

Kolton Andrus13:06:33

Thank you all for the opportunity to present! Happy to answer any other questions now or later, DMs are also open or via email (kolton @ gremlin). πŸ™‚

Rosalind13:06:38

Great session, thanks for sharing.

Jonathan Evason13:06:47

Great talk. thanks!

Pat Eyler13:06:09

Thanks @kolton - Great session!

Narendra13:06:27

@kolton Nice presentation! πŸ‘

πŸ˜€ 1
πŸ‘ 1