👋 Hi everyone, my topic and is Supply Chain Security in open source. Happy to chat about it or other appsec topics and questions with you

⭐ Welcome, welcome @dave.karow for our next session's Q&A. Thank you https://doesvirtual.com/split!

Thanks @mollyc - Got my tea poured (in honor of London and so to avoid running the coffee grinder at 4:30am for the fam) 🙂

I wasn't able to find the slides in https://github.com/devopsenterprise/2021-virtual-europe

can you help me find it?

"learning during the process" - part often skipped!

Yep!

Not just about slowly rolling out but about learning by starting with partial exposure.

if you don’t automate the learning, it’s more hyper-vigilance and toil 😞

…which means you’ll do it less often. That’s bad.

what does yellow mean here?

How does one automate learning?

Checking the box… Does it Avoid Downtime, etc…

too busy to improve is a common challenge I hear - limiting WIP so counter-intuitive

@benk691 Yellow means getting the “Benefits” in the left column

@rgorham By ingesting telemetry that’s attributed to the gradual exposure cohort and the not exposed cohort. Perform automated stats comparison.

super interesting - don't wait for it to happen, can measure on the fly

curious to get everyones thoughts on "starting wiht the end in mind" - getting into the backlogs early on, involving Product Ownership/Product Management or similar to start planning these improvements/capacity for it in?

The danger of showing code is that it can be wrong 😉. Looks like you used 'treatment' and then 'feature'.

WIP and Flow are inextricably linked.

Love the matrix a few slides ago, btw

Deck is here if you can’t find it on conference site. https://speakerdeck.com/davekarow/does-layered-approach-to-pd-2021

How feature flags would work for mobile apps where you have to release a new app release? #ask-the-speaker-track-4

@manzi.g Can't you tie flags into your deploy with android and apple?

We're using the Flow Framework to help visualize how we're tracking to the business outcomes as they make improvements to see if their "experiments" are impacting the flow metrics/outcomes. Seeing that visualized can be eye opening 👀

@manzi.g You can put flags in your mobile apps… it’s how many teams protect recent changes/run cohorts.

"without actually asking people to do more work"

(not a fan of toil or hypervigilance)

"how to separate signal from noise" -!!!

oh my god! a release! surprise haha

I think I need a stats engine for meetings 😂

Feature flags are a very powerful architectural construct! Highly recommend. We even have flags on our mainframe code that help us deploy changes with minimal impact.

…to determine if they have any impact?

@scott.prugh I’ll bet you don’t miss multi-hour hotfix/rollback exercises.

…which are super fun on a mainframe 🙂

direct evidence of effort - love this slide

Reminder: slides are here if you’d like: https://speakerdeck.com/davekarow/does-layered-approach-to-pd-2021

What does "Painted Door" mean? Sorry, but I didn't understand this point.

The appearance of a feature but no code behind it.

(or at least not the biz logic yet)

Just a test for interest

love the point on sustainable flow

I have a hypothesis that feature flags are a key architectural construct that increases a developers understanding of operability...

So the opposite of dark launch, like the Lean Startup thing of buying a Google ad without a real thing behind it?

@rshoup yep

@rshoup Look into “Pretotyping” by Alberto Savoia https://www.pretotyping.org/

Well presented talk, thanks!

really great presentation thank you!!!

definitely sharing this with my colleagues!

Glad you liked it folks. Deck will be on DOES site shortly but is here as well: https://speakerdeck.com/davekarow/does-layered-approach-to-pd-2021

Great presentaion @dave.karow!! I love the four shades slide!!

that is an amazing vision and possibilities. I'm working with our team to reduce branches and start to use feature flags... we have a long way to go

@dave.karow inspiring talk, thank you! Now the tricky part is, as you said, separating singal from noise. For me that includes what to measure (because, in any app, there is a ton of things I could measure). Any tips here? Does Split help with that?

The Limit WIP/Achieve Flow is so powerful.... Queues and Dependencies are a massive problem: Team A waits for Team B and queues up work and they both need to arrive together. With Feature Flags you can remove a dependency and a queue!

Welcome @liran.tal for our next session's Q&A. Thank you https://doesvirtual.com/snyk!

@jakub.holy Great question. Start with the “guardrail” metrics your company cares most about. We are taking specific tech or biz metrics not massive details like tailing a log.

Looking forward to the presentation @liran.tal 👋:skin-tone-2:

@jakub.holy hold a sec and I’ll share a link

Any good resources for finding the right (biz) metrics to monitor?

https://www.split.io/blog/how-to-choose-the-right-metrics-for-your-experiments/

https://help.split.io/hc/en-us/articles/360031135192-Experimentation-Essentials-101-Metrics

https://www.split.io/blog/how-to-avoid-lying-to-yourself-with-statistics/

Loving the hat!

Progressive Delivery folks: I’ll be in the Split Channel now at #xpo-split-feature-flags

@liran.tal are we going to use less, or more open-source software?

are your slides available on github?

Great blog post about this topic https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610

thx liran!

Thanks @liran.tal!

thant was great, thanx

👏:skin-tone-2:

I love the passion @liran.tal ! 🚀

👏

Thanks folks!

How effective do you believe automated tooling will be in helping us detect malicious code/edits to packages in our tool chains, @liran.tal?

Thanks for attending. Love to engage in more discussions on this so don't be shy to shoot anything in the channel here or elsewhere. A lot of smart people here for us to debate some tough questions :)

👏:skin-tone-2:Welcome @paul120 who will be moderating for today's VendorDome Q&A between @nlevine and @swhite941 Thank you https://doesvirtual.com/anchore & https://doesvirtual.com/gitlab!!

let's get it started!

Let's do a little poll to see what the audience is doing with DevSecOps!

One more question about where you do security checks^

You can select multiple

Do you think that pen testing skills have a valuable place in DevSecOps to flip the security viewpoint on its head and start “thinking like an attacker” and are those skills that more Devs + Ops should start looking at gaining?

Interesting that in the second poll above most people are doing security scans in CI/CD and not at other points.

Hi, @paul120 @nlevine @swhite941 — I’d love to hear how the potential leak of cloud credentials due to codecov issue has affected how orgs do CI/CD and cloud credentials. I was blown away by the scope of the problem that this revealed!

Yeah -- the list of people impacted by CodeCov is mind boggling because it's software vendors -- so you get a fan out from codecov to a bunch of other SW vendors which then impacts many more companies.

It was also the first one I've seen that was around containerized software

I agree — truly breathtaking!

CodeCov gave me 2 days of just running over every piece of Terraform in our central codebases 👀 Not fun

It took me days to start getting my head around the implications of this were…

I opened a lot of these, I still love the required version string for modules supporting older versions 😂

So the new Executive Order in the US is going to be interesting here: they are going to start mandating an SBOM from SW companies that sell to them, I think that will impact all SW companies and the industry at large

Reminds me of something Robert C. Martin said (paraphrasing heavily): "Software will be regulated, it is on us to decide our own ethical codex and how we deal with these things, else someone from politics will"

And you don't want to be regulated like that 😄

From the Executive Order:

(vi)    maintaining accurate and up-to-date data, provenance (i.e., origin) of software code or components, and controls on internal and third-party software components, tools, and services present in software development processes, and performing audits and enforcement of these controls on a recurring basis;
          (vii)   providing a purchaser a Software Bill of Materials (SBOM) for each product directly or by publishing it on a public website;
          (viii)  participating in a vulnerability disclosure program that includes a reporting and disclosure process;
          (ix)    attesting to conformity with secure software development practices; and
          (x)     ensuring and attesting, to the extent practicable, to the integrity and provenance of open source software used within any portion of a product.
     (f)  Within 60 days of the date of this order, the Secretary of Commerce, in coordination with the Assistant Secretary for Communications and Information and the Administrator of the National Telecommunications and Information Administration, shall publish minimum elements for an SBOM.

https://anchore.com/blog/latest-cybersecurity-executive-order-requires-an-sbom/

Anyone have any live questions for the speakers?

^^

In working with US Gov and Platform 1, they are definitely worried re insider threats!

This was an interesting survey from Linux Foundation: https://www.techrepublic.com/article/open-source-developers-say-securing-their-code-is-a-soul-withering-waste-of-time/

"soul-withering waste of time" is a bit concerning 😱

"The survey, which included questions designed to help researchers understand how contributors allocated their time to FOSS, revealed that respondents spent an average of just 2.27% of their total contribution time https://www.techrepublic.com/resource-library/whitepapers/incident-response-policy/ Moreover, responses indicated that many respondents had little interest in increasing time and effort on security. One respondent commented that they "find the enterprise of security a soul-withering chore and a subject best left for the lawyers and process freaks," while another said: "I find security an insufferably boring procedural hindrance."

That's always been the issue. People want to do the fun things like adding new features. Few want to muddle through security and documentation.

Yup

Part of the issue is false-positives. There are so many vulnerabilities that aren't relevant, that developers end up wasting time explaining why things don't need to be fixed. If you can reduce false positives it becomes easier to make the case to developers.

Here's a quote from a recent interview we did: "People don't hate running vulnerability scans. What they hate is fixing them because they don't actually identify problems"

"So, if my noise is say less than 15-20%, I'm good, but right now noise levels are like 60%. And that is very high. It's a big problem because I spend a lot of time trying to satisfy those things."

"Yes, it’s a huge waste of time (chasing after false positives)"

The pain is very real

We’ve also recently published our DevSecOps Survey at GitLab https://about.gitlab.com/developer-survey/ which will be of interest to most of you and see how 4,300 of your peers see the current state of affairs in DevSecOps

Hm, do you mean scanning the actual production or rather create like a summary of what is deployed in production so that you can always check that specific version of code or application at any time?

When scanning in production, you may have new vulnerabilities associated with running code. eg there wasn't a vulnerability when it went into prod, but there is now

I think it would be cool to have an index/tool/benchmark for scanning that measures false positives for container images since that's the wave of the future

Hello Everybody, this is Sacha Labourey from CloudBees and I'm with @sanmat.jhanjhari475 and @ben.angell from Nationwide Building Society, happy to be here with y'all!

‼️Welcome @sacha, @sanmat.jhanjhari475 and @ben.angell for our next session's Q&A. Thank you https://doesvirtual.com/cloudbees!

To continue discussion with @paul120 @swhite941, visit #xpo-anchore-devsecops or #xpo-gitlab-all-in-one-devsecops - thanks for tuning in to Anchore / Gitlab Vendordome!

Curious, what do others feel is the biggest challenge in convincing your business to support your software initiatives?

baselining so key. if you don't know where you're at or how fast you're going, how do you know if you're improving? @ben.angell do you use any specific metrics to track progress from business /customer perspective to complement DORA etc?

@ben.angell Loving some of the metrics to measure efficiency. How quickly you can pass code though SDLC and how automation can take out time. Infrastructure as a code being a priority and delivering the right way.

📣Welcome @gustav.lundsgard1 and @mdahl for our next session's Q&A. Thank you https://doesvirtual.com/synopsys!

We love appsec process optimizing - equally as much as Star Wars!

Inner creative Yoda you find...

How do you "find champions"?

Listen now, Craig

lol

Great timing on your question!

So at what level will Cyber Jedis receive their light sabers? Master?

That clarity of vision is crystal and inspirational - for all customers including internal customers

Fully agree Ffion!

Lack of starting metrics is a challenge. I am starting to build a PagerDuty community at IBM. We do have a form of starting metrics though. Will be interesting to see if any of them improve.

Again - good timing here - Gustav reflecting on the improvement process of the project - but indeed: Metrics just visualize the process underway.

By saying just = meaning indeed

Thank you for the introduction to the Cyber Jedi Academy @mdahl and @gustav.lundsgard1. Loved it.

Great talk thank you, love the community of learning!

You are most welcome - thanks for your attention and time - much appreciated! Please reach out in the "real world" for further dialogue!

Thanks for listening 🙏