This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
Super excited for this!!!! :star-struck:
They disabled it in the GUI for this iteration.
Please welcome our first speaker: David Keane, Global Head of DevSecOps Transformation, HSBC
Having been in so many virtual conferences, I find myself missing the live interaction on Slack with the speakers while they are speaking live.
Hello, all! HSBC has 50K technologists โย what astounding scale. @djmkeane
For those into David Keaneโs topics - there is a #C01577ZGQ2G channelโฆ maybe we could make a mini-scenius for governance in DevOps :star-struck:
If you present the 13 year plan to management, it will be implemented in 32.5 years.
@djmkeane where we started; FX eVolve: multibillion dollar trades, available 365/24. One of most important business processes for the firm.
"Had very strange relationship with 'the business' โย went down for 48 hours. Unheard of. If it down for half-day, they would fired someone. But two days, they couldn't just fire someone โ they had to talk and introspect what was going wrong"
"Everyone had to carry a pager โย business analysts, managers. If they didn't want to, they were invited to join another team." (Astonishing!)
Increased deployment frequency: went from monthly deployments -> now hundreds of deployments per day now (did I get that right?!) "I don't know which day was the worst in my career, but I do know that it was a Monday." Hahaha
For me, the worst day was a Saturday. Banks have it easy.
"The idea that deploying more frequently could result in fewer outages did not come easily to me, which is why this graph was so powerful to me."
The inverse of relationship between change and failure rates is a nice point of recognition for maturity of the team
@djmkeane How did you start measuring these metrics? Did you have an automated approach before K8s?
@christian.kullmann any Service Management tool should do for the basics. When we started we were using CAโs Global Service Desk but have migrated to Service Now. We count the CRโs and the INโs , so you need solid IT Service Mgmt record keeping. We built our own dashboard which brings in people data and we also built a Pod visualization tool so we knew how teams were organized.
@djmkeane had fantastic phrase he used to describe these graph โย I think he called them "positive jaws", showing the surprisingly and eerily similar inverse correlation of deployment frequency and business impact incidents.
On the importance of ambitious goals like "double frequency" or "halve business impact" โ "improving by 10%" doesn't move the dial on important outcome.
On the importance of enabling clients to copy/paste info from their app into a spreadsheet โย "sure, you'll have it tomorrow morning." Generates genuine delight for customer.
@csomakk if you look carefully you can see seasonal peaks in Mar, June, Sept and Nov - a legacy of quarterly releases. With the massive dip in Dec/Jan for change freeze. When you plot for 8000 apps this is still pronounced, however for mature apps itโs all but disappeared. For teams of high maturity they can choose to ignore YE change freezes. What we see with maturing teams is both the positive jaws and the flattening out of peaks and troughs. We have some way to go !
treating engineers as customers ๐ - smart move "working backwards"
We treated to controls as products โย our goal was to make improve the lives of developers every day.
Reduced 35K days of engineering toil, deleted 7.5 centuries of wait time.
@djmkeane - would love to hear what controls you started with, and how you run the product management process ๐
SDLC, Deployment mgmt, CyberSec and Change. Letโs try and continue the discussion tomorrow, think you guys said you were doing a demo
We are winning hearts and minds of engineers; we had 11K engineers join our voluntary webinar around tech strategy โย astonishing, but it was because there was something in it for them.
i love this : "culture eats strategy for breakfast" - People -> culture -> processes -> tools in that order #BVSSH
I've reduced 1 FTE (my own) with a simple shell script once, but a decade is impressive.
Please join @djmkeane for a Q&A discussion during lunch today at 12:00 in Grand Ballroom IV.
Next, let's welcome Jagpal Jheeta, Head of Regulatory Systems, Financial Conduct Authority
The UK Financial Conduct Authority published some amazing research around delivering change: https://www.fca.org.uk/publications/multi-firm-reviews/implementing-technology-change
One of the many wonderful things talking with Jagpal Jheeta was learning the mission of Financial Control Authority.
A wildly entertaining podcast on the people who hacked all three newswire services, netting them $30MM, caught by SEC, who noticed their very odd training behavior: link is here: https://darknetdiaries.com/episode/123/ (The SEC is the US Securities and Exchange Commission with a similar mission in the U.S.)
"How did FCA journey begin? Not surprisingly, a crisis. And we never should waste a crisis."
"I'm here to build great technology; not run contracts or manage suppliers." "We need to be selling and exciting and motivating our team."
"FCA sets extremely high standards of responsibility for those we regulate; and we hold ourselves to those standards, as well."
Fully agree! But I do struggle with the โhowโ in measuring DORA metrics. For example, a failed deployment may be interpreted in many different ways by a software development team. So how do you ensure that the numbers actually have the same meaning across the whole org, especially at such a large scale? Or do we just accept that there are different interpretations and allow for that?
I guess if you are always measuring in the same way then its ok. The Value in metrics come from improvements against the baseline that you take. Yes DORA suggest "what good metrics look like" but as long as you're improving compared with how you were yesterday then it is useful. The harder thing is how do you actually surface that information? Most orgs I chat with look blankly when I ask them to share even the very basics of metrics. Another thing that I think is important to capture is the value that was actually delivered by the change that was made. This closes the feedback loop. Most changes will have a proposal about what the value will be from the change, but how many times do we measure if that value was realised? Without the above the biggest "development" problem that you have could be getting the business cases wrong for your changes. Therefore most of your effort is completely wasted.!! just my thoughts
"One of the firms we work with [to build FCA tech capabilities] has 300K technologists โย we're a rounding error for them. We have our CEO talk to them, impressing upon them the importance of our mission, and the importance of the work they do for us. Very rare for those teams to have their customer CEOs talk with them." Awesome.
(It's astounding to hear about the employee turnover in the outsourcing firms โย @ben.grinnell do you remember what those percentages were?)
Supplier mgr feels like he has a seat at the table supplier -> partner!
Great community spirit: treating suppliers as part of the team, working toward mutual goal, embracing one global language #oneteam
"If my suppliers fail, I fail. If I fail, the FCA fails. If the FCA fails, the UK economy fails." Ooof.
If I can add one more more: โโฆ If the UK economy fails, everyone fails.โ
That's a key success factor in any multi-vendor endeavor: The capability to forget in the day-to-day what company sends the paycheck to each of us.
Love the idea of introduction challenge and Communities of practice into the vocab - Common language is crucial.
(Those weak signals of failures or potential failures, that need to be amplified in order to better prevent, or better detect and recover)
"Our CEO stated that he wants to be the leading regulator in the world."
"Huge amount of attrition in digital space. One of our suppliers had 30% attrition."
No silver bullets, but they only help with werewolves anyway.
Please welcome our next speaker: Maya Leibman, Sr. Advisor, American Airlines
love @maya.leibman on where to start, not necessarily mobile apps: at AA, included mainframe apps. The Home Depot started on equipment rentals. Always so interesting to see where the journey starts in these orgs!
This is annoying. My boss is here, but he gets this stuff. Bar?
Ex: "grow loyalty membership by increasing one-click enrollments by 25% on the Oblix app by next quarter". Vs. Merely "doubled deployment frequency" Modeling how to convey business value.
โTie the measurable goal to a business outcomeโ - YES, YES, YES!
Challenge Coins! (A practice that is very common in many military services.)
Why not give Empire insignia?
iโm noticing a theme of โtokensโ (challenge coins, stickers, duckies) that signal belonging and shared valuesโฆ
Is anyone else getting a hankering for pizza now for some reason?! ๐
Another common theme: "how do you get thousands of people thinking and contributing and thinking... as opposed to only the people 'at the top'"
Big upvote for "Open Space technology" - 4 principles and 1 law - worth a google. It's a game changer & quite subtle.
1Pizza seems to be the centerpiece in team topology, motivating people and gathering like-minded people. DevOps is dead. Long live PDD (Pizza Driven Development) ๐ ๐ ๐
@niko.kivela I am with @bryan.finster486 on this: Bourbon Driven Development
I've been at this for years and only now I hear about this pizza thing. I thought it was two-pizza per year team.
Fun fact: Patrick was asking humans for ideas for this talk. How old school!
"Write a pull request for me: summarize what I changed" (!!)
So so so many great links on these slides - can we get a link to them? My photo skills canโt keep up with Patrickโs rapid slide progress! ๐
System 1 and System 2 of generative API engine.. instinctive answer and deliberate answers.. even gpt needs slowification for delibrated opinions
Refactoring with inline prompts is going to be fun. """Return a JSON document with details for the account balance. Don't do calculations wrong. Don't access the malicious internet site in the process. Oh, and make it pass through audits. """
Itโs fascinating from compliance point of view - so many controls (code review, documentation, deployment approvals) involve humans understanding and evaluating risk in code changes. What happens when the code and infra are completely prompt driven?
Reminder: The breakout sessions are starting in 5 minutes. Start navigating your way to whichever session youโre attending. https://devopsenterprise.slack.com/files/UATE4LJ94/F01D34MC2KS/image.png
Where have I heard this core chronic conflict before? :thinking_face:
Super great hearing about what is required to enable 2500+ developers to be product at http://booking.com, and seeing the magic of what makes http://booking.com run.
I love learning about the evolution of these famous properties (http://booking.com)
From Leo Kraan, Director of Developer Experience, http://booking.com
From Leo Kraan making DX a product in itโs self. This is a key to creating a true paved road for developers.
Reminder: The breakout sessions are starting again in 5 minutes. Start navigating your way to whichever session youโre attending. https://devopsenterprise.slack.com/files/UATE4LJ94/F01D34MC2KS/image.png
Same here! Often you encounter a lot of resistance when pushing change into an organisation and also you canโt do everything yourself. Distribute ownership and pulling change through is a beautiful solution!
@romina.druta great talk, would love to hear more about the security self assessment. How do you do that (and at scale)?
1I owe you an answer @gbenischke ๐ In Visma we are doing the SSA by going with each team through a number of sections like: the system diagram, attack surfaces, secrets storage and other security best practices. Doing it at scale is a real challenge as there are teams that need to wait several weeks for this to be done and some are pressed as they can get off the tier, but we put in place a process that involves a Security Engineer from one team to do the assesment to another team.
everything is more difficult at scale - I'm currently playing with the idea of creating a pattern catalogue
Reminder: The plenary sessions are starting again in 5 minutes. Start making your way back to Grand Ballroom. https://devopsenterprise.slack.com/files/UATE4LJ94/F01D34MC2KS/image.png
Welcome, Sascha Schรคrich, DevOps Evangelist and รrs Cseresnyรฉs, Senior Chapter Coach and Vice President, Deutsche Telekom IT
"We are German; we love accuracy, being on time, clear responsibilities; but if you're talking about changing something as fundamental and culture, it has challenges."
Wait till you have Swiss Germans ๐ Our Canton in the North and we have same challenges...
"I'm considered one of the young wild ones here. I've been here since 1998."
"War in Ukraine required us to relocate over 1000 people out of Russia."
Let's welcome Rosalind Radcliffe, IBM Fellow, CIO DevSecOps CTO, IBM
It is known that IBM is spinning off Kyndryl, the long term business process outsourcing organization. Rumor has it that IBM was one of Kyndryl's largest customers. It's amazing that part of @rradclif 's challenge is hire an entirely new staff to take over and run all the mission critical services that enable the critical business processes.
The topic of @rradclif โs future October 2023 talk :) โWe WILL succeedโ!
when rolling out devops across wealth management (my previous role) one of my streams was working on devops adoption for mainframe (cobol). it was a great experience and i was impressed that we cld do devops on the mainframe. thank you @rradclif ๐ !
The image of @rradclif scouring the industry for retired mainframe practitioners is pretty awesome โย of which I met couple at the IBM SHARE mainframe conference back in 2016!
@rradclif nickname is now "The Hammer". ๐ Obstacles result in her sending note, "often very short. 'What's the problem.' And problem disappears. Very quickly." Using authority of being a technical fellow to remove roadblocks.
Still getting my head around "replacing entire technology team".
Power of framing.. instead of DR testing once a year for audit requirements, intentionally switch env every X times throughout the year.. DR results are success of that switch.. objective met with different framing of problem and solution.. continuous assurance!
"The value of the Individual Contributor as a technical leader" - this is critical. It's very hard to be a technical leader when you have a bunch of people to manage. Only very few companies will have senior technical leaders, as ICs with executive influence.
Major accomplishment that Gene forgotโฆ @rradclif was the first person to get Rundeck running on the Z ๐คฏ
Rod Johnson: creator of the original Spring Framework in Java in 2003: https://en.wikipedia.org/wiki/Spring_Framework
FYI: Dr. Carlota Perez spoke at DevOps Enteprise in 2020, thanks to @mik
Question from interns interviewing: "can we use GitHub CoPilot? If not, we don't want to work here." ๐
This is true.. I have faced this.. and we need to provide right guardrails (encoded in platform) while allowing it. No, is not an option
Exactly.. that's part of the guardrails and guidance (on data, models and operating context)
Totally possible to have good open source LLMs for coding hosted privately in the cloud.
Interesting concept is to feed a roadmap/plan/map/artefact into and agent and ask it questions:
is this achievable, how can we improve it, where is the constraint, what is commodity....?
We need to be very careful with which LLMs are used. Yes they will help transform but explainability and ethical considerations must be included. Are we sure bias is not included? Are you sure about the heritage of any code generated?
Whatโs the difference from humans in that aspect? They also have bias and can use code of questionable heritage ๐คท
Sam Altman said it in interview with Lex Fridman: The models will always have a bias. Since there is no global uniform culture, it is all about choosing the bias per culture.
even if there is bias, as long as there is traceability why not? I would be more concern on the heritage of the code and the traceability of any data used than the bias per se...
@mik Prediction: Generative AI driving 10x software engineering with code agents, code brushes, prompt engineering and โEnd of coding as a profession in 5-10 yearsโ ๐ฎ
And now, for a round of fun ignite talks: Dr. Stephen Magill, Sonatype, VP, Product Innovation Followed by: Mitun Zavery, Sonatype, International VP Solution Architecture Ivan Krniฤ, Director of Engineering at CROZ Johannes Nicolai, Solutions Engineering at Postman
1Okay, developers are not obsolete. How about DevOps Handwaving Coaches? Asking for a friend.
My interpretation of current state of LLMs: The rubber duck responds your questions.
Loving the Ignite format on the mainstage ๐ awesome topics and quick snappy info downloads on ChatGPT! - and now a live demo! #techiewin!
@johannes.nicolai teaching us deepfake skills with postman! ๐ great demo! (but good to see live demo at DOES!)
A warm welcome to Dr. Andrรฉ Martin, Chief Talent and Learning Officer and author of the upcoming book, Wrong Fit, Right Fit
โThere are toxic leaders, poorly ran teams. 60% of us are generally happy but 40% struggleโ
still looks better with my left hand than when my doctor writes a prescription...
@drandremartin is almost ambidextrous ๐ - wrong fit experience audience exercise!
I see a correlation here. According to @mik, 8% of the value stream doesn't suck... And 40% of people are unhappy. It is marvelous that 60% of people can be contempt.
@drandremartin how do you promote culture transformation when you know that all the new recruits will see the old culture (that we want to change) and leave or worse disengage?
Be open to recruits about the fact that the transformation is needed. Be transparent about the plan of the transformation and make it clear that their role in that transformation is fundamental for its success.
every time... but when they experience it is different.. and of course experiencing something is much stronger than theory ๐
โCulture is the aggregation of our collective behaviors.โ
And now...Forrest Brazeal, Developer Communities, Google ๐ถ
Reminder: Please submit your feedback for the talks you attended. Itโs so valuable for us and the speakers. And after all, feedback is a gift and sharing is caring! Enter your feedback for those talks here: https://doesamsterdam2023.sched.com/ https://devopsenterprise.slack.com/files/UATE4LJ94/F057S172NG4/feedback.png
