The problem is further exacerbated by a class of people who received their MBAs and think they know it all. Just yesterday, a lead product manager was arguing with security folks about why his service needs to be patched for log4j vuln if its not internet facing. He had trouble fathoming that even though his service is not internet facing, it processes and logs user controlled data.

Look at recent Azure vulns, I am pretty sure their internal security team knew about these and after some back and forth some exec might have signed off an exception. They would rather be shipping features than fixing the mess they created. Most infosec peeps have trouble getting teams to prioritize of security stuff and some of the blame falls of infosec teams too for making everything sounds like a end of world scenario. But did Azure lose a single customer or did the stock price go down or loss of revenue? Nope, so whats the point of investing so much in security if it truly the only harm was some loss of reputation.

Even most security execs I have had a chance to interact with dont understand security topics properly, surely they can use some jargon to throw around in all-hands meetings and such. Unless from a security background these execs often confuse security with compliance and instead of investing in defense in depth techniques they look for check-boxes against security controls.

Paradoxically, when someone has a pure (or at least focused) cybersec program (a few 3-4 year programs are taught by reputable institutions near me), and a Sec+ or equivalent, all of the old guard shout about needing years of experience (decades preferably) before you should be allowed to even think about security.

It only takes a few days in r/cybersecurity or r/securitycareeradvice to see these people in action, yelling at kids coming out of a 4-year university course focused on cybersec to "put in their dues" and work a call-center/help-desk for a few years resetting people's passwords before being allowed the honor of applying to an "entry-level" security position.

If a 4 year program cannot prepare you for an entry-level position, either the program is broken or the hiring expectations are broken.

Just in this thread someone was saying they would require 10 years of system administration AND 5 years of security experience before considering to hire them. In the same amount of time you can become a doctor or lawyer, and be operating on people or have established your own law firm.

- Compliance auditing (PCI, ISO, WebTrust, etc.).

- Software auditing.

- Delivering basic consumer-level security awareness training.

- Tier 1/2 SOC and NOC duties.

- Member of an incidence response team.

- Member of a penetration testing team.

- Policy development, deployment and management.

- Jr. Researcher for XYZ (PKI, cryptography, authentication systems, malware, etc.)

Part of the problem are the for-profit schools and bootcamps cranking out 'cyber security' graduates. They know the least out of all the people I interview. How can you pretend to know anything about cybersecurity when you don't actual know anything about programming or networking?

The classes cover buzzwords like vishing/phishing/smishing, you run Kali Linux and 'hack' something, and then you get your certificate.

I got a lot of good mileage out of explaining the Equifax Struts vulnerability, which allowed attackers to move freely through Equifax internally once outer security was breached because internal security controls, especially around patching, were so weak. Might be worth trying if you encounter the same situation again.

So much this. I had a security review failed because an API would respond with http 422 on invalid input. When I asked why that is a security issue I got shut down with “defense in depth”. After a longer discussion the problem was that 422 was not part of “the original http spec” but rather some ldap extension.

Here's the issue: cyber security is seen as a cost center. As long as it's viewed a cost center good CS programs won't care for it. Which means right now it's relegated to certificates and extension schools... we all know what that means.

If companies/governments start caring for cybersecurity, ie, create a prestigious and visible organization that directly reports to the White House for instance, then you'll see the good CS degrees adding more of it to their curriculum.

> The problem is further exacerbated by a class of people who received their MBAs and think they know it all. Just yesterday, a lead product manager was arguing with security folks about why his service needs to be patched for log4j vuln if its not internet facing. He had trouble fathoming that even though his service is not internet facing, it processes and logs user controlled data.

I remember being in a room like that. At one point several people were arguing and the lead engineer just tapped his brass rat on the table to get everyone's attention. I remember the PM was furious but what was he going to do? They don't sell those at the gift shop...

Truth is, PM orgs need to exist in a parallel way to engineering orgs. PMs managing engineers is a red flag, and a true tech company should ideally have engineers all the way to the CEO position. So if there's security work and engineering deems it necessary, it's done no matter what some non-technical employee thinks.

Engineers could honestly take a page from MDs here. Opinions of non-MDs are basically regarded as irrelevant...

I once had to argue back and forth with someone (circa 2008) that JavaScript did not mean "mobile code" in the sense of their checklist. I had to explain what JavaScript was, how it worked, but they were more than willing to tell me I had to remove it from the app I was working on. Which would have rendered my app and all the other apps for that client much less functional.

What the hell does that even mean?

Of course the mainstream stuffs are tolerated but anything outside of that would need a long list of approvals.

Why do we have so many security disasters? Because those people are rare unicorns, ridiculously expensive, with no way to show added value.

I don't agree that it takes 15 years though. I think you're setting the standards way too high for no good reason, especially for "decent".

By the time you're done creating the perfect "Information Security Certification" test everything will have changed. Even the most nebulous of security certifications (CISSP; which has a super generic test that doesn't cover much in the way of "practical security") still requires 5 years of experience before you can even take it.

It's just as bad as the JavaScript ecosystem. Maybe even worse, actually.

Information security is an ultra fast moving target. The only way for companies to effectively manage it is to hire people who constantly fuck around (with technology) and are always learning (the limits of) new things. It's incredibly hard to hire (and retain) people like that.

The "safest bet" for someone who really wants to be a great InfoSec professional is to get really good at Linux systems administration then start learning how to break into things. Because once you've broken into a system you need to know how to create/execute payloads. Otherwise you're going to get stuck on the first step every single time: Finding the vulnerability. You need to be able to exploit one host and then use that one to break into another system (pivot).

Learning Windows systems administration isn't as useful IMHO because there's fewer systems and they're all the same for the most part (monoculture). You can pick up everything you need to know about exploiting Windows in a short time and then exploit it limitlessly (haha) later. Whereas Linux sysadmin skills are applicable to a very wide array of systems from embedded stuff all the way to supercomputers.

Also, if you're going to get into hardware hacking or making physical devices that help you test the security of things Windows skills are basically useless. Nobody actually loads Windows 10/11 on to something like a Raspberry Pi in order to place a physical back door somewhere (or interface with SCADA systems, air conditioners, etc).

>The only way for companies to effectively manage it is to hire people who constantly fuck around (with technology) and are always learning (the limits of) new things

Then make the curriculum about fucking around with technology, taught by people who fuck around with technology for a living. Then you get a nice certificate that says you fucked around with technology for a bit and showing that you're capable of fucking around with more technology.

You're right about all the previous certs, but the solution is simple: make the curriculum match how people actually learn in the industry.

>The "safest bet" for someone who really wants to be a great InfoSec professional is to get really good at Linux systems administration

Don't get really good: get pretty good then go learn programming. You talk about jumping between embedded and supercomputers later, but programming/AppSec is more important and way more useful (esp. if you don't already know how to program)

>Learning Windows systems administration isn't as useful IMHO because there's fewer systems

Oh no mate, AD is everywhere and those skills are immensely useful to a large amount of companies. Offensive Security even changed their OSCP exam to have an AD target set.

The imaginary skilled professional you are describing clearly originates in the mind of an engineering worker.. a person gains skill through experience and is promoted. This is opposite of what management builds over time.. Management specifically and exactly destroys this career path because it costs them more money. As long as you can commoditize and outsource, you drive costs down, not up.

Meanwhile, it is "eternal September" in the job world, with streams of 20-somethings lining up to get into the markets. Add lower cost engineers, for example in Eastern Europe, South East Asia and South Asia. Rinse and repeat.

I'm a 15-year infosec vet. I'm not nearly as technical as some of the HN crowd would like for infosec guys to be, in large part because high technical is not something employers generally want and are willing to pay for. If you want to maximize pay, the best path is to learn just enough to be regarded as competent, then move into management, sales, or PM work. There's barely room for the highly-skilled, highly-technical cyber guy in most large companies, let alone SMBs. Most companies chop this ideal infosec role into multiple parts too minimize cost and risk, just as you describe.

Then it outsources to Eastern Europe.

When I was working in infosec consulting, by far the best colleagues were those who had software engineering experience and could empathise with developers at the client in order to understand how systems would be built, where corners might be cut, which areas might be more ropey than others etc (and then use that understanding to help inform their thinking from an attacker's perspective).

You could tell at interview too - the folks with a Computer Science background and a side interest in security were much, much better than those who took the dedicated-cyber-security degree/masters route.

You absolutely need a real generalist for security. With that said, I don't think it's unreasonable to expect a developer to know about CIDR notation, networking and cloud systems though we're perhaps straying into more DevOps-y style roles.

Sysadmin/ops has too many offramps that drain talent before year 10. If you can integrate software/systems well, manage projects or do advanced troubleshooting; you will likely be pulled out of ops. Conversely there are an ocean of security certifications being issued to people who have very little operational/technical experience.

Data security in practice is being reduced to a policy and procedure checklist. It is frustrating for an engineering group to receive non-specific or contradictory policy guidelines written by non-technical people, but I have yet to see that change hiring or decision making. Businesses want someone who will agree to check the box. If that someone doesn't know all the details, that makes checking the box easier.

The future of cybersecurity is not skilled coordinator/PM but instead yet another non-technical management arm handing down mandates that are blind to technical reality. There isn't another option. There aren't enough people to fulfill demand, and the compensation for cybersecurity positions are often less than a senior infrastructure role. How many sysadmins really understand networking, programming, databases, etc; While also having the people skills to not alienate both management and highly technical development and operations teams? We will never have enough people at the intersection of that many skills.

Plus nowadays more and more companies are going on cloud, so there are fewer sysadmins jobs anyway.

I'm someone who really wants to be in some admin jobs, be it DBA or sysadmin, problem is I first joined as a business analyst, and now I'm a DWH developer/Data engineer hybrid, every step towards a admin-ish job takes way too long and difficult for me :/

This points to two issues: education needs to be addressed with more input from industry, and expectations for hiring need to be realistic. 10 years before you're able to work on something security related is not realistic, nor is it sustainable.

In all seriousness, your point brings up the idea of where does the responsibility for this immensely difficult task (securing networks) fall? If we could spread out the "required" 15 years of experience into each of the developers, would that have the same effect? Building software with security baked in would reduce the need for so much work after the fact.

But there is also a fundamental disconnect between what schools are teaching and what industry is hiring for. The answer right now is "Go to school for cybersec, get your certs, then work for X years as a low-level help desk agent or call-center phone jockey".

Industry needs to tell educational institutions what candidates get from being a password-resetter that isn't taught in school, and work with those institutions to get those skills into the curriculum.

I have a lot more to say on the topic of cybersecurity and hiring, but I'm getting into rant territory.

Edit to add: You mentioned 'spreading out the 15 years of required experience'. I firmly do not believe it takes anywhere near 15 years of experience to become competent at cybersec.

Once upon a time there were programmers.

Then there were systems programmers and application programmers. Systems programmers wrote operating systems and utilities for them. App programmers wrote apps. There was a lot of crossover.

Then there were operators, systems programmers and application programmers. Operator was a junior position who did physical things (mount tapes, plug in cables) and ran commands to do things on the systems. They usually moved up to being…

Systems administrators, who did some programming in service to the systems, but not too much. The more senior a sysadmin was, the more time they spent programming and the less time they spent doing physical things… unless they wanted to do that.

Sysadmins started to specialize. People who configured switches and routers and talked to telephone companies became “network engineers”. People who spent time working on firewalls and security policies and thinking about that became “security engineers”. Junior people who read scripts to end users became the helpdesk. And so forth.

Then we noticed that a bunch of people were doing things manually when they should be automated. This was especially bad in places where there were no senior sysadmins or systems programmers. But we did have the internet, and senior sysadmins got together and started writing tools to make their lives easier: infrastructure automation.

Remind me, which kind of engineer?

The idea that senior sysadmins are behind the push towards automation is amusing. The biggest shift in the field in the past two decades came from Google when they decided to solve the tension between developers and sysadmins by more or less firing their sysadmins and hiring engineers to do the job instead.

Many true things are amusing, including this one. I think you are operating with a remarkably narrow and historically ill-informed definition of "sysadmin", and your prophecies are self-fulfilling. If you have a tension between developers and sysadmins, it's a cultural problem.

But I do believe that for an entry level you don't need 15 years. Maybe 5 years of sysadmin or devops should be good enough.

The entire industry plays a game of:

- Create a checklist (or use an existing checklist - ex: FIPS)

- Check off all the boxes on the checklist (any way they can - however they can, with complete and utter disregard for the spirit of the checklist)

- Confirm with legal that checklist is complete

- Advertise that they are "secure" to customers who happen to care (not many do, honestly) and present them with the required completed checklists

- Get hacked LEFT AND RIGHT because the whole fucking game has nothing to do with security, and everything to do with liability.

- When they're hacked, whip out the checklist again and go "couldn't have been our fault! we followed the checklist."

Repeat.

----

Now - Software security is hard. Unfathomably hard to most people (as in - they literally don't understand). People STILL fail to realize that software security is not like building a bride - I see it still even here on HN, where folks spout off bullshit comparisons to things like restaurant health/safety inspections, or architectural reviews.

The difference is that the bridge is not constantly being assaulted by an intelligent, evolving, malicious, human force. The software usually is.

And the security team can't just win one battle - they have to win every battle. Whether that's old systems, or a tired employee clicking an email link.

So I think you're basically between a rock and a hard place as an honest security worker. The job is literally impossible - so the folks who make money are the ones who compromise fastest and check off the most checklists (again - spirit of the checklist be damned).

I think the ballooning insurance payments (and the obvious eventual halt to offering cybersecurity insurance) will eventually bring the whole house of cards down, but we're still a few years out from that.

Well, maybe more checklists and consultants.

It may not make more revenue but poor security certainly affects profits.

What we need is regulation regarding putting personal data at risk to provide a financial incentive for companies to take security seriously.

If you do an in-depth read of the PCI security standards, you’ll see that the standards are about protecting the card brands, not you.

Risk is free.

A risk-aware competitor faces a higher cost function and a market which won't support it.

What we need is regulation, and direct liability of corporations, stockholders, creditors, and executives.

"A stitch in time saves nine" is probably more relevant to security.

Fines would therefore be the obvious solution to the lack of cybersecurity. Network breach / data leak due to not patching software x days after vuln disclosure? Here's your fine!

I believe the real problem is effective security is hard, and most merely want to pretend than actually invest in doing it.

The reasoning was we probably don't use base64. I was amazed.

Why would they? Does capitalism incentivise "caring" on a technical and ethical level about doing the right thing, or does it incentivise spending the minimum amount of resources to be covered by insurance and not criminally liable for anything? If they did the "right thing", someone in management is wasting resources.

Of course, if your company is private and the shareholders are decent enough people to make sure the board are doing things properly, this can work. With public companies I don't see how it is remotely feasible?

We have to legislate to compel companies to do this and expand the definition of negligence, which itself is quite complex. Make the people at the very highest levels criminally liable for breaches that happen due to lax, box checking behaviour on their watch. It is the only way.

Personally, I am helped by being a client server. It constantly amazes me the kind of risk a business is willing to accept for barely anything in return, and if I actually had a personal stake in what I am seeing at clients I think I would be much more stressed.

Also, the issue is not just 'are security professionals ok'. Good security starts with good operations, and good operations is a rarity. We need devs to ship products that can run with least privileges and have secure defaults. We need operators to have a good understanding of their own environment and to design things on purpose rather than just improvising. We need security people that can offer more guidance than just printing out a nessus scan. We need business analysts who are pragmatic with concessions and who are willing to spend the resources needed to do things right the first time.

The landscape is desperate with hundreds of apps, servers, networks, employees etc. in most companies. The tooling is either difficult or expensive or both - how many people allow outgoing firewall traffic by default because it is too complicated to whitelist everything that needs to go out? Even with the best will in the world, basic things like Windows updates, SSL cert updates (no we can't all use Lets Encrypt), Linux updates can be a full-time role.

But to be fair, our industry is still quite immature. We do not have the regulatory backup across the globe to assure all products are developed securely etc. We use open-source libraries with barely any checks (and what would we check anyway?) and IT is mostly intangible so how do we even know what we are doing anyway?

This sounds doom and gloom but I think every industry has a Wild West stage that teaches people what is and isn't important and allows industries and products to mature eventually to something that is sustainable.

Yet, it still isn't; elsewhere someone mentioned doctors and by comparison a doctor must pass nationally recognised exams and afterwards take personal responsibility for what they do every day, hold malpractise insurance and risk being struck off medical registers and forbidden from practising again for e.g. incompetence, unethical behaviour, malpractise. What are the equivalent of these in cybersecurity?

Customers or users whose data is stolen in a hack, employees who lost their jobs to a ransomware attack shutting down a company, what equivalent of "medical malpractise" lawsuits can they bring against the infosec team that was employed and paid to keep them safe? Can it even be determined by the harmed people whether the infosec team acted unethically or incompetently vs. sensibly? What reassurance can they take when the infosec employee says "I told them I shouldn't be opening this connection and they made me, so it's not my fault"? Imagine a surgeon performing a surgery they disagree with on a patient because non-medically qualified hospital management told them to. What happens to the security teams who incompetently don't protect against those harms and they carry on to their next job without any obligation to disclose their association? What board of reasonably trusted people is overseeing them, holding them to account, what register can they be struck off? It's "mistakes were made" all the way up to the CEO.

I don't agree that it's the infosec employee's fault. (Insert popular civil engineering/bridge design comparison here, and the need for a competent qualified senior engineer to sign their name against a design which they can personally be held responsible for).

My organization takes security seriously but and the end of the day we serve customers who don’t. That’s been the bulk of our issues this year.

log4shell has just been the icing on the cake.

That's the rhetoric from young people with no strings attached. People who are older, often with families, always with local connections and long-term relationships, have far more variables when it comes to making that sort of decision.

Pay and working environment (unless either are truly terrible) are rarely even at the top when it comes to enumerating the factors that go into deciding to stay or uproot your entire life and start again elsewhere. If you have a spouse with a good job in a niche area, doubly difficult.

EDIT: Unless you live in an area where you have many nearby options, of course. But those sorts of areas are sometimes harder to move around because the competition for 'good' employers is much fiercer.

- Remote work is now standard, you don't have to relocate

- US companies and well-funded startups opening up outside of US and hiring remote devs (though this is not always officially advertised, but for senior-enough people it's almost always negotiable)

- Tons of reports of people getting 30/50/100% raises (by changing jobs, or counter-offers)

Source: first-hand on all above (I'm frontend dev though, not in security)

Since the last two companies have been headquartered in different cities, this wouldn't have been possible before COVID.

The silver lining on this damned pandemic has been a lot more options for people, like software developers, in the privileged position of being in demand with work that can be done remotely.

1. Try to identify if you are vulnerable

2. Inform people in your company

3. Contact vendors of vulnerable products and ask for a patch

4. Have sysadmins install said patch

Cybersecurity people don't do anything. You're not patching. You're not finding the new flaws. You're reactively trying to solve problems when they make the news.

This is actually a bit of a fun exercise!

What do programmers even do?

1. Client tells them what they want solved.

2. Stack overflow has code examples for these problems.

3. IDE tells you what will compile, what wont, even highlights stuff for you (programmer just copy/pastes).

4. Compiler makes the program, and warns you if it doesn't work. If it doesn't work, go back to step 2.

Programmers don't do anything.

See how dumb that sounds when you skip over a bunch of the minutiae? Or when only looking at a single event? Or generalize to the point of nothingness?

Cybersecurity is SUPER broad though and there’s a range of many different roles. I’m a bit surprised all the folks here saying they’re not okay. Best of luck to them and maybe it’s time for a role swap.

I used to work in a SOC so I get it and probably would be struggling if I was still there.

Also somewhat related - what's the best path for a senior software developer to enter that space? Is the pay the same?

> what's the best path for a senior software developer to enter that space? Is the pay the same?

Well it depends what you’d wanna do. I work in application security which imo is the easiest transition into security for a senior dev.

Security experts who possess a strong programming background are quite rare. It’s been a huge advantage for me personally.

The most straightforward way into appsec might be to get your oscp. People will take you seriously if you’re a senior dev and have that cert. I hear it’s a beast but it can be done with a few months of serious effort.

As for pay, It’s hard to tell cause devs have a wide range. I make ~170k and live in Atlanta for reference after 4 yrs experience.

The roles that come my way are between 150k-225k. Obviously if you can land a FAANG role the compensation can be much greater.

Holy cow, that's much better than I figured. I make that much and I have 15 years of dev experience. I live near Portland, OR and work for a company based in LA. Portland's cost of living is much higher than Atlanta too so you're doing really well.

My work-life balance is excellent, so that was really my main concern, but sounds like yours is just about equal if not better. Thanks for your response, I think I'll have to seriously consider a transition now.

Even within appsec the roles can vary quite a bit…some folks just do sdlc security for example.

1. Not Budgeted for.. 2. To Much Down time (Excuse on systems even, that are fully load balanced).. 3.. WHHHHHHNNN But we need that legacy system (Which is real, because if it goes down the whole network does)... 4. This doesn't sound critical, let's bring this up next year. 5. Because I (Non-techy boss man) said we're not going to do it. As a matter of fact I am going to sue: HIPPPA, OR PCI, OR Some Reg agency for governmental overreach of a proper business.

Am I okay?

On the other hand my colleguaes who look for hackers, do forensics and help customer who are/have been attacked are having the time of their life. Rarely do you see people as excited about their works as these guys during this weekend.

I feel bad for anyone working in an org that doesn't have the ability to proactively find bugs in their stuff. Lots of places are cheap and don't account for the debt they accrue by not updating their systems. 'if it aint broke don't fix it' doesn't apply to software. What you are shipping is always broken. 'safe' languages are written in unsafe languages. The interpreter has bugs, the vm has bugs, the virtualization stack has bugs, the OS has bugs, the libraries to do everything have bugs. What is there and what is known are both moving targets. If you are not hiring the offensive minded individuals who will find the bugs with or without your support then you will not know about the bugs until they are out in the wild. If you aren't willing to pay those people you are accruing debt that will come due later.

There has been serious underspending by companies for cybersecurity for at least a decade now. Companies are slowly waking up to the fact that the security team can't be less than 1% the size of the development team.

Companies have let developers do whatever they want for so long, that when infosec comes in and says we need to change this so we have better visibility in to what is being used, or how, it's "Oh this will hurt productivity, so no".

The shit I have heard because companies don't want to spend money on cybersecurity, because putting out new features is more important than something that "might" happen. They just keep spending more on endpoint security and letting everything inside do whatever it wants.

and why would they? Bad hacks blow over after a year or two. Equifax is still ticking along, so is Citi bank, so is capital one. Nobody cares if you get hacked, just pay a fine and give it some time and things will go back to normal.

Saying bottleneck implies there's expectation of a better future ahead, but so far there's very few repercussions for neglecting this stuff and so it's unclear whether it'll improve or just become worse.

Large orgs should already have sufficient documentation as to which packages and versions are in use and what systems pulled them from their proxy repo.

Key word there: "should".

Let's say you have all your 50,000 applications well-documented. You think those docs are all going to be searchable in one place? That's an information disclosure vulnerability! No, all 50,000 applications documentation will be silo'd and only accessible to a select few people who work on them (you hope).

So now something like the log4j vulnerability crops up: You need to find out which systems are using log4j and what version. Best you can do is ask around... Demand that every application team cough up the details ASAP.

Now let's say you get data (emails) back suggesting that 5,000 applications are using log4j for certain, 1,000 may be using it (they're Java based apps), and you've confirmed that 14,000 most certainly are not using it. That leaves 30,000 applications where you have no idea if they're vulnerable.

You get data from the Artifactory ("proxy repo") team and they tell you, "we have 150,000 servers that have pulled down log4j (various versions)." Well, that's not particularly helpful so you get the raw data and try to correlate servers to applications only to find that's not helpful either: Because multiple "applications" could be using the same server and just because a log4j version was pulled doesn't mean it's actually being used by anything (in production).

After a few days of investigating the issue you find out that some thousands of applications actually are using log4j but it was included as part of a dependency. You tell them to update it.

Then you find out that 10,000 applications at present have no active development teams which explains why you got no response. Then there's an ungodly number of applications where no one has access to the source code anymore, 3rd party applications, etc.

So even if you have a central proxy repo and excellent documentation on all your stuff that doesn't mean it's going to be easy to hunt down and patch everything (that needs to be patched).

It's not an easy situation either way, but the identification portion should be trivial.

AFAIK the IT security folks at a financial institution that I used to work at are living in hell at the moment. That organization grew internationally through acquisitions across more than 80 countries, and lacks unified structure across the entire organization due to legal and compliance constraints unique to each jurisdiction.

A web development shop I provide security guidance to (owned by one my TTRPG buddies) had to do a couple of searches in github and update some hardware as patches became available. We made sure he was sorted out over a couple of drinks before we played board games.

It varies.

Nothing is trivial in large organizations.

I know I'll see half of y'all online with me :-/

Developers make products: they are indirect profit centers, and while everyone sees room for improvement, get treated relatively well.

Conversely, outside of areas like say finance, big tech, & gov, sec teams get starved and ignored as cost centers. Their event log DBs (SIEMs) are often from 15+ years ago and might even be SQL-based (think MySQL, not bigquery), if they even have one.

Not fun even before all this - automated attacks with bad support has been going on for years.

The likes of FAANG or banks, they have a big target painted on their backs; so they are scrambling for cover. However, the overwhelming majority of other businesses are not under similar pressure, because it's unlikely they will be targeted first - if at all.

I was actually talking about this with a friend who works for a company that provides a few niche services. They've had log4j 1.x in production for eons, which is also vulnerable to bad remote exploits, and nothing ever happened - simply because hackers are extremely unlikely to target their services. Obviously it doesn't mean they shouldn't upgrade, but the pressure is basically not there - at least until something Really Bad actually happens. He was actually pissed off at his manager making a big deal out of this exploit simply because it ended up on the mainstream press.

https://twitter.com/SwiftOnSecurity/status/14694518560910090...

https://twitter.com/SwiftOnSecurity/status/14703213781550202...

https://twitter.com/SwiftOnSecurity/status/14703215750145802...

https://twitter.com/SwiftOnSecurity/status/14704312736342179...

https://twitter.com/SwiftOnSecurity/status/14707958604349030...

FUD, bullshit, lack of skilled people, lack of budgets, lack of understanding from adjacent departments, chaos, mayhem, overtimes, incidents and creeping "I'm not sure what's going on" have always been parts of the profession. Learning to accept frustration, constant change, ill-formed perception and rejection is part of your career choice and a selection factor in the long term. Learning to look at the world from a certain angle which is hard to unlearn (especially if you're good at it) is a mental equivalent of firefighter's calluses.

If you can bear with it all - being on defensive side and being a kind of digital first responder (regardless of where exactly you are in the industry) is a fun job and calling for some.

(Edits: Typos)

Now, I write software for a niche Information Security team at a Fortune 250. I am still impacted by these major events, but it is usually in the form of "can we add X detection for Y vulnerability". The work is challenging/enjoyable but I am still able to maintain a good work/life balance.

Yet here we are, talking about burnout with more than a few people ITT _still_ talking about teams being understaffed.

Meanwhile there is no shortage of people obtaining credentials and education in cybersecurity who have nowhere to go because many parties feel these are effectively worthless. So what's the answer when the onus of training has shifted to workers, but none of the major players want to develop these people.

[1] https://www.csis.org/analysis/human-capital-crisis-cybersecu...

This has me thinking no one cares anymore about the stack of software being used - it’s almost sad to think about how it is going to be get worse with insane dependencies software comes with nowadays.

I’m thinking there’s a business opportunity for someone to curate list of software and it’s dependencies and provide subscription service for vulnerabilities alert on the stack of said software.

PS: Looks like Vuldb [0] is halfway there, but wonder if they alert based on dependencies or even version of said application / product.

[0] https://vuldb.com/?kb.alerting

At my previous job, everything fell on deaf ears and I wasn't given access to the things I needed to make sure things were secure. I couldn't access Jira board for the projects I was supposed to secure to create tickets for security remediations, and issue reports were ignored.

Where I'm at now is better. I can create the tickets, but they tend to get ignored until I tag the PM for them to get assigned, after which they typically get fixed pretty quickly.

Reminder that this exists: https://paulbellamy.com/vulnerability-name-generator

At some point we need to stop the coddling. Sink or swim people.

¯\_(ツ)_/¯

As an industry? We are getting better. Better recognition for neurodiversity, and the needs and capabilities for each other. Better at calling out the bad actors who can be poison pills in our teams, companies, and communities. Better at mentoring folks who are coming up in the industry.

None of that changes the disastrous toll that the security industry can take on individuals. Incident responders and investigators often have to deal with many of the psychological issues that police officers have to (exposure to CSAM, graphic imagery, deep knowledge of cybercrime and the relationship to real world crime, including human trafficking, drug trafficking, etc). The stress of often working in sensitive roles where you can't share or talk about things, and in the case of many colleagues (at least in Canada and the United States), having access to work approved "counselling services" as opposed to actual mental health professionals with a direct clinician-patient relationship.

Many IT Security folks work in teams that are under resourced, function in "Teams" of one, and are often expected to fulfill the roles of SRE, IT Support, Architecture, Engineering, and Compliance for any technologies or tools they use to do their jobs, in addition to actually doing their jobs.

It gets to be a bit much at times :)

> Have studies about the mental health and anxiety levels of this group of professionals been conducted?

I don't know if there are any formal, rigorous, academic studies, but mental health in the cybersecurity space has been a long discussion, with the earliest significant discussions I remember being back in 2008-2009 at some conferences.

Depending on the role and the organization it can be a high stress job. I often tell a story at talks and conferences about my 72 hour work day, which was my first incident response activity early in my career. It involved life critical and personal safety systems affecting thousands of people, and ended up working with the team for literally 3 days straight with no sleep because there was no one else in our team that had the perspective on how to address the issues. There was serious physical fallout, as well as mental issues (I developed aphasia for a short period of time, amongst other things) Many of the folks I know in the industry have similar (but not as grim as a full 72 hours), and that's setting aside the toll of building a career on telling people that there are big issues, and having to be helpful and constructive instead of saying "I told you so..." when things go poorly.

Yes. Risk is the primary part of the job, and the job is lucrative.

In other words, it's business as usual.

The major contributing factors for me:

- Reactive panic instead of proactive strategy

- - Detail: The suits in most economic sectors have zero interest in investing in security and best practice ahead of time, preferring to sell buzzwords to customers, and engaging in actual security only reactively. This leads to lots of scrambling and working weekends for cybersecurity professionals.

- - Example: can't tell you how many times I heard companies say "AWS handles our security." This is virtually never true, and the realization of the fallacy of this idea led to many panicked board meetings and embarrassing disclosures.

- Budget by buzzword

- - Detail: The latest cyber machine learning web 2.0 agile next gen thing will always get the funding, while tried and true practices like a good NIDS/HIPS pairing and a robust incident response process are relegated to the budget back seat

- - Example: Much of my time was for a government agency, and it was consistently easier to get funding for security hardware than for people. As a result, we often ended up with amazing, sexy, buzzword-covered hardware, and absolutely nobody (or nobody trained) to actually monitor it or analyze the data it produced.

- Ambulance-chasing parasites

- - Detail: Name a major cybersecurity incident that hit the public consciousness between 2015 and 2020, and I can name at least a dozen startups that cold called me over the next three months offering to address the problem for our organization, and prevent it the next time. These ambulance-chasing companies offer very little real value, and often drain resources from real improvements in an organization's cybersecurity posture.

- - Example: WannaCry/Eternal blue. I received months and months of aggressive marketing promising to fix this across our org, to the point that dodging the spam calls actually interfered with the on-the-ground work of patching endpoints and monitoring for IoCs.

This was my own experience, and may well not be representative of what others have gone through... but after years of 2am NOC calls, all-weekenders, being treated with disdain by other disciplines, and watching org funds consistently get flushed away on snake oil, I threw in the towel and switched industries. I took two decades of IT experience with me when I left. Now I'm paid better, and I'm in a much better place health-wise.

If only we had sanitized our inputs.

We sanitized all the form parameters! Did you sanitize the user agent string? What.

This is one of the reasons why I avoid log4j and log4net like a real weirdo. People use a logging framework because they are supposed to--so we can send logs to something sensible instead of paying a billion dollars to suck files from an ec2 instance into Splunk. We use log4j and still spend a GDP on splunk sucking in files.

99% of these people running around with their hair on fire could have just wrote to a log file with three lines of code. Instead they mvn a whole fucking library for no reason.

Well, they were supposed to. It's the right thing to do, according to some blog posts.

The fact that you think there is "no reason" to address the issue of logging through a method other than writing three additional lines of code shows that you really have no comprehension of how or why logging frameworks provide benefits (even if some of the inclusions like JNDI are ludicrous and should been defaulted off, or even behind a build flag, with big red warning labels).

I have yet to stumble on an app that uses any logging features other than writing to a file. And yet the default is to mvn log4j to do that because they don't want to look foolish doing it the simple way or "they might need it."

The JNDI string expansion and lookups was a pants on head stupid feature, but having highly configurable logging is super helpful when building and debugging systems at scale, both when you building and testing, and when you are scaling up so that you can granularly tune what is logging and where. This is true whether it's written in python, golang, rust, c, or any other language. If you don't know that, then perhaps you should try branching out in your experience.

Logging to a text file is nice for a toy app. For large-scale Production apps, you're gonna need log rotation to not overwhelm your disk space, transport to a centralized service for aggregation and bulk querying, and a good mechanism to append some tags about the request, service, instance etc to the log data to make it easy to query. That's not so easy without a nice logging framework.

My stock answer was "yeah, well, it depends on the day: On a good day, you're saying the same thing to new people. On a bad day, you're saying the same thing to the same people."

Those repeat conversations were often after an incident. The <insert fairly senior to C suite executive title here> would ask "how could this have happened" and I would say something along the lines of "<sir/ma'am/Alice/Bob, as appropriate>, do you remember my presentation and report of about 6 months ago when I mentioned that X-Y-Z required remediation and both greater control depth and greater control strength? Q hit Y, which hadn't been remediated, and that's why we're here...".

IF you can report to a interested and motivated executive with pull and credibility and IF the organization has risk governance and understands its risk tolerance and IF people understand the differences in various degrees of injury and IF people understand the differences between mission-critical and mandate-vital OR people are at least willing to learn what these things are AND willing to put in place the structures that support analysis, remediation, continually, forever, THEN, yes, it can be very satisfying.

(I taught for CSE for a few years, from beginner-level intros to in-depth multi-day courses for SMEs. I almost always made sure to include in every deck that one slide that showed the relationship between law, enabling legislation, central policy, departmental policy, governance, and risk tolerance and management. I'd say something along the lines of "if your department/agency/organization/unit does NOT have this, then the first document you should prepare is your CV....)

As to me, I'm doing OK, because I joined a cyber startup 1.5 years ago. Our hardware product is a unidirectional gateway (think diode with extra management capabilities on either side), our software product is a wicked cool (if I do say so myself) IRM/GRC/ComplianceManagement engine with integrated visualization and workflow, so I get to do really cool stuff all the time. And fortunately, we aren't subject to this one, so no one lost their weekend. Well, not because of this.

But only OK, because a) plague, b) some of the lovely bonus situations that come with age (related to both one's health and the health of one's loved ones), and c) startup, with all that implies, especially with A and B. Fortunately, I have great colleagues and a fantastic boss, so that makes it easier, but, still, it's a lot.

Heterogeneous tech stacks have benefits and drawbacks and we're seeing the drawbacks play our in real-time as increased risk.

First, we can work remote. Compare that to a fast food worker who has to risk exposure to feed their family.

Second, if you spend your money right you can say f it and leave any job that's too rough.

I was actually thinking of taking 6 months off, until I realized travel is still restricted

As a remote worker I can be at home and present with my family, with short breaks for actual activity, and longer periods for active response. This is not speculation - I have been active on incident response in the last month while helping my kids with homework, side by side at my home office desk).

We have a ton of software written in unsafe languages (C and C++). Our operating systems, web browsers, email readers, file editors, etc. Our governments and cyber-criminals have stock-piled 0-day exploits against these unsafe systems. They have hundreds or maybe thousands of these exploits.

On top of that businesses want features added to user-facing apps as fast as possible (beat the competition). These apps sit atop the unsafe C operating systems and often times run as root or Administrator. These apps have many logic flaws. Governments and criminals have stocked-piled 0 days against these flaws as well.

So until our systems and apps are re-written in safe languages (Go, Rust, C#. Java) and most processes run in an un-privileged context and mandatory access control (MAC) is applied to all running processes (all the time), we will continue to be hacked in ever-growing catastrophic ways.

IMO, this may be the explanation to the Fermi paradox. The aliens did themselves in with insecure software.

That's one of the favorite stances to take here on HN, and is driving the adoption of .NET, Go, Rust, etc (as far as I, a mere Pascal programmer, can tell)

It is not, however the root cause of our cybersecurity woes. It's ignoring the principle of least privilege at all levels, especially that of the OS kernel.

Instead of trying to rewrite everything in a "perfect" language, it would be far wiser to redirect some of this effort towards adopting a microkernel/capability based OS such as Genode, Fuchsia, etc, there's a handy list at

https://en.wikipedia.org/wiki/Capability-based_operating_sys...