I think there's 2 opposing forces that will affect "us" (rich country engineers) over the coming decade.
1) An absolutely insatiable demand for software as _everything_ needs to be remote friendly and digital.
2) An equally insatiable demand to offshore engineering talent for a fraction of the cost.
I'm not sure which will have a greater effect. Technically, if the demand for software grows faster than the world's ability to educate developers, salaries might actually grow. However, there's a real chance the gold rush of super high California $300k+ software salaries is coming to an end.
Personally, I'm seeing moving into management as more attractive everytime I see an article posted here about remote work. People here are understandably dismissive of this, but it's happened before in a lot of industries.
Rationally, it's hard to see why one branch of engineering--especially in a few locations in one country today--should generally have significant compensation premiums relative to other branches of engineering. There are a lot of dynamics involved but even just talking US universities, a lot of would-be engineering majors don't have strong preferences for one specific branch of engineering. And if there's a big salary gap...
Of course, there are legit differences in demand for all sort of reasons but it's hard to imagine some equalization of demand and compensation not happening. Especially given that for programming, as opposed to computer science more broadly, the barriers of entry are lower than for engineering more broadly.
I mean, trivially speaking, the reason you might see a continued mismatch is that supply is only one factor - demand is the other. Increased compensation is a factor of both the size of the mismatch in supply and demand, as well as the willingness-to-pay on the demand side. Willingness-to-pay is largely a function of the returns on that kind of labor, which are much greater for software engineering than they are for other engineering disciplines, and those returns are increasing over time, not decreasing.
And, like... we _do_ see that the growth in CS degrees granted (broadly speaking - the most granular data I could find is "Computer and information sciences") does slightly outpace other the growth in other Engineering degrees: https://nces.ed.gov/programs/digest/d20/tables/dt20_322.10.a...
CS/CIS went from 43,066 in 2010-11 to 88,633 in 2018-19, while Engineering went from 76,356 to 126,687 over the same time period. ~106% increase vs ~66% increase. Of course, degrees granted are a lagging indicator; if you graduated in 18-19 you didn't start later than 14-15. It's possible that the rate of increased enrollment in CS has continued to itself increase, since compensation really only obviously went on a tear a couple years after that. Nonetheless I'm very bullish on the 5-10 year outlook for SWE compensation, to the extent that it's reasonable to make predictions that far out in the future.
A: Two thoughts. First, you can leave a foreign country behind in an orderly way. Ask the British Empire, which mostly ended its colonial occupations in an orderly way.
The interviewee says this as though it's common sense, but is it true? The examples that stick out to me are India, Israel/Palestine, and the South African colonies. Can these really be described as safe and orderly withdrawals? What are the examples of successful withdrawals? Are they really that much better than US withdrawal from Vietnam or Afghanistan?
> What are the examples of successful withdrawals?
Canada, Australia, New Zealand, and the Caribbean territories. Arguably Hong Kong.
You could also make a reasonable case for Egypt, Ghana, India/Pakistan... The withdrawal itself was peaceful and didn't leave the state in immediate civil war, even if it wasn't stable in the medium term.
I’d say the difference is that those territories were more or less “at peace”. The colonists had “won” the war, at least per some definition of winning.
Afghanistan was a barely held together mess even before we withdrew.
I’m not arguing one way or the other for Taiwan—I don’t want to go to war with China for all sorts of obvious reasons—but we would have the backing of the Taiwanese people, and that’s a major difference versus Afghanistan.
You don't leave the country when you have tens of thousands of military troops still on land with major military equipment stationed there. The US never left Japan and certainly never left Germany.
Given what it costs to station troops outside the country (including an aircraft carrier with its air wing), there has to be a strong reason to keep them there for 70+ years. And it's not because we needed to maintain an occupying force [0] - it was to have bases with pre-positioned equipment near likely foes where troops could be quickly sent and move-out.
The REFORGER exercises [1] went on for nearly 30 years, testing the ability to rapidly move troops to Germany in the event of a Soviet attack. Not only was this expensive in monetary terms, it was expensive in human lives - each year people died from various causes associated with being around heavy equipment. Such as sleeping under vehicles that would roll over them in the night. Or crossing rail lines with their antenna still up and getting electrocuted.
The US didn't do this for any imperial reasons. It was to keep commitments made to those governments in the post-war years.
[0] Disclaimers: Dad crossed into Germany at the Remagen bridge before it fell, and I was stationed in Germany during the Cold War.
I'm not even claiming that. The US has clear strategic reasons to be positioned in many places around the world (like at Okinawa, for example). And Japanese people, despite wanting these troops out, are not going to see them leave anytime soon. So Japan has about no say in it.
> The US never left Japan and certainly never left Germany.
It left the the running of their governments. The US military is/was not needed to keep order and prevent a collapse of civil society in either of those two countries. Contra Afghanistan.
>>India/Pakistan... The withdrawal itself was peaceful and didn't leave the state in immediate civil war, even if it wasn't stable in the medium term.
The partition of India was a very bloody event in History. The mess left behind by British, doesn't just include partition, it also includes the Kashmir problem over which wars have happened till date.
Not to mention the horrible economic and social conditions that British left behind in the then Indian subcontinent.
Egypt? The British left in June 1956 and immediately tried to invade their way back in, October 1956. There is a reason that for a generation, "Suez" was a trigger for humiliation.
India/Pakistan? Do you mean the event that caused the death of somewhere between 200,000 and 2 million people? And the displacement of another 10-20 million people? That was much, much more catastrophic than what just happened in Afghanistan.
Ghana did have a somewhat smoother path to independence than either of those.
Canada, Australia and New Zealand were always a separate category: settler colonies where the new arrivals almost completely displaced the natives. Ireland shows the difficulty when you try and send settlers but don't genocide the natives first.
The dissolution of the Soviet Union and Russia leaving the former soviet constituent countries in Europe (e.g. Ukraine) and Central Asia (e.g. Kazakistan) turned out OK in the end, tension didn't flare up until this decade.
That's a non-sequitur. Did the British leave Canada & Australia in a peaceful state? As far as I know, there were no civil wars or disarray in those countries, whether ex British subjects or original stock of native populations.
That may be so. But was it governable when they left?
Whether or not we wiped out the masterminds of 9/11 or not, when we left Afghanistan, despite all the military activity up and till August 31, we left them in disarray. Most of the blame goes to the utterly corrupt and despised Ghani government but it’s quite clear we left it ungovernable.
I think that's an altogether different question. The original question was if left governable? I think the answer is yes.
Did The Romans leave England governable when they pulled out? I think yes. Did the Romans do shit, sure, but that's not the question.
Pullout from Afghanistan regardless of military actions and who was killed was left Ungovernable. Australia it appears to me was left governable when they left.
The British left Jamaica probably with few Tainos, but the question isn't were Tainos treated fairly, was Jamaica governable by the new government when they left?
> Did The Romans leave England governable when they pulled out? I think yes.
Would knowing that pretty much all trace of Roman rule, down to the material cultures and currency, completely vanished within a generation change your mind? Of all of the the Roman Empire, England saw the most severe, and most abrupt, fall in living conditions. And this is centuries before the Vikings wreaked yet more havoc on the region.
And, for what it's worth, while we don't know that much about what the remnant Breton polities looked like, the Anglo-Saxon petty kingdoms that replaced them were a variety of small kingdoms that constantly jostled each other for power, until the Vikings destroyed several of them outright and the Wessex King Albert conquered the rest. That's pretty damn close to the notion of an array of warlords jostling for power that you seem to be categorizing as ungovernable.
I agree with you that this looks like it has diverged into two separate questions.
But I think it is the view from the majority compared to the minority.
The majority see things as peaceful, quiet, fine. After the original slaughter, it was a peaceful federation of states (at a time where killing a native was 50/50 to come with any consequences) and no large scale civil wars.
The minority - the original inhabitants of this land - the war never stopped. Just changed in nature. Instead of outright killing, it was disenfranchisement/segregation, then blatant racism, now subtle/systemic racism. There is no peace.
And that can carry through generations. As the recent riots in the US show - which had similar sentiment here in Australia - even the killing on basis of race hasn't really stopped 100%.
Ask the minority, if the withdrawal was peaceful? YeahNah...
Where are you picking that up? That's your own making. nowhere were such things being thrown. The original question was was the place left governable, nothing more. You are giving the question with your own meaning.
Did the Soviet Union leave Cuba governable when they pulled out? Yes. The question isn't did the Soviets allow the Cubans to jail and kill dissenters. That would be a totally different question.
* India: the Partition led to massive loss of life, displacement in the tens of millions, traumatized communities, and deadly polarization between peoples to this day.
* Palestine: the end of the Mandate and the 1947-1949 Palestine war saw largescale displacement, loss of life, and (as is obvious to everyone) once again a deadly and unshakeable polarization between peoples that persists to this day.
On the one hand, it is certainly true that the British did not intend for these consequences, but on the other, the British were fully in charge as the sovereign government of India and Mandatory Palestine (unlike the US in Vietnam or even Afghanistan) - in particular having ruled India for over three centuries by the time they left - and so more deserving of blame.
The intention was there - according to Pakenham's "The Scramble for Africa" the idea was that prior to independence you educate a generation of civil servants who will then run the country along essentially British lines. I'd argue this worked well in India, largely though because Indians were already running the country in most important particulars. In Africa less so because independence movements gained unstoppable momentum which preempted such planning, plus many of those countries had been colonies for decades rather than centuries.
> civil servants who will then run the country along essentially British lines
While this may mean structural continuity, I can’t help but feel this is also a bad outcome - it’s just a continuation of imperialist force/cultural projection.
I won't say that a working electoral / parliamentary / judicial system, civil engineering, medicine, etc left intact is a bad outcome. It's a much better outcome than a civil war.
With some exceptions, there were no comparable systems in native cultures; those that existed (some definitely did) often were not capable to keep together a country, even if they worked well on tribal level, or were scalable but belligerent by design.
I also don't think that cultural projection, to a degree, is not a bad thing. It happens constantly, and allows for cultural cross-pollination. It is, of course, best done in peaceful ways.
> the idea was that prior to independence you educate a generation of civil servants who will then run the country along essentially British lines. I'd argue this worked well in India…
I certainly wouldn’t draw that conclusion. Immediately after the British left Nehru adopted 5 year plans based on the soviet model. Nehru and Stalin didn’t get along because he was still too Britain-friendly for Stalin, but India and The USSR developed very close relations within a very short period of time.
Some elements of India’s governance structure were distinctively non-Soviet, like having a multiparty democracy. But if you look at the way the country was run, it was done in the style of soviet central planning, most certainly not along British lines.
I don't want to speak well of the USSR/Russia but it's not hard to make an argument that they left Eastern Europe in an orderly way... well at the end. Honestly between Hungary, Czechoslovakia, Poland, and others there were some bad times in the there.
Worth noting that there was no Russian presence in Romania at the fall of communism back in '89. Russian military withdrew peacefully starting in '58 through early 60s.
Would you add a similar disclaimer before pointing something positive about the U.S. because they’ve done some bad shit too, like, I dunno, being the only nation to actually nuke another?
Pedantically, I don't think the US was ever part of the UK. The UK ultimately had sovereignty, but that's not the same thing. Even now the United Kingdom's former colonies are not part of the UK: it "has sovereignty over 17 territories which do not form part of the United Kingdom itself: 14 British Overseas Territories and three Crown dependencies.¹
But that's kind of the same point that I didn't articulate. A colony is not part of the country that is ruling it. The Baltic states were part of the Russian Empire, independent between the wars, and then reincorporated into the USSR. They weren't "militarily occupied", they were part fully incorporated. When someone says "militarily occupied Eastern Europe", I'd only assume they meant the countries that were "behind the iron curtain" but de jure independent, like Poland or Romania.
In that case, a better example would be Ireland, which was incorporated fully into the UK with the Act of Union, and was colonised. Another example would be Algeria, which was a French department.
Technically the UK (and Spain and France) conquered and colonized several native American nations, almost wiping them out. The US didn't even exist as a separate nation state until 1776.
The Soviet withdrawal was a bit different since the Union was actively falling apart and the occupied territories were pushing for independence. It wasn't all that smooth either - Georgian civil war and the first Nagorno-Karabakh War directly resulted from USSR's dissolution.
Those are settler colonies, a completely different kettle of fish. From the point of view of the (majority) settler population, they gradually ceased to consider themselves as British and underwent a multi-step disentanglement of legal and constitutional connections to the UK beginning almost a century ago but still ongoing. From the point of the (minority) indigenous populations, well I'm sure it varies but they may not agree that a withdrawal has occurred...
It still strikes me as a bit weird that Canada did not have its own constitution, as in technically completely independent of the British Parliament, until 1982.
And technically the head of state is…well it’s complicated.
Yeah, similar stories and timeline in other former dominions. I find https://en.wikipedia.org/wiki/Sue_v_Hill fascinating, when a court in Australia was forced to consider whether the UK was a foreign power.
Yeah this statement kinda discredits everything he says. The world is littered with countries that exist only because some existing adjacent kingdoms happened to both be colonized by the British, and they merged them into one colony. Countries fell apart after they left because there was no shared national identity to begin with.
The withdrawl from India was safe (for the British, let's ignore the partition bloodshed) and didn't lead to immediate regime collapse... so I certainly don't think it went as badly as Afghanistan or Vietnam.
No let’s not ignore the loss of over 1 million lives. Borders are still contested, with several wars having been fought, the scale of the disaster dwarfs Vietnam.
I personally don't think those withdrawals are truly orderly. Let's take India - leaving aside the partition of India (which is a huge matter in itself), a big crime is that the country that was left behind by the British did not resemble its historical culture or way of governance. It was instead turned into a secular nation, which just means defaulting to a status quo of colonial Western values. Ironically the partitioned areas (like the Islamic Republic of Pakistan) were not secular, but Hindus no longer had a land to practice a life that was fully dictated by their principles and their principles alone - instead they would live by rules and structures the British left behind. India also retained a lot of colonial structures outside of its government. For example the British education system continued, meaning that Indian children were taught out of Western-controlled/sourced materials that glossed over the horrors of British rule. Those children were also not taught the cultural things they would have learned in a pre-colonial era. This pattern isn't unique to India, but I wanted to call it out as an example of how seemingly peaceful withdrawals still result in long-lasting damage to a people and their culture. Is that peace? Maybe in the most literal sense. But it's not the same as making that place whole.
It's no coincidence that the following events seem to coincide
* Distrust of traditional media
* Collapse of local media
* Rise in social media platforms
Traditional media isn't competing with rival newspapers on a stand anymore. It's competing on platforms like facebook, twitter, and reddit. Getting clicks on these platforms is correlated with what drives engagement - usually some cultural outrage or some "out-group mocking". Newspapers are really just catching up to the psychology hacking that social media has already profited from. It's more economics than ideology that enforces this imo.
Frankly, you see this shift on HN as well. Half the time I log on here, at least 2/10 of the top articles are appeals to whatever cultural outrage triggers this site's demographics (let's be honest, this article does that too), and these articles will always have 10x the comments of other on the front page.
I guess your American? If so, please consider that you are not just one of the voices putting US at the lowest trust in the media, but currently also arguing that it would be better if America was literally at the bottom of the list.
The internet has opened pandora's box of mass psychological hacking.
The only way from here is to wait and see whether we can survive like this indefinitely - maybe we'll all collectively build a resistance to media constructed to hijack our emotions. In the meantime we will each have to one by one watch our friends and relatives dive down rabbit holes of propaganda and rage. I've seen this firsthand over the past year and it's not pretty.
I wouldn't hold my breath. This stuff operates on an emotional level, not a cognitive/intellectual level. Emotions are powerful because they are largely opaque to most people most of the time and are a strong driver of behavior. Like the old saying: you can't reason a person out of a position or belief that they didn't reason themselves into. As long as this emotional manipulation is profitable (e.g., 'engagement'), I don't see a way out of this.
USA is not the world. There are other countries that have social media as well as struggling local media yet distrust in media is not skyrocketing.
I can’t help but speculate that the extremely polarized and partisan politics of the US has some thing to do with this. I know it gets old connecting everything to Trump, but he did hold the highest office in the country for four years and during that period systematically claimed that every bit of media coverage he didn’t like was false and fake news, while also helping to spread obviously fake conspiracy theories that still seem to be popular with half the population.
I'm a little suprised they're still around tbh. Their competitor in New York, Knotel, straight up went bankrupt and got sold for parts. They must've had a lot more cash to endure this.
Selling short term leases on your own long term leases can probably work in places with expensive office space, but covid in NYC just completely annihilated this. The market rate for their office space now has got to be less than what they owe on their lease.
It's probably been great for the businesses who only had short term leases through them when WFH started though.
Idempotency is a pretty critical concept in system design, and I think most developers have run into issues related to it even if they aren't directly familiar with the term.
To give another simple example as the OP - Suppose you have a product that relies on time series data. For demo purposes you might create a curated data set to present to clients, but the presenter doesn't want to show data from 2019 as the "most recent"
Naturally, you decide to write a script. Do you
A) Write as script that moves the data forward by 1 week explicitly, and simply run this once per week or
B) Write a script that compares the current date to the data and moves it forward as much as it needs
At first glance, these two approaches work the same, but what if (A) triggers twice? What if it runs once every 6 days by mistake? (B) is idempotent however - subsequent executions won't change the state. It's usually impossible to predict all of the ways that software breaks, but designing with idempotency in mind eliminates a lot of them.
I don't think B is technically idempotent either. Change still occurs but with minimal difference. You cannot cache the results and use them again next week.
An idempotent change would be to pass in the current time instead of checking system time. In this case, as long as the input is the same, the result is the same. You could use cached results, but most likely you want to use new inputs.
Yes but this doesn't matter really. What's important is that running the script twice will still give you right values. From a practical point of vue it is the same as idempotency, which is what matters.
If you design for it from the start it makes your system much less complex. Consider all the errors, special cases, and ultimately data cleanup you need to handle about if your transactions are not idempotent. Idempotency is table stakes for any production app.
Speaking anecdotally, going back to school for an MSc allowed me to pivot from an engineering career in the transportation industry to the data/software field. It would've been more difficult to get interviews for entry level roles without the shiny new degree from a school that was - quite frankly - more prestigious than my undergrad, however I can't say for sure that it's benefited me materially beyond that initial "start". Experience is much more important once you're in the field you want.
I also continued working full-time while pursuing the degree half-time, which alleviated the risk of leaving a stable job. MSc degrees are much less likely to be fully funded than PhDs, and some companies will even reimburse your tuition as well.
You're asking this in good faith, so you don't deserve to get downvoted. I think people on hn are a bit sensitive to comments they perceive as "reductionist" that may oversimplify complex problems, even if they're honest questions.
But you're right, at it's core this kind of problem will use techniques like memoization, and most projects that need something like this will have adhoc approaches to solve this problem. The advantage of differential data flow is that it's a generalized approach to this problem. The business logic behind these workflows, between tracking dependencies and updates can get pretty damn complicated and difficult to maintain. Having a generalized approach would make building these dataflows much simpler.
I think I maybe a bit environmentally damaged from mainly using Clojure. Algorithms similar to this are fairly common in the Clojure ecosystem. Memoization is part of the standard library too.
The claim (in the article) that no one cares about Differential Dataflow seems to be only true when talking about this specific library. The general concept surely translates to some combination of simple concepts like memoization, topological sorting, partial application, etc. so it's obvious to me that many adhoc implementations would exist tailored to more specific needs in a different programming languages with different feature sets. Sometimes buying into a framework is a lot more work than rolling your own, especially if it means having to switch to a different programming language.
Importantly, this doesn't just use memoization (it actually avoids having to spend memory on that), but rather uses operators (nodes in the dataflow graph) that directly work with `(time, data, delta)` tuples. The `time` is a general lattice, so fairly flexible (e.g. for expressing loop nesting/recursive computations, but also for handling multiple input sources with their own timestamps), and the `delta` type is between a (potentially commutative) semigroup (don't be confused, they use addition as the group operation) and an abelian group. E.g. collections that are iteratively refined in loops often need an abelian `delta` type, while monoids (semigroup + explicit zero element) allow for efficient append-only computations [0].
To be fair, HN does remove a ton of political posts. If they didn't, this tech forum would be overrun with irrelevant political discussion.
Conservative think tanks posts can definitely stimulate good discussion, but if you don't moderate consistently, the community will eventually form a hivemind politically.
I agree with this and, for what it's worth, I've found discussions like this are always far more cordial in person versus online - even if you already know the person online.
I genuinely blame social media for this "descent into dogmatism". Any time my belief is challenged in an online discussion, I can just retreat into my echo chamber and feel even more confident than before.
I don't care about you trying to cancel me lol that's why this is a sock account, but it's even more telling that you think basic descriptive (not prescriptive) observations about reality are not moderate.
Yeah, but why not stuff like immigration, stances on environmental regulation, income tax rates on high earners, gun rights...
You've gotta agree that those fall under the moderate umbrella a bit more right? I've lived/worked in some very conservative environments and I don't think anyone would've walked along with the race IQ path.
My comment was in relation to the topic of the parent comment, which is why I mentioned it at all. Income tax rates and environmental regulation very rarely get brought up in conjunction with why there are gaps in racial outcome that aren't the result of racism. I think we can both agree that the things I mentioned are more more on point with arguments pertaining to racial outcomes, even if you completely disagree with those arguments.
That's fair! I think I may have confused you with the OP that responded as such to the moderate point.
For what it's worth, I think you may be surprised how much you can discuss on this topic without crossing "the line" for most people. I think that if partisanship wasn't as divisive as it currently is than you'd be able to discover a lot more common ground with people.
I definitely agree with you on that. Most people are far more likely to have a productive discussion when at least one party goes out of their way to be nuanced and careful in how they word things as well as the discussion not taking place in a performative environment (online or in front of other people.) It is definitely a lesson I have learned over time and through my own biases and misbehavior causing myself unneeded problems that were/are quiet foolish in hindsight.
With that said, all of these topics are not worth getting too into, since even 100% agreement on HBD or esoteric human rights philosophy topics doesn't really accomplish much. I have arrived at a sort of internal understanding that it doesn't matter if most people would disagree with me on this stuff. Also, if I find the common perspective to be ridiculous or stupid that doesn't really matter or entitle me to pressing the issue when it doesn't matter. What genuinely matters is finding common humanity/shared goals that are mutually beneficial and not getting bent out of shape over stuff that doesn't really matter, which in this case is trivia pertaining to generalized (which is practically useless individually) racial outcomes. Hopefully I've explained myself clearly and shown that I don't have any ill will intended towards you or anybody who has different opinions or may even hate or take offense at how I see the world. Have a nice Sunday :)
Of course, I hope that you didn't take too personally the snark in my original reply =).
At most other points in human history, our discussion probably would've been a cordial trade of livestock for bread and there would never be a reason to fear ill will. It's bummer of the 21st century that this time it was meta discussion about politics on the internet.
Don't worry, I don't take much personally and this was not one of those times. The little back and forth that may or may not be sharp is what makes these discussions fun (for me at least) in the first place. I think the added danger and possibility for severe personal consequences might end up being a net positive for everybody in the long run, as you can see in Eastern Europe where Stalinism and its restrictions on free speech resulted in a group of people who generally play things close to the chest by nature but have a very rich inner life and are able to contemplate serious or emotionally charged subjects in a way I find most Americans and first worlders in general lack.
New negatives in environments always come with new positives in the environment's survivors, and I am hopeful that we will all end up with a more thoughtful introspection similar to that which came from "oppression" (I hesitate to call what we are experiencing oppression due to how minor it is compared to gulags.) Some of the world's most beautiful literature as well as brilliant scientists have come out of Eastern Europe, I am optimistic that something similar will be the fruits of what is now underway globally with the advent of social media and its soft tyranny over social relations.
1) An absolutely insatiable demand for software as _everything_ needs to be remote friendly and digital.
2) An equally insatiable demand to offshore engineering talent for a fraction of the cost.
I'm not sure which will have a greater effect. Technically, if the demand for software grows faster than the world's ability to educate developers, salaries might actually grow. However, there's a real chance the gold rush of super high California $300k+ software salaries is coming to an end.
Personally, I'm seeing moving into management as more attractive everytime I see an article posted here about remote work. People here are understandably dismissive of this, but it's happened before in a lot of industries.